Disable HCP + slow Meteor build system (Isobuild, Gulp?)

help
1

Hi there

I’m working with a group on an OSS project. Our Meter app is one component among others (+ Chrome Extension, Electron app). We’re noticing slower + slower build times. The typical dev flow: Make a JS or SCSS change, wait a few seconds for HCP to kick in… switch to the browser, wait 20-30 seconds until our app is fully loaded.

This is making UI development slow and tedious. I dont have a new machine, but certainly not an unusable one, Core i5 Macbook Pro, and I’ve been resorting to doing as much code as I can in Codepens to try to move faster on the UI design side.

I’ve also tried to disable HCP, but node still kicks off under the hood to compile the templates(?) on every file save. This is not helpful for refactoring - often I’ll [want] to make lots of changes to many files, but every tiny change causes a CPU spike, a slow+unresponsive computer.

I read a little bit about the streaming build system in the Meteor 1.2 blog update (I cant post it here, 2 link limit for new users?) Glad to hear Meteor is tackling this problem.

I’ve heard about Gulp and its lightning fast build, but havent tried it.
Is Meteor going to incorporate these kinds of improvements? What can we expect?

Are there guidelines or documentation regarding how to improve/optimize build times - is it a linear thing, or are certain folders, modules, packages taking a disproportionately longer time than others?

Can we configure which parts of our app, maybe via a config file, that would be “hot”?
Would we only benefit from these updates if we’re using React (that update isnt clear)

In the short term - can I prevent node from watching files, unless I explicitly ask it to build?

I feel I’m missing something - how are folks who are running production apps maintaining a smooth workflow?

Thoughts appreciated
=: steven

2 Likes
2

Also struggling with this. I have a 13" macbook pro early 2015. My ‘Building the application’ step is taking ~25 seconds when I METEOR_PROFILE it and all I have is a tiny start on a meteor/react project with one jsx file. Making any change is really painful to wait for the refresh. Was on Angular/Grunt before with a larger app and refreshing on changes was near instant.

Really want to make this work, but it’s killing me.:

Here is a paste of the profiling:
| Rebuild App: 27146.6
| Isopack#getUnibuildAtArch: 0.9
| Isopack#_ensurePluginsInitialized: 1913.2
| JsImage#load: 1911.2
| files.exists: 361.1
| other JsImage#load: 1550.1
| other Isopack#_ensurePluginsInitialized: 2.0
| files.readdir: 2.5
| files.stat: 7.1
| files.realpath: 1.2
| files.readFile: 3.8
| bundler.readJsImage: 5.1
| JsImage.readFromDisk: 5.1
| files.readFile: 4.5
| other JsImage.readFromDisk: 0.6
| other bundler.readJsImage: 0.0
| JsImage#load: 41.0
| files.exists: 0.1
| other JsImage#load: 40.9
| bundler.bundle…makeClientTarget: 302.3
| Target#make: 302.1
| Isopack#getUnibuildAtArch: 2.9
| Unibuild#getResources: 63.3
| Isopack#getUnibuildAtArch: 1.6
| other Unibuild#getResources: 61.7
| ClientTarget#mergeCss: 216.3
| bundler.readJsImage: 10.2
| JsImage.readFromDisk: 10.2
| files.readFile: 9.5
| other JsImage.readFromDisk: 0.7
| other bundler.readJsImage: 0.0
| JsImage#load: 162.4
| files.exists: 0.9
| other JsImage#load: 161.4
| other ClientTarget#mergeCss: 43.7
| other Target#make: 19.6
| other bundler.bundle…makeClientTarget: 0.2
| bundler.bundle…makeServerTarget: 108.7
| Target#make: 108.5
| Isopack#getUnibuildAtArch: 3.1
| Unibuild#getResources: 99.2
| Isopack#getUnibuildAtArch: 2.3
| other Unibuild#getResources: 96.9
| other Target#make: 6.1
| other bundler.bundle…makeServerTarget: 0.2
| bundler…writeSiteArchive: 972.3
| files.rm_recursive: 0.4
| files.stat: 0.4
| files.mkdir: 0.2
| Builder#write: 320.3
| files.readFile: 0.2
| Builder#_ensureDirectory: 0.4
| files.mkdir: 0.2
| other Builder#_ensureDirectory: 0.2
| files.writeFile: 319.1
| other Builder#write: 0.5
| bundler…writeTargetToPath: 627.1
| files.rm_recursive: 0.4
| files.stat: 1.3
| files.mkdir: 130.0
| ClientTarget#write: 229.6
| Builder#reserve: 2.8
| files.mkdir: 0.6
| other Builder#reserve: 2.2
| Builder#writeToGeneratedFilename: 141.9
| Builder#generateFilename: 4.7
| Builder#_sanitize: 2.4
| Builder#reserve: 1.4
| other Builder#generateFilename: 0.9
| Builder#write: 136.1
| Builder#_ensureDirectory: 0.5
| files.writeFile: 133.4
| other Builder#write: 2.2
| other Builder#writeToGeneratedFilename: 1.0
| bundler…writeFile: 39.1
| Builder#write: 38.1
| Builder#_ensureDirectory: 1.0
| files.writeFile: 33.7
| other Builder#write: 3.4
| other bundler…writeFile: 0.9
| Builder#writeJson: 2.1
| Builder#_ensureDirectory: 0.0
| files.writeFile: 0.6
| other Builder#writeJson: 1.5
| other ClientTarget#write: 43.8
| Builder#complete: 0.9
| files.rename: 0.6
| other Builder#complete: 0.2
| ServerTarget#write: 263.6
| Builder#reserve: 0.0
| Builder#writeJson: 0.7
| Builder#_ensureDirectory: 0.0
| files.writeFile: 0.6
| other Builder#writeJson: 0.1
| Builder#write: 7.1
| files.readFile: 3.4
| Builder#_ensureDirectory: 0.1
| files.writeFile: 2.2
| files.symlink: 0.7
| other Builder#write: 0.9
| JsImage#write: 212.2
| Builder#reserve: 0.6
| Builder#generateFilename: 3.2
| Builder#_sanitize: 0.1
| Builder#reserve: 2.9
| files.mkdir: 2.5
| other Builder#reserve: 0.4
| other Builder#generateFilename: 0.2
| Builder#writeToGeneratedFilename: 107.3
| Builder#generateFilename: 14.8
| Builder#_sanitize: 6.1
| Builder#reserve: 6.4
| files.mkdir: 2.3
| other Builder#reserve: 4.1
| other Builder#generateFilename: 2.4
| Builder#write: 90.3
| Builder#_ensureDirectory: 8.6
| files.writeFile: 76.0
| other Builder#write: 5.7
| other Builder#writeToGeneratedFilename: 2.1
| Builder#copyDirectory: 10.1
| Builder#_ensureDirectory: 0.4
| files.symlink: 7.6
| other Builder#copyDirectory: 2.1
| Builder#writeJson: 1.6
| Builder#_ensureDirectory: 0.1
| files.writeFile: 0.8
| other Builder#writeJson: 0.7
| other JsImage#write: 89.4
| other ServerTarget#write: 43.5
| other bundler…writeTargetToPath: 1.4
| Builder#writeJson: 0.6
| Builder#_ensureDirectory: 0.0
| files.writeFile: 0.5
| other Builder#writeJson: 0.1
| Builder#complete: 22.0
| files.rename: 0.3
| files.rm_recursive: 21.6
| other Builder#complete: 0.1
| other bundler…writeSiteArchive: 1.3
| other Rebuild App: 23788.6
|
| other Rebuild App: 23788.6

3

Hmm… I had really long build times pre v0.9, which was partially solved by refactoring my app into feature oriented packages.

Things are still a little slower than they (c|sh)ould be. I’m looking forward to v1.2.

4

I just installed HEAD from github and the profiling output seems a bit more descriptive. Here is the part of the build that is taking a long time. Again I just have a tiny tiny meteor-react test project.

bundler…writeTargetToPath: 23528.4
| files.rm_recursive: 38.8
| files.stat: 364.9
| files.mkdir: 290.7
| ClientTarget#write: 22503.3
| files.mkdir: 1208.4
| Builder#write: 10704.8
| sha1: 9.5
| files.stat: 2010.8
| files.writeFile: 8679.1
| other Builder#write: 4.8
| bundler…writeFile: 10503.1
| Builder#write: 10502.4
| files.stat: 2471.8
| files.writeFile: 8024.9
| other Builder#write: 4.8
| other ClientTarget#write: 86.5

5

hi @travischoma, thanks for your input…

Interesting to hear you have a “tiny” project… this is more surprising, as I assumed build time is related to number of files.

I never heard of METEOR_PROFILE, and I guess this is exactly what I had in mind, in terms of profiling…
We’re using the latest stable , I think METEOR@1.1.0.2

How did you run/generate this report/stats? Is this a meteor thing, or a browser console thing?

@nathan_muir: We’ve started moving large chunks of app into packages as well, Telescope style … have you noticed diff in build times between code in vs out of packages?

=: s

6

that last profiling block I posted is nested in ‘bundler…writeSiteArchive’

7

@liveinchords profiling with meteor: METEOR_PROFILE=1 meteor

8

Thanks!
just generated our profile - analyzing it now
Will post shortly

=: s

9

I fixed my issue. The problem was meteor hacks:npm, somehow babelify had gotten into packages.json, while it was a dev dependency it should not have been in there, and it’s size was killing the build i guess.

10

Hi @travischoma
Sorry for the delay - we’ve been pushing forward on other things…

Thanks for posting how it turned out.

Mine is below. I’m not quite sure how to read this and take action on it - anything here jump out at you?

=> Started proxy.
=> Meteor 1.1.0.3 is available. Update this project with ‘meteor update’.
=> Started MongoDB.
|
| bundler.readJsImage: 24.6
| JsImage.readFromDisk: 24.6
| files.readFile: 22.7
| other JsImage.readFromDisk: 1.9
| other bundler.readJsImage: 0.0
| JsImage#load: 734.9
| files.exists: 1.6
| other JsImage#load: 733.4
| files.writeFile: 9.8
| Select Package Versions: 4748.5
| new CS.Input: 6.4
| Input#loadFromCatalog (sqlite): 3173.2
| check for previous versions in catalog: 0.4
| new CS.Solver: 54.6
| Solver#analyze: 54.3
| analyze allowed versions: 5.3
| analyze root dependencies: 0.5
| analyze reachability: 5.8
| analyze constraints: 41.4
| analyze pre-releases: 0.9
| other Solver#analyze: 0.5
| other new CS.Solver: 0.2
| Solver#getAnswer: 1512.7
| new Logic.Solver (MiniSat start-up): 17.4
| require root dependencies: 1.4
| generate package variables: 155.2
| generate dependency requirements: 85.5
| generate constraints: 160.4
| pre-solve: 699.0
| forbid packages with no matching versions: 0.2
| minimize unknown_packages: 1.4
| minimize conflicts: 144.5
| minimize unanticipated_prereleases: 49.5
| calculate previous_root distance costs: 8.5
| add terms to previous_root_incompat: 0.1
| minimize previous_root_incompat: 19.6
| calculate update version costs: 0.0
| minimize update_major: 0.0
| minimize update_minor: 0.0
| minimize update_patch: 0.0
| minimize update_rest: 0.0
| minimize previous_root_major: 1.3
| minimize previous_root_minor: 8.3
| minimize previous_root_patch: 7.4
| minimize previous_root_rest: 0.4
| calculate previous_indirect distance costs: 3.5
| minimize previous_indirect_incompat: 13.2
| minimize previous_indirect_major: 0.8
| minimize previous_indirect_minor: 5.1
| minimize previous_indirect_patch: 6.5
| minimize previous_indirect_rest: 0.6
| calculate new_root version costs: 0.0
| minimize new_root_major: 0.0
| minimize new_root_minor: 0.0
| minimize new_root_patch: 0.0
| minimize new_root_rest: 0.0
| lock down important versions: 3.6
| calculate new_indirect version costs: 0.6
| minimize new_indirect_major: 1.5
| minimize new_indirect_minor: 4.1
| minimize new_indirect_patch: 1.0
| minimize new_indirect_rest: 0.1
| minimize total_packages: 103.9
| generate version map: 1.5
| other Solver#getAnswer: 6.5
| other Select Package Versions: 1.1
|
| Input#loadFromCatalog (sqlite): 3173.2
| other JsImage#load: 733.4
| pre-solve: 699.0
| generate constraints: 160.4
| generate package variables: 155.2
| minimize conflicts: 144.5
| minimize total_packages: 103.9
| generate dependency requirements: 85.5
| minimize unanticipated_prereleases: 49.5
| analyze constraints: 41.4
| files.readFile: 22.7
| minimize previous_root_incompat: 19.6
| new Logic.Solver (MiniSat start-up): 17.4
| minimize previous_indirect_incompat: 13.2
| files.writeFile: 9.8
| calculate previous_root distance costs: 8.5
| minimize previous_root_minor: 8.3
| minimize previous_root_patch: 7.4
| other Solver#getAnswer: 6.5
| minimize previous_indirect_patch: 6.5
| new CS.Input: 6.4
| analyze reachability: 5.8
| analyze allowed versions: 5.3
| minimize previous_indirect_minor: 5.1
| minimize new_indirect_minor: 4.1
| lock down important versions: 3.6
| calculate previous_indirect distance costs: 3.5
| other JsImage.readFromDisk: 1.9
| files.exists: 1.6
| generate version map: 1.5
| minimize new_indirect_major: 1.5
| minimize unknown_packages: 1.4
| require root dependencies: 1.4
| minimize previous_root_major: 1.3
| other Select Package Versions: 1.1
| minimize new_indirect_patch: 1.0
| analyze pre-releases: 0.9
| minimize previous_indirect_major: 0.8
| calculate new_indirect version costs: 0.6
| minimize previous_indirect_rest: 0.6
| other Solver#analyze: 0.5
| analyze root dependencies: 0.5
| check for previous versions in catalog: 0.4
| minimize previous_root_rest: 0.4
| other new CS.Solver: 0.2
| forbid packages with no matching versions: 0.2
| add terms to previous_root_incompat: 0.1
| minimize new_indirect_rest: 0.1
| calculate update version costs: 0.0
| minimize update_major: 0.0
| other bundler.readJsImage: 0.0
| minimize new_root_major: 0.0
| minimize update_minor: 0.0
| minimize update_patch: 0.0
| minimize update_rest: 0.0
| minimize new_root_minor: 0.0
| minimize new_root_patch: 0.0
| minimize new_root_rest: 0.0
| calculate new_root version costs: 0.0
| Total: 5517.8
|
| files.readFile: 33.0
| bundler.readJsImage: 23.4
| JsImage.readFromDisk: 23.3
| files.readFile: 21.0
| other JsImage.readFromDisk: 2.3
| other bundler.readJsImage: 0.1
| JsImage#load: 363.3
| files.exists: 0.9
| other JsImage#load: 362.4
| files.exists: 368.8
| Rebuild App: 6881.0
| Isopack#getUnibuildAtArch: 2.4
| Isopack#_ensurePluginsInitialized: 2471.3
| JsImage#load: 2468.5
| files.exists: 469.3
| other JsImage#load: 1999.2
| other Isopack#_ensurePluginsInitialized: 2.7
| files.readdir: 16.5
| files.stat: 30.2
| files.realpath: 6.1
| files.readFile: 50.6
| bundler.readJsImage: 3.0
| JsImage.readFromDisk: 3.0
| files.readFile: 2.6
| other JsImage.readFromDisk: 0.4
| other bundler.readJsImage: 0.0
| JsImage#load: 23.2
| files.exists: 0.1
| other JsImage#load: 23.1
| bundler.bundle…makeClientTarget: 1907.0
| Target#make: 1906.6
| Isopack#getUnibuildAtArch: 9.8
| Unibuild#getResources: 1418.6
| Isopack#getUnibuildAtArch: 6.3
| other Unibuild#getResources: 1412.3
| ClientTarget#mergeCss: 427.6
| bundler.readJsImage: 9.1
| JsImage.readFromDisk: 9.1
| files.readFile: 7.8
| other JsImage.readFromDisk: 1.3
| other bundler.readJsImage: 0.0
| JsImage#load: 188.1
| files.exists: 0.8
| other JsImage#load: 187.3
| other ClientTarget#mergeCss: 230.4
| other Target#make: 50.6
| other bundler.bundle…makeClientTarget: 0.4
| bundler.bundle…makeServerTarget: 105.7
| Target#make: 105.4
| Isopack#getUnibuildAtArch: 9.4
| Unibuild#getResources: 79.4
| Isopack#getUnibuildAtArch: 4.8
| other Unibuild#getResources: 74.6
| other Target#make: 16.6
| other bundler.bundle…makeServerTarget: 0.3
| bundler…writeSiteArchive: 1786.9
| files.rm_recursive: 1.3
| files.stat: 9.4
| files.mkdir: 0.4
| Builder#write: 11.8
| files.readFile: 8.6
| Builder#_ensureDirectory: 0.5
| files.mkdir: 0.2
| other Builder#_ensureDirectory: 0.3
| files.writeFile: 2.1
| other Builder#write: 0.6
| bundler…writeTargetToPath: 1706.5
| files.rm_recursive: 1.0
| files.stat: 0.9
| files.mkdir: 0.5
| ClientTarget#write: 1371.4
| Builder#reserve: 10.6
| files.mkdir: 4.3
| other Builder#reserve: 6.3
| Builder#writeToGeneratedFilename: 61.8
| Builder#generateFilename: 8.0
| Builder#_sanitize: 4.4
| Builder#reserve: 2.0
| other Builder#generateFilename: 1.5
| Builder#write: 52.5
| Builder#_ensureDirectory: 1.0
| files.writeFile: 48.3
| other Builder#write: 3.2
| other Builder#writeToGeneratedFilename: 1.3
| bundler…writeFile: 152.2
| Builder#write: 150.0
| Builder#_ensureDirectory: 4.6
| files.writeFile: 137.4
| other Builder#write: 8.1
| other bundler…writeFile: 2.2
| Builder#writeJson: 5.5
| Builder#_ensureDirectory: 0.0
| files.writeFile: 0.6
| other Builder#writeJson: 4.8
| other ClientTarget#write: 1141.3
| Builder#complete: 1.3
| files.rename: 1.0
| other Builder#complete: 0.3
| ServerTarget#write: 329.5
| Builder#reserve: 0.0
| Builder#writeJson: 0.6
| Builder#_ensureDirectory: 0.0
| files.writeFile: 0.5
| other Builder#writeJson: 0.1
| Builder#write: 6.6
| files.readFile: 3.4
| Builder#_ensureDirectory: 0.1
| files.writeFile: 2.0
| files.symlink: 0.3
| other Builder#write: 0.8
| JsImage#write: 304.9
| Builder#reserve: 0.0
| Builder#generateFilename: 2.8
| Builder#_sanitize: 0.3
| Builder#reserve: 2.3
| files.mkdir: 1.7
| other Builder#reserve: 0.6
| other Builder#generateFilename: 0.2
| Builder#writeToGeneratedFilename: 140.8
| Builder#generateFilename: 28.8
| Builder#_sanitize: 13.0
| Builder#reserve: 11.5
| files.mkdir: 2.7
| other Builder#reserve: 8.7
| other Builder#generateFilename: 4.4
| Builder#write: 108.5
| Builder#_ensureDirectory: 2.5
| files.writeFile: 95.8
| other Builder#write: 10.1
| other Builder#writeToGeneratedFilename: 3.5
| Builder#copyDirectory: 3.1
| Builder#_ensureDirectory: 0.1
| files.symlink: 1.8
| other Builder#copyDirectory: 1.1
| Builder#writeJson: 2.5
| Builder#_ensureDirectory: 0.0
| files.writeFile: 0.5
| other Builder#writeJson: 2.0
| other JsImage#write: 155.6
| other ServerTarget#write: 17.4
| other bundler…writeTargetToPath: 2.0
| Builder#writeJson: 0.5
| Builder#_ensureDirectory: 0.0
| files.writeFile: 0.4
| other Builder#writeJson: 0.1
| Builder#complete: 55.0
| files.rename: 0.2
| files.rm_recursive: 54.7
| other Builder#complete: 0.0
| other bundler…writeSiteArchive: 2.0
| other Rebuild App: 478.1
|
| other JsImage#load: 2572.0
| other Unibuild#getResources: 1486.8
| other ClientTarget#write: 1141.3
| files.exists: 840.0
| other Rebuild App: 478.1
| files.writeFile: 287.6
| other ClientTarget#mergeCss: 230.4
| other JsImage#write: 155.6
| files.readFile: 127.0
| other Target#make: 67.2
| files.rm_recursive: 57.0
| files.stat: 40.4
| Isopack#getUnibuildAtArch: 32.8
| other Builder#write: 22.9
| Builder#_sanitize: 17.7
| other ServerTarget#write: 17.4
| files.readdir: 16.5
| other Builder#reserve: 15.7
| files.mkdir: 9.8
| Builder#_ensureDirectory: 8.4
| other Builder#writeJson: 7.0
| other Builder#generateFilename: 6.1
| files.realpath: 6.1
| other Builder#writeToGeneratedFilename: 4.8
| other JsImage.readFromDisk: 4.0
| other Isopack#_ensurePluginsInitialized: 2.7
| other bundler…writeFile: 2.2
| files.symlink: 2.1
| Builder#reserve: 2.1
| other bundler…writeTargetToPath: 2.0
| other bundler…writeSiteArchive: 2.0
| files.rename: 1.2
| other Builder#copyDirectory: 1.1
| other bundler.bundle…makeClientTarget: 0.4
| other Builder#complete: 0.3
| other bundler.bundle…makeServerTarget: 0.3
| other Builder#_ensureDirectory: 0.3
| other bundler.readJsImage: 0.1
| Total: 7669.5

Much appreciated,
=: s