I think 1.7 has an issue right now where it’s doing legacy browser builds during development. They’re adding a flag to turn that off.
If you’re not using 1.7, then I’m not sure.
1 Like
Yes, indeed workflow, crashes, changing apps (we work on 4 different ones), changing for PR-reviews, reseeding, setting changes, etc. I didn’t intend for my example to be taken literally but an illustration of the cost therein.
Yes, I saw that, I looked into 1.7.1 times still appeared to be the same, maybe I was missing a magical flag
Yes I believe you have to pass a flag into meteor run or it will still do the legacy build.
I really wonder how you all manage to get these amazing rebuild times? In all of my Meteor apps, every code change results in about 10-30 seconds of rebuilding (which is pretty frustrating compared to milliseconds in Webpack based apps). Only changes in SCSS files compile a bit faster. Maybe it’s because I’m using a lot of custom packages, which I do because I re-use them across apps? I wouldn’t mind at all about initial build times. Quick rebuild is what I am most looking for.
4 Likes
Follow tips here:
Also avoid precompilers (scss, less, sass, coffee, etc), instead if you must transpile them adhoc on your own.
2 Likes
When 1.7.1 is ready, you won’t need to use a flag. See the following for the full details (these changes were just merged around the middle of last week, so if you tried the release-1.7.1 branch before then, please give it another shot).
2 Likes
@hwillson yeah, I saw that branch, I even tried it on the todos app way before posting this. Didn’t seem to help or do anything, I was running it from a git checkout… I’ll check it again as soon as it’s out. I checked the release-1.7.1.
Still any thoughts on parallelizing the process or avoiding rebuilding on no changes (think switching apps and reusing ports, save mem, easier workflow)
Thanks for the follow up
@hwillson @robertlowe I just updated to the latest beta (18 maybe?) and my build times were night and day. They went from like 30+ seconds to 4 seconds. Were there other changes beyond delaying the legacy builds?
That’s the big change, but as @benjamn has been working on it, he’s been uncovering several parts of the older codebase that could be improved. He’s making a lot of tweaks to the build system that definitely help speed things up.
A few other notable performance improvements:
- Support lazy
inputFile.addJavaScriptfor substantial (re)build time savings - Cache minified CSS in
minifyCssFilesrather than relying on plugins - Cache
File#getPrelinkedOutputinlinker.js
In short, Meteor 1.7.1 is primarily focused on improving (re)build performance.
The irony is, at first I didn’t want to implement #10055 because it felt like cheating, so instead I found some other ways to (more than) halve rebuild performance first. After that, I realized it was possible to delay the legacy build automatically, without any developer intervention/configuration, which made me more comfortable with that strategy. I don’t love the extra complexity that it adds (e.g. #10077), but cutting rebuild times in half seems worth it.
13 Likes
We’re having a relatively slow refresh time, I think fast refresh time is the one of the the most important KPIs for developer satisfaction.
- We’ve a medium size react app
- A large number of NPM react components
- We’ve a package based architecture and each asset add it’s own files (images, css translation)
Our client refresh time is between 14 to 20 seconds, is this normal? Here is the output of the profiler, any insight is appreciated.
| (#4) Profiling: ProjectContext prepareProjectForBuild
|
| ProjectContext prepareProjectForBuild.........................9,451 ms (1)
| └─ _buildLocalPackages........................................9,299 ms (1)
| └─ _ensurePackageLoaded(special:cruni).....................9,254 ms (1)
| └─ IsopackCache Build local isopack.....................9,254 ms (1)
| ├─ compiler.compile(special:cruni).....................281 ms (1)
| │ └─ files.withCache..................................265 ms (3)
| │ └─ compileUnibuild (special:cruni)...............265 ms (3)
| │ └─ PackageSource#_findSources.................176 ms (3)
| │ └─ files.withCache 175 ms (3)
| └─ Isopack#saveToPath................................8,951 ms (1)
| ├─ Builder#write....................................239 ms (648)
| │ └─ files.writeFile 113 ms (648)
| ├─ Builder#copyNodeModulesDirectory...............6,306 ms (1)
| │ ├─ Builder#_ensureDirectory......................546 ms (1557)
| │ │ ├─ files.stat 306 ms (4674)
| │ │ ├─ files.mkdir 132 ms (1558)
| │ │ └─ other Builder#_ensureDirectory 108 ms
| │ ├─ optimistic readdir 129 ms (1557)
| │ ├─ optimistic lstatOrNull........................991 ms (8247)
| │ │ ├─ optimistic lstat...........................608 ms (8247)
| │ │ │ ├─ files.lstat 205 ms (8223)
| │ │ │ └─ other optimistic lstat 403 ms
| │ │ └─ other optimistic lstatOrNull 383 ms
| │ ├─ optimistic statOrNull.........................265 ms (6690)
| │ │ ├─ files.stat 118 ms (6663)
| │ │ └─ other optimistic statOrNull 147 ms
| │ ├─ optimistic readFile.........................2,384 ms (6690)
| │ │ ├─ files.readFile 2,164 ms (6690)
| │ │ └─ other optimistic readFile 220 ms
| │ ├─ files.writeFile 1,567 ms (6690)
| │ └─ other Builder#copyNodeModulesDirectory 423 ms
| └─ Builder#complete...............................2,291 ms (1)
| └─ files.renameDirAlmostAtomically.............2,291 ms (1)
| └─ files.rm_recursive 2,291 ms (1)
|
| Top leaves:
| files.rm_recursive.......................................2,291 ms (3)
| files.readFile...........................................2,185 ms (7094)
| files.writeFile..........................................1,681 ms (7344)
| files.stat.................................................507 ms (14111)
| other Builder#copyNodeModulesDirectory.....................423 ms (1)
| other optimistic lstat.....................................403 ms (8247)
| other optimistic lstatOrNull...............................383 ms (8247)
| other optimistic readFile..................................220 ms (6690)
| files.lstat................................................206 ms (8250)
| other optimistic statOrNull................................147 ms (6690)
| files.mkdir................................................144 ms (1755)
| files.readdir..............................................110 ms (2162)
| other Builder#_ensureDirectory.............................108 ms (1557)
|
| (#4) Total: 9,451 ms (ProjectContext prepareProjectForBuild)
|
| (#5) Profiling: Rebuild App
|
| Rebuild App...................................................6,593 ms (1)
| └─ files.withCache............................................6,593 ms (1)
| ├─ compiler.compile(the app).................................523 ms (1)
| │ └─ files.withCache........................................523 ms (3)
| │ └─ compileUnibuild (the app) 523 ms (3)
| ├─ bundler.bundle..makeClientTarget........................5,444 ms (1)
| │ └─ Target#make..........................................5,444 ms (1)
| │ ├─ Target#_runCompilerPlugins..........................472 ms (1)
| │ │ └─ plugin static-html...............................443 ms (1)
| │ │ └─ wrapped.fs.readFileSync 409 ms (625)
| │ ├─ Target#_emitResources.............................3,877 ms (1)
| │ │ ├─ PackageSourceBatch.computeJsOutputFilesMap.....1,852 ms (1)
| │ │ │ ├─ ImportScanner#scanImports for special:cruni.1,159 ms (1)
| │ │ │ │ ├─ Babel.compile 365 ms (154)
| │ │ │ │ └─ ImportScanner#_resolve.....................641 ms (1736)
| │ │ │ │ ├─ optimistic statOrNull...................440 ms (12581)
| │ │ │ │ │ ├─ files.stat 225 ms (2082)
| │ │ │ │ │ └─ other optimistic statOrNull 170 ms
| │ │ │ │ └─ other ImportScanner#_resolve 168 ms
| │ │ │ └─ ImportScanner#scanMissingModules for the app..480 ms (2)
| │ │ │ ├─ ImportScanner#_realPath 152 ms (3034)
| │ │ │ └─ other ImportScanner#scanMissingModules for the app 120 ms
| │ │ ├─ PackageSourceBatch#getResources................1,397 ms (89)
| │ │ │ └─ PackageSourceBatch#_linkJS..................1,395 ms (89)
| │ │ │ └─ linker.fullLink..........................1,332 ms (1)
| │ │ │ ├─ linker Module#getPrelinkedFiles.........640 ms (1)
| │ │ │ │ ├─ linker File#getPrelinkedOutput 254 ms (561)
| │ │ │ │ └─ other linker Module#getPrelinkedFiles 358 ms
| │ │ │ └─ linker Module#computeAssignedVariables..688 ms (1)
| │ │ │ └─ linker File#computeAssignedVariables 682 ms (561)
| │ │ └─ other Target#_emitResources 558 ms
| │ └─ Target#minifyJs 1,011 ms (1)
| └─ bundler writeTargetToPath.................................620 ms (1)
| └─ ClientTarget#write.....................................618 ms (1)
| ├─ bundler writeFile 157 ms (3614)
| └─ other ClientTarget#write 362 ms
|
| Top leaves:
| Target#minifyJs..........................................1,011 ms (1)
| linker File#computeAssignedVariables.......................682 ms (561)
| other Target#_emitResources................................558 ms (1)
| wrapped.fs.readFileSync....................................409 ms (625)
| Babel.compile..............................................371 ms (313)
| other ClientTarget#write...................................362 ms (1)
| other linker Module#getPrelinkedFiles......................358 ms (1)
| linker File#getPrelinkedOutput.............................254 ms (561)
| files.stat.................................................248 ms (2442)
| other optimistic statOrNull................................170 ms (12581)
| other ImportScanner#_resolve...............................168 ms (1736)
| sha512.....................................................132 ms (7010)
| other ImportScanner#scanMissingModules for the app.........120 ms (2)
|
| (#5) Total: 6,593 ms (Rebuild App)
|
=> Client modified -- refreshing
=> Finished delayed build of web.browser.legacy in 5334ms
Are you by chance not using an SSD or slower SSD, or very large assets? Maybe a lot of pre-compilers?
I’m using Macbook Pro, 16 GB ram (could the ram be the issue?), 3.1 Ghz Intel Core i7 with SSD storage. I did remove the assets and it saved I think 3 to 5 seconds.
I think I’ll need to spend sometime to understand how the code base impact the refresh time, but the issue became more noticeable in 1.7.0 due to compiling two bundles, so now we’re on 1.7.1 but it still not optimal.
But is this line normal?
Builder#copyNodeModulesDirectory…6,306 ms
And why is this step needed on refresh!?
Edit: Builder#copyNodeModulesDirectory looks faster with second refresh, 2 seconds…but the average time still greater then 15 seconds unfortunately.
===
Other tips: Ensure no assets are getting import'd in the wrong environment (ex aws-sdk on the client if you don’t need it there, etc). Look over your package-lock.json. Try hunting for big NPM modules checkout my shitty CLI https://github.com/RobertLowe/fatapp. Remove meteor-node-stubs (its a hog).
RAM is fine.
Edit: Typically I find the more IO meteor can see, the slower it gets, but that’s not the root issue as I’ve tested on machines w/ 3GB/s high IO NVMe drives (Samsung 970 Pro)
2 Likes
Thanks for the tips and nice CLI 
Here is the output from your tool
sanitize-html@1.18.2 size: 287 KB in: require failed.
draft-js@0.10.5 size: 293 KB in: require failed.
lodash@4.17.4 size: 303 KB in: require failed.
react-bootstrap@0.32.1 size: 323 KB in: require failed.
showdown@1.8.6 size: 335 KB in: require failed.
react-avatar@2.5.1 size: 414 KB in: require failed.
eslint@4.19.1 size: 477 KB in: require failed.
react-mentions@2.0.1 size: 493 KB in: require failed.
ajv@5.2.3 size: 497 KB in: require failed.
moment@2.22.1 size: 506 KB in: require failed.
react-dom@16.4.1 size: 536 KB in: require failed.
react-tag-input@5.0.1 size: 558 KB in: require failed.
survey-react@1.0.10 size: 565 KB in: require failed.
core-js@2.5.3 size: 573 KB in: require failed.
meteor-node-stubs@0.4.1 size: 750 KB in: require failed.
vt-pbf@3.1.0 size: 814 KB in: require failed.
react-spinkit@3.0.0 size: 1009 KB in: require failed.
react-icons@2.2.7 size: 1 MB in: require failed.
faker@4.1.0 size: 1 MB in: require failed.
watson-developer-cloud@3.0.7 size: 1 MB in: require failed.
recharts@1.0.0-beta.10 size: 1 MB in: require failed.
mapbox-gl@0.45.0 size: 3 MB in: require failed.
npm@5.6.0 size: 6 MB in: require failed.
I think I can get rid of mapbox-gl it’s taking 3 MB
Nice, keep in mind those are TAR.GZ sizes as fatapp only fetches http headers for sizes from NPM for performance.
I think the issue I’m facing is due to the way we’re using the package system. We develop within the packages so the react components are there and I think packages are not optimized for quick refresh the way the code core is. At least that’s my hypothesis for now, I’ll test it by moving the package code to the core import directory and see how the performance change.
Thanks for this tip. But tbh, this would take away pretty much of the “awesomeness” of Meteor, i.e. that everything is compiled “automagically”. I also do not really get why pre-compilation of SCSS should take so much time, since in Webpack it is done in an instant. And, as said before, pure SCSS compilation in fact works faster in my project. It’s the compilation of the source code that takes that long.
How did you profile this? I would be curious to see what causes the delays on my side.