Other tips: Ensure no assets are getting import'd in the wrong environment (ex aws-sdk on the client if you don’t need it there, etc). Look over your package-lock.json. Try hunting for big NPM modules checkout my shitty CLI https://github.com/RobertLowe/fatapp. Remove meteor-node-stubs (its a hog).
RAM is fine.
Edit: Typically I find the more IO meteor can see, the slower it gets, but that’s not the root issue as I’ve tested on machines w/ 3GB/s high IO NVMe drives (Samsung 970 Pro)
Thanks for the tips and nice CLI 
Here is the output from your tool
sanitize-html@1.18.2 size: 287 KB in: require failed.
draft-js@0.10.5 size: 293 KB in: require failed.
lodash@4.17.4 size: 303 KB in: require failed.
react-bootstrap@0.32.1 size: 323 KB in: require failed.
showdown@1.8.6 size: 335 KB in: require failed.
react-avatar@2.5.1 size: 414 KB in: require failed.
eslint@4.19.1 size: 477 KB in: require failed.
react-mentions@2.0.1 size: 493 KB in: require failed.
ajv@5.2.3 size: 497 KB in: require failed.
moment@2.22.1 size: 506 KB in: require failed.
react-dom@16.4.1 size: 536 KB in: require failed.
react-tag-input@5.0.1 size: 558 KB in: require failed.
survey-react@1.0.10 size: 565 KB in: require failed.
core-js@2.5.3 size: 573 KB in: require failed.
meteor-node-stubs@0.4.1 size: 750 KB in: require failed.
vt-pbf@3.1.0 size: 814 KB in: require failed.
react-spinkit@3.0.0 size: 1009 KB in: require failed.
react-icons@2.2.7 size: 1 MB in: require failed.
faker@4.1.0 size: 1 MB in: require failed.
watson-developer-cloud@3.0.7 size: 1 MB in: require failed.
recharts@1.0.0-beta.10 size: 1 MB in: require failed.
mapbox-gl@0.45.0 size: 3 MB in: require failed.
npm@5.6.0 size: 6 MB in: require failed.
I think I can get rid of mapbox-gl it’s taking 3 MB
Nice, keep in mind those are TAR.GZ sizes as fatapp only fetches http headers for sizes from NPM for performance.
I think the issue I’m facing is due to the way we’re using the package system. We develop within the packages so the react components are there and I think packages are not optimized for quick refresh the way the code core is. At least that’s my hypothesis for now, I’ll test it by moving the package code to the core import directory and see how the performance change.
Thanks for this tip. But tbh, this would take away pretty much of the “awesomeness” of Meteor, i.e. that everything is compiled “automagically”. I also do not really get why pre-compilation of SCSS should take so much time, since in Webpack it is done in an instant. And, as said before, pure SCSS compilation in fact works faster in my project. It’s the compilation of the source code that takes that long.
How did you profile this? I would be curious to see what causes the delays on my side.
Yea, I would say do not use the packages/ system as much as possible, the rebuild process is different, meteor is best when things are vanilla and small and tight.
I completely understand this, I used to write in HAML/SASS/COFFEE, but gave them up for various reasons, speed one of them. With that said, CSS supports a lot more features now so I feel CSS preprocessors are less required. As for JS, it’s already transpiled by babel, so a transpile in a transpile seems weird, and like CSS new features really pull it more in line with what JS preprocessors provided. As for HAML, it’s really not needed, blaze does it’s down thing, and while I love it, vanilla just feels easier to write.
Webpack behaves differently and can given be configured to use threads to parallelize asset packaging. I hope meteor comes to compete on this level.
METEOR_PROFILE=1 is the main one, but also time <command>, comparing node ./.meteor/local/build/main.js to meteor and various other ways observations
I couldn’t live without SCSS’ nesting and (cross-file) variables capabilities. And besides, if you would like to support older browsers, the most recent CSS features won’t help you much (unless you have a preprocessor in place).
Valid points, I wonder if transpiling them manually via a separate watcher or adhoc provides better performance…
I mostly work on projects that support evergreen only and when you look at the market shares of browsers, it divides me if the ladder are even worth the effort for many applications.
Ok so I did more digging and yeah building within the package is adding almost 7s (out of ~16s) on every refresh. We’re using package based architecture and we use packages as a “mini apps” that contain all the code (routers, pages etc.) however Meteor build system is not optimized for this use case.
When we modify the package, Meteor prepareProjectForBuild.. stage adds almost 5s to the build process and two tasks cause the majority of the added time:
Builder#copyNodeModulesDirectory...............1,972 msI assume this is copying the package node modules to the project node modules folder, so the more and larger package node modules the slower this refresh will probably get when changing a package.
files.renameDirAlmostAtomically.............2,330 msOther then that the refresh time seems to increases slowly relative to how many files you’ve since it seems like the build system is going over everything (scanning and maybe processing too) anytime something change, which is not best behavior if that’s the case, Meteor gurus please keep me honest here.
I think there a room to make the build system faster by making the processing more granular.
So two takes-away for me:
I’m still impressed you’re getting only a 1-3 seconds! if this is a large app with many node modules/components then maybe there is something else going on with my environment/code for the extra 5-6s.
I just used METEOR_DEBUG_BUILD=1 METEOR_PROFILE=1 meteor run and a phone stopwatch.
And I did some experimentation by moving the the code around to see how it impacts the performance. The suspects were:
And now I think (1) has added around 6s relative to a fresh meteor project which make sense since we’re about 3 times larger then the starter project and (3) added 7s We’ve some refactoring to do
Try out https://github.com/qualialabs/profile for much much more fine grained rebuild profiles
That’s exactly how my apps are organized. But we don’t use package-level npm modules.
We had a developer do that it was a disaster for various reasons, we instead use imports heavily and organizational level NPM packages, they’re easier to manage and can simply go in package.json. We maintain a few different apps: frontend, backend, and two ternary services. All our apps use vanilla code (js/css/html). We receive burst traffic of 6000+ sessions which maintain for 30+minutes each, we’ve tested our system to 12k+ sessions.
I am constantly on watch for bloat from devs and externally. We/I spend a strong amount of effort keeping meteor lean.
The problem is that npm packages still do not cover all the features I get from Meteor packages. For instance, I have to include different code for web and mobile. With Meteor packages, this can be done pretty easily using the web.browser and web.cordova flags. I would love to see Meteor bringing these must-have features to npm, but unfortunately mobile development doesn’t seem to be the main focus.
BTW: How do you maintain these org level npm packages? Do you host your own npm server, or do you use the npm cloud service?
Actually the mini-vertical app structures works extremely well since it allows heavy code re-use and shared foundation. So basically instead of splitting the app server/client (horizontally), you do the split vertically. And here I’m only talking about how to structure web apps (platform with mini-apps).
With regards to the refresh/rebuild performance with this architecture implemented via meteor packages, the bottleneck was with the NPM modules in the packages but as @waldgeist hinted it’s better no define them there. Once those NPM definition removed from the packages, the performance of the refresh with and without the package based architecture is almost the same (still high though 8 to 10s due to the large number of page/react components I think).
This image from this article illustrate the concept:
And here is an article detailing the concept, they call it micro-frontends.
I use verdaccio for npm package cache. It speeds up meteor builds a lot.
This helps in development?