How I made our app 35%~ faster in build, rebuild, browser load time & size


#1

Three things:

  1. Remove meteor-node-stubs
  2. Track bloat in NPM dependencies ( try my package-lock.json scanner: https://www.npmjs.com/package/fatapp npm install -g fatapp & fatapp --help to get started)
  3. Review client / server imports so they are not importing packages/code in an environment needlessly

Results may vary, happy hacking :+1:


Does the meteor development need high config machines?
Build Speeds for Development are unreasonable, can we parallelize them or improve them further?
#2

What was the biggest package/module, which you got rid of? I am looking forward for next Meteor version, where we will be able to remove e.g. jQuery.

Although, for Meteor, you can use Bundle Visualizer - also great tool, but coming straight from MDG.


#3

I’ve seen NPM packages pulling in react deps unnecessarily, which fed out a via meteor-node-stubs.socket.io-clienthas some weight. Yup jQuery is another it and underscore may be double loading. I savagely ripped jQuery out from:useraccounts/core,blaze,accounts-ui-unstyled,kadira/blaze-layout,amplifyandtwbs/bootstrapthey're all local to the/packagesdirectory now and jQuery loaded from CDN. Another issoftwarerero/accounts-t9nwhich loads all translations which adds a fair bit of weight, it also using coffeescript (<3), but I generatedjsfiles to save transpile time, there's little reason to transpile a pretty much static asset, I'm not sure how that's cached though. Loading less languages was a gain in size. I switched:highcharts,jqueryandsocket.io-clientto pull from public CDNs, although I may bundle them to avoid too many external requests, but the jQuery CDN seems like a safe bet. I pretty much wiped outtwbs/bootstrapto also load from public CDN. There seems to be something having less data in a project that increases speed (less transpiling, less file watchingstat, etc), Odd various deps not really need that I was able to cut out after reviewing thepackage-lock.json and .meteor/packages files.

Cheers


#4

Thanks for sharing @robertlowe. If you use softwarerero/accounts-t9n as a npm package no language files will be imported automatically, instead you need to import the languages you need explicitly. Also using CoffeeScript is optional now, by importing files from the build directory plain JS can be used.


#5

What are the implications of removing meteor-node-stubs. I mean it’s there for a reason, what do you loose by removing it?


#6

Should be none, they’re stubs after all? I haven’t noticed any issues.


#7

Many thanks, I also tried to use the environment variable but didn’t dig into why it wasn’t working. Thanks for the NPM heads up, I’ll probably pull from there in future, but as of right now, I think user accounts still requires it.


#8

Let’s see if Ben can see any pitfalls from doing it


#9

The meteor-node-stubs package provides client “stub” implementations of Node.js built-in functions (e.g. crypto, path, util, etc.) which are not implemented on the client. So while the Node.js crypto module on the server uses the native C++ implementation, the Node stub would use a JavaScript implementation since that’s what’s available on the client. Webpack and Browserify implement similar behavior and use many of the same stub implementations as Meteor does, such is the case as how Meteor implements crypto on the browser, which is done with crypto-browserify. In an isomorphic application, where client and server code is shared, you can probably understand how critical these stub packages might be — which is why we include the package in every new Meteor application. (Surely we wouldn’t include it just for fun, right? :slight_smile:)

Now while the meteor-node-stubs package is included in all new Meteor applications, the individual stub implementations (e.g. crypto, etc., as itemized in the package.json from the actual source for meteor-node-stubs, here) are only included in the generated client bundle if their usage is detected by Meteor’s import scanner. If you don’t provide the meteor-node-stubs package, then it’s possible that your client code could fail at run-time due to the absence of the necessary stub package which mimics the server implementation.

If you are finding (e.g. via the bundle visualizer) that your client bundle is including stub packages which you don’t expect to be included, you’ll want to analyze your module entry points carefully. Meteor will include a stub implementation if it finds Node.js module usage while bundling your application. It may be that your own application code is doing a require('util') or require('crypto'), or it may be one of your Node module dependencies! Either way, it’s worth investigating, as you may find that you’re unnecessarily including a large amount of unneeded code by not being careful about what you import!

So while @robertlowe’s recommendation number “two” and “three” are very reasonable, I would use caution before just removing meteor-node-stubs and urge you to understand the implications. If Meteor thinks it’s needed, you should probably gain understanding of why that is. Only then will you be able to confirm that those packages aren’t needed.


#10

Absolutely agree on caution, I’ll add that if you have something like require('util') or require('crypto') it should fail because it’s clearly not intended to be client-side.

I would hope anyone making such changes has some acceptance testing in place for safety.


#11

Just to be clear: Both util and crypto can be used (at least as much as their client implementations will allow/have implemented) on the client side (thanks to crypto-browserify and util) if you include meteor-node-stubs. The meteor-node-stubs package is exactly what makes this possible and that’s exactly the point I’m trying to make.

When client code is detected to have used either of those module identifiers (crypto or util), then the stub defined in meteor-node-stubs will be included. If client code is not detected that uses either of those module identifiers, then the stub provided by meteor-node-stubs will not be included.


#12

Understood, but it opens up a world of bloat.


#13

Any way to modify meteor-node-stubs to output some context when it detects it is needed during development? It’d be great if the tool were to tell you "module xyz has requested crypto" or something like that.


Faster Meteor Reloads