Upgrading to 1.3 made javascript payload size double

[ryanswapp]

I recently upgraded an app at work to 1.3 and noticed that the javascript payload went from 400kb to about 1MB. The only real differences between the 1.2 and 1.3 versions of the app are that I removed meteorhacks:npm in the 1.3 app and npm installed the node modules. I’ve been trying to track down which libraries are causing the bloat but all I can see is a large js file called modules.js that is 2mb in development. Has this happened to anyone else? Any suggestions for reducing the payload?

@benjamn I assume you are most familiar with the 1.3 changes as you did most of them Do you know why my app payload grew substantially after upgrading?

[pauljohncleary]

@ryanswapp, it might help to list your npm modules & atmosphere packages

Does anyone know if there is a way to add some statistics into the build system to monitor this? It would be great to be able to test for this and fail the build if the payload becomes bigger than x size

[serkandurusoy]

Have you checked the gzipped sizes?

The thing is, importing modules into different places can cause code duplication on the bundle, but since gzipping mitigates the duplication, the end result that actually that gets sent down to the client is only negligibly larger.

[hwillson]

Here’s a really quick manual hack to guesstimate what’s causing the size increase.

1. Download a copy of your production Meteor combined/minified JS file (e.g. f53167628aaaa6029bf1bba83b167b18af7265a6.js).

2. Use csplit to split the file into multiple new files based on blank lines. For example (on OS X but can run some variation of this on any OS):

csplit -k f53167628aaaa6029bf1bba83b167b18af7265a6.js '/^.$/' {1000}

3. Sort the newly created files by size and crack open the largest file:

ls -l xx* | sort -r | more

The minified Meteor JS file splits most packages/modules up by blank lines, so this will at least allow you to narrow in on the biggest culprit. Open any of the files and look at the last few characters of the file - in most cases you’ll see something like ...Package["forwarder:autoform-wizard"]... to help give you a hint.

[ryanswapp]

@hwillson You’re the man! I wish I was a little better at bash utils… Hopefully I’ll get there some day. It looks like the entire lodash library is being pulled in. I will have to investigate and see what the issue is… Thanks for your help!

[ryanswapp]

Anyone know if the 1.3 meteor build system pulls in all installed node modules? Or just the ones you’ve imported?

[serkandurusoy]

Just the imported ones.

Perhaps you are importing the complete library somewhere.

You could import specific methods from the lodash library for the places in the app that you use them.

[ryanswapp]

@serkandurusoy Ya I’m pulling in just the lodash methods that I need (e.g. import assignIn from ‘lodash/assignIn’). In looking at the bundle more closely I think that underscore may be the issue. I’m not pulling it in but some of my dependencies are. In addition, it looks like the Meteor build system may be pulling in multiple copies of underscore and other libraries. I count at least 5 copies of tracker and over 10 copies of underscore…

[serkandurusoy]

In that case, you sould definitely compare the gzipped bundle sizes across the versions.

Those multiple underscore copies will practically count as a single underscore copy in terms of bytes transferred.

EDIT:
You should definitely read this write up on the subject: https://www.tjvantoll.com/2014/01/27/only-the-gzip-size-matters/

[alanning]

Another forums thread also has info about this topic:

Curious why Deploy takes 3 times longer, after update to Meteor 1.3?

Hi, I ran a meteor build on my medium sized app and ended up with a 30M bundle size (compressed) the node_modules directory is the biggest in size > 100M meteor package being the heaviest: 40M I also noticed that my --dev-dependencies packages are included as well - which I found … unnecessary would really like to know how this compares to others thanks

This post in particular links to an open issue on github and shows a workaround that may help:

Curious why Deploy takes 3 times longer, after update to Meteor 1.3?

This is a known issue on Github: https://github.com/meteor/meteor/issues/6750 Looks like it will hopefully get fixed in a future 1.3.x release. A current workaround as provided by Ben Newman: To that end, I recommend you run mv node_modules .node_modules.backup meteor npm install --production in your app before running meteor build, so that you don’t have to copy the devDependencies. Then the node_modules directory will only be 46MB, and the copy should take 8% as long as it did befor…

If I’m understanding correctly, it seems the npm modules are mostly to blame with non-production assets being included.

[tcastelli]

Also, if you are using devdependencies in your package.json, it seems like they will be included in your final bundle,
check the open issue https://github.com/meteor/meteor/issues/6836 for more details. One proposed solution would be to install npm packages with npm install --production before building and deploying. I still haven’t tried though

[ryanswapp]

Thanks for the links! I just gave the workaround a go and the results were interesting… I’m using mupx to deploy and without the workaround the payload comes in 991kb. With the workaround the payload of the app is 1.3mb. Isn’t that bizarre? Perhaps mupx is already doing some kind of production build optimizations for 1.3. I guess I’ll keep looking for the culprit of my payload bloat

@tcastelli Yikes! That’s not a fun bug. Fortunately I don’t have any devDependencies declared in the package.json. I’ll be sure to avoid any for the time being.

@serkandurusoy Great gzip article! Thanks for sharing.