Help us test build times in Meteor 1.3

$ time meteor build /tmp/build --debug --server localhost
real 1m17.487s
user 0m51.284s
sys 0m15.120s

$ time meteor build /tmp/build --debug --server localhost
real 1m1.387s
user 0m49.824s
sys 0m13.728s

$ time meteor build /tmp/build --debug --server localhost
real 1m0.071s
user 0m48.096s
sys 0m14.292s

āÆ time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost 51.27s user 13.88s system 115% cpu 56.460 total

āÆ time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost 50.90s user 13.68s system 116% cpu 55.396 total

āÆ time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost 51.20s user 13.81s system 116% cpu 55.800 total

1.3
āÆ time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost 165.25s user 17.15s system 106% cpu 2:51.67 total

āÆ time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost 67.61s user 14.68s system 113% cpu 1:12.58 total

āÆ time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost 68.25s user 15.02s system 114% cpu 1:12.98 total

:frowning:

Iā€™ve also noticed that beta4 is much slower than the previous betas, no hard evidence, but just from working with it. Were you using beta4, the latest 1.3 release?

Yes, it was the latest release (beta4)

We have a simple meteor project, that imports a private npm package that contains most of our shared components.

The private npm package exposes a bundle.js file that is built with webpack, in the main property.

The size of this bundle.js file significantly impacts Meteorā€™s build time, even if itā€™s already compressed. Is this expected?

For example, if the bundle.js file is 400kb, Meteorā€™s build time looks like (on beta4, beta5 has stylus issues for us):

time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost  26.55s user 5.13s system 104% cpu 30.232 total

time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost  24.33s user 4.86s system 109% cpu 26.711 total

time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost  23.79s user 4.66s system 112% cpu 25.311 total

Fully loaded, the package is 2.6mb (uncompressed), and meteorā€™s build time doubles (if we compress the bundle.js file, times donā€™t change):

time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost  51.93s user 8.56s system 104% cpu 57.715 total

time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost  55.48s user 9.23s system 103% cpu 1:02.35 total

time meteor build /tmp/build --debug --server localhost
meteor build /tmp/build --debug --server localhost  54.21s user 9.04s system 103% cpu 1:01.25 total

Rebuild times are killing our productivity, and we were hoping that we could improve build times by separating most of our code into an npm package thatā€™s managed by webpack, which meteor would import as any other npm package. The code inside the actual meteor project is minimal - it mostly just imports components from the compiled npm package.

1 Like

Here are my times with the newest METEOR@1.3-modules-beta.5

My installed packages:

accounts-password
blaze-html-templates
ecmascript
email
jquery
jsx
mobile-experience
mongo
random
session
standard-minifiers
tracker
underscore

agnito:raphael
aldeed:collection2
aldeed:simple-schema
dburles:collection-helpers
flowkey:bowser
harrison:papa-parse
juliancwirko:s-alert
juliancwirko:s-alert-flip
kadira:flow-router
maximum:computed-fields
meteorhacks:kadira
meteorhacks:kadira-profiler
meteorhacks:search-source
meteorhacks:subs-manager
meteorhacks:zones
mrt:jquery-easing
richsilv:owl-carousel
ultimatejs:tracker-react

I have the following installed with npm:


    "material-ui": "^0.14.3",
    "moment": "^2.11.1",
    "nouislider": "^8.2.1",
    "react": "^0.14.6",
    "react-addons-css-transition-group": "^0.14.6",
    "react-dom": "^0.14.6",
    "react-komposer": "^1.3.0",
    "react-mounter": "^1.0.0",
    "react-tap-event-plugin": "^0.2.1"

Initial build time with no files in the entire project:

real	0m35.736s
user	0m29.775s
sys	0m3.304s

Pretty good. When I add a single (small) client/FlowRouter.jsx file, build times stay roughly the same. Adding a console.log to a route and testing file modifications, subsequent reloads are roughly 7 seconds after file saves. Not bad.

But as soon as I import three other component/JSX files (located in client/imports directory) to the FlowRouter.jsx file, build times are immediately exacerbated.

Initial load time:

real	1m2.826s
user	0m51.577s
sys	0m6.625s

Now, making the exact same change to the FlowRouter.jsx file as before (changing a console.log line), reload times are now 25 seconds.

There is something seriously wrong here. This is a project with 4 files.

I tried disabling all modules, adding each one back in and testing times, however this didnā€™t change build times at all. I did manage to port my entire medium-sized app to 1.3, however it was a lot of blind coding, then sipping my coffee for a few minutes between builds. Itā€™s not possible to develop within itā€™s current state.

Iā€™m very excited about the future though! Meteor 1.3 is a pleasure to develop in (Iā€™m imagining 7 second page reloads or less here :wink: ). The new import/exports functionality and npm integration is fantastic.

Update: I forgot to mention what build times are with the current app on Meteor 1.2. Here is initial build:

real	0m26.592s
user	0m19.694s
sys	0m3.154s

And modifications take roughly 17 seconds to reload. However, this is with all of the Meteor-specific packages and no npm (react, material-ui, etc.).

7 seconds is still very long compared to something like webpack where itā€™s 2-3 seconds.

I really hope 1.3 brings some serious improvements because my builds clock at 15-20 seconds and I still have to wait for the page to reload after that.

Totally agree. We moved from plain react + rest api to meteor, and have generally been pleased. Lately though, rebuild times in 1.2 have seriously impacted our teams productivity. With our previous setup, we could make a change to a file and by the time we tabbed to the browser that change would be represented on the screen. This is enormously powerful for productivity! Now we need to wait ~10-15 seconds for each change, which really adds up over the course of a day.

While the rebuild times in 1.3 are too large for us to realistically use it right now (30-45 seconds for our project), Iā€™m confident MDG understands the importance of quick rebuilds, and will figure it out :smile: .

3 Likes

Rebuild times in 1.3-modules-beta.8 are looking substantially better!

Iā€™ve also done some work on React hotloading in native Meteor (which only until beta.7 but should work again with beta.9 when it comes out).

2 Likes

Hm, this is interesting.

Iā€™m running this on C9 and I get

real 3m37.857s
user 2m11.712s
sys 0m50.248s

and after the update

real 4m12.896s
user 2m40.608s
sys 0m53.856s

this does not look good.

1 Like

I ran the build test on Wekan again. 1.3 build time are sensibly the same as 1.2.2 with the patch for faster rebuild:

1.2.2-faster-rebuilds.0
meteor build /tmp/build --debug --server localhost 53,90s user 6,38s system 95% cpu 1:03,19 total
meteor build /tmp/build --debug --server localhost 51,55s user 5,17s system 103% cpu 54,803 total
meteor build /tmp/build --debug --server localhost 50,74s user 5,08s system 101% cpu 54,788 total

1.3-modules-beta.8
meteor build /tmp/build --debug --server localhost 60,80s user 6,74s system 104% cpu 1:04,37 total
meteor build /tmp/build --debug --server localhost 53,62s user 5,99s system 105% cpu 56,578 total
meteor build /tmp/build --debug --server localhost 55,54s user 6,17s system 104% cpu 58,926 total

@dgreensp used Wekan specifically as the example app when optimizing rebuilds.

I donā€™t think he optimized (much) on initial builds. How do rebuilds compare between 1.2.2-faster-rebuilds.0 and 1.3-modules-beta.8?

Yes I experienced faster rebuilds, though donā€™t have any precise number right now. Also Wekan still use one legacy handler (.i18n.json) that eats ~700ms in every rebuild and that I would really like to replace by the new API (I started a PR on that).

No appreciable difference in Workpopā€™s app on these builds unfortunatelyā€¦ both 1.2 and beta.8 are ~75minutes long.

@luisatlive: Do you mean 75 seconds?

Ha! Yes, sorryā€¦ hasnā€™t gotten that slow yet :slightly_smiling:

Good to hear, you got me worriedā€¦

1 Like

Is that just for initial builds? Are the rebuilds faster?

This is a totally empty project, just with the content that meteor create xpto does.
I modified a HTML file, and after five seconds the console detect it and rebuild.

Time to see changes in the browser was 5~6 seconds.

Select Package Versions: 3851.8
|     new CS.Input: 3.0
|     Input#loadFromCatalog (sqlite): 3728.0
|     other Select Package Versions: 120.8
|
| Input#loadFromCatalog (sqlite): 3728.0
| other Select Package Versions: 120.8
| new CS.Input: 3.0
| Total: 3851.8
|
| files.writeFile: 71.0
| Rebuild App: 207.5
|     compiler.compile(the app): 14.1
|         files.readdir: 3.7
|         files.stat: 2.3
|         other compiler.compile(the app): 5.5
|     Isopack#getUnibuildAtArch: 2.3
|     bundler.bundle..makeClientTarget: 123.4
|         Target#make: 123.4
|             Isopack#getUnibuildAtArch: 5.2
|             Target#_runCompilerPlugins: 30.0
|                 sha1: 2.9
|                 Isopack#getUnibuildAtArch: 2.6
|                 processing files with ecmascript (for target web.browser): 5.1
|                     other processing files with ecmascript (for target web.browser): 5.0
|                 processing files with templating (for target web.browser): 5.4
|                     other processing files with templating (for target web.browser): 4.8
|                 other Target#_runCompilerPlugins: 13.5
|             Target#_emitResources: 65.2
|                 PackageSourceBatch#getResources: 62.6
|                     PackageSourceBatch#_linkJS: 61.1
|                         files.readFile: 18.9
|                         Isopack#getUnibuildAtArch: 3.0
|                         other PackageSourceBatch#_linkJS: 37.4
|                     other PackageSourceBatch#getResources: 1.4
|                 other Target#_emitResources: 2.6
|             Target#minifyJs: 3.1
|             ClientTarget#minifyCss: 1.1
|             sha1: 7.7
|             other Target#make: 10.9
|     bundler..writeTargetToPath: 63.0
|         ClientTarget#write: 62.2
|             Builder#write: 8.1
|                 sha1: 3.0
|                 other Builder#write: 3.5
|             bundler..writeFile: 4.5
|                 Builder#write: 3.7
|                     other Builder#write: 2.7
|             other ClientTarget#write: 48.9
|     other Rebuild App: 4.5
|
| files.writeFile: 72.1
| other ClientTarget#write: 48.9
| other PackageSourceBatch#_linkJS: 37.4
| files.readFile: 20.3
| sha1: 14.6
| Isopack#getUnibuildAtArch: 14.0
| other Target#_runCompilerPlugins: 13.5
| other Target#make: 10.9
| other Builder#write: 6.2
| other compiler.compile(the app): 5.5
| other processing files with ecmascript (for target web.browser): 5.0
| other processing files with templating (for target web.browser): 4.8
| other Rebuild App: 4.5
| files.readdir: 3.7
| Target#minifyJs: 3.1
| files.stat: 2.8
| other Target#_emitResources: 2.6
| other PackageSourceBatch#getResources: 1.4
| Builder#_ensureDirectory: 1.3
| ClientTarget#minifyCss: 1.1
| Total: 273.8

Now, when I change a JS file, the console dectects right when I press ā€œCommand + Sā€ and rebuild it. But to see changes in the browser and get it reloaded takes 5~6 seconds too.

|  Selecting package versions                /
| Select Package Versions: 3931.5
|     new CS.Input: 4.4
|     Input#loadFromCatalog (sqlite): 3801.5
|     other Select Package Versions: 125.6
|
| Input#loadFromCatalog (sqlite): 3801.5
| other Select Package Versions: 125.6
| new CS.Input: 4.4
| Total: 3931.5
|  Building the application                  |
| Rebuild App: 673.3
|     compiler.compile(the app): 14.8
|         Isopack#getUnibuildAtArch: 1.2
|         files.stat: 4.1
|         other compiler.compile(the app): 7.2
|     Isopack#getUnibuildAtArch: 1.7
|     bundler.bundle..makeClientTarget: 147.1
|         Target#make: 147.0
|             Isopack#getUnibuildAtArch: 5.3
|             Target#_runCompilerPlugins: 49.4
|                 sha1: 2.8
|                 Isopack#getUnibuildAtArch: 3.0
|                 processing files with ecmascript (for target web.browser): 27.1
|                     other processing files with ecmascript (for target web.browser): 27.0
|                 processing files with templating (for target web.browser): 1.3
|                     other processing files with templating (for target web.browser): 1.2
|                 other Target#_runCompilerPlugins: 14.0
|             Target#_emitResources: 68.9
|                 PackageSourceBatch#getResources: 66.1
|                     PackageSourceBatch#_linkJS: 64.6
|                         files.readFile: 19.8
|                         Isopack#getUnibuildAtArch: 3.1
|                         other PackageSourceBatch#_linkJS: 40.8
|                     other PackageSourceBatch#getResources: 1.5
|                 other Target#_emitResources: 2.8
|             Target#minifyJs: 2.8
|             ClientTarget#minifyCss: 1.9
|             sha1: 7.7
|             other Target#make: 10.9
|     bundler.bundle..makeServerTarget: 116.9
|         Target#make: 116.7
|             Isopack#getUnibuildAtArch: 7.3
|             Target#_runCompilerPlugins: 30.3
|                 sha1: 2.8
|                 Isopack#getUnibuildAtArch: 3.5
|                 processing files with ecmascript (for target os.osx.x86_64): 7.1
|                     other processing files with ecmascript (for target os.osx.x86_64): 6.9
|                 other Target#_runCompilerPlugins: 16.3
|             Target#_emitResources: 66.0
|                 PackageSourceBatch#getResources: 62.4
|                     PackageSourceBatch#_linkJS: 60.7
|                         sha1: 1.1
|                         files.readFile: 22.6
|                         Isopack#getUnibuildAtArch: 3.6
|                         other PackageSourceBatch#_linkJS: 33.3
|                     other PackageSourceBatch#getResources: 1.7
|                 other Target#_emitResources: 3.7
|             other Target#make: 13.0
|     bundler..writeSiteArchive: 388.1
|         Builder#write: 3.0
|         bundler..writeTargetToPath: 318.9
|             ClientTarget#write: 129.6
|                 Builder#write: 48.1
|                     sha1: 4.0
|                     files.stat: 8.2
|                     files.writeFile: 18.2
|                     files.rename: 7.0
|                     other Builder#write: 10.0
|                 bundler..writeFile: 45.6
|                     Builder#write: 44.7
|                         files.stat: 7.7
|                         files.writeFile: 20.1
|                         files.rename: 7.4
|                         other Builder#write: 8.7
|                 other ClientTarget#write: 34.7
|             ServerTarget#write: 186.8
|                 Builder#write: 8.2
|                     files.readFile: 2.5
|                     files.writeFile: 1.9
|                     other Builder#write: 1.7
|                 JsImage#write: 175.2
|                     files.mkdir: 2.1
|                     Builder#write: 112.2
|                         Builder#_ensureDirectory: 1.7
|                         sha1: 11.4
|                         files.stat: 18.4
|                         files.writeFile: 42.3
|                         files.rename: 16.3
|                         other Builder#write: 22.1
|                     Builder#copyDirectory: 1.9
|                         files.symlink: 1.1
|                     other JsImage#write: 58.3
|                 other ServerTarget#write: 2.9
|         Builder#complete: 59.6
|             files.rm_recursive: 59.3
|         other bundler..writeSiteArchive: 4.9
|     other Rebuild App: 4.3
|
| files.writeFile: 84.9
| other PackageSourceBatch#_linkJS: 74.1
| files.rm_recursive: 60.1
| other JsImage#write: 58.3
| files.readFile: 46.3
| other Builder#write: 43.2
| files.stat: 41.8
| other ClientTarget#write: 34.7
| files.rename: 33.1
| sha1: 31.7
| other Target#_runCompilerPlugins: 30.3
| Isopack#getUnibuildAtArch: 28.7
| other processing files with ecmascript (for target web.browser): 27.0
| other Target#make: 23.9
| other compiler.compile(the app): 7.2
| other processing files with ecmascript (for target os.osx.x86_64): 6.9
| other Target#_emitResources: 6.5
| other bundler..writeSiteArchive: 4.9
| other Rebuild App: 4.3
| Builder#_ensureDirectory: 3.5
| other PackageSourceBatch#getResources: 3.2
| other ServerTarget#write: 2.9
| Target#minifyJs: 2.8
| files.mkdir: 2.7
| ClientTarget#minifyCss: 1.9
| other processing files with templating (for target web.browser): 1.2
| files.symlink: 1.1
| Total: 667.1

Using version --release METEOR@1.3-modules-beta.8
Macbook Pro I5 with 16GB RAM and 256 SSD.

Should be so slow like this?

Unfortunately those are rebuild times I quoted :frowning: