Very long rebuild time? Are other getting that? How to improve on it?


#1

Every time I save a file, I’ve to wait for 20s. That’s killing my productivity and increasing my twitter presence by a lot. What’s the excepted rebuild time for a medium size app?

Here is what I know from METEOR_PROFILE=1 meteor:

 Top leaves:
| files.readFile...........................................3,496 ms (1907)
| files.readdir............................................3,246 ms (5667)
| other compileUnibuild (the app)..........................3,229 ms (2)
| files.stat...............................................2,588 ms (45068)
| files.realpath...........................................1,854 ms (2934)
| files.writeFile............................................995 ms (4)
| Babel.compile..............................................555 ms (168)
...
| 
| (#14) Total: 19,028 ms (Rebuild App)
| 

I really don’t get why there is so much file things involved when all the changes will typically be on 1-2 files. I know Meteor is doing great smart things, but it also sounds like it doesn’t make that much sense. Anyway, it’s really hard to work like that.

For info, I’ve 350 files that I’ve personally written and 25k in my node modules, plus a small amount of meteor packages.


#2

Upon getting Meteor 1.3 working, rebuild time went WAY WAY down.

When I upgraded to 1.3.2.2, on my first attempt, the upgrade itself took about 30 mins to complete. Once it finally finished, the rebuild time went DRASTICALLY higher. Worse than it ever was.

I had to revert back to 1.3. Then I tried the upgrade again later that night, and it went fast as it should. Rebuild time was back to normal again.

I hope this could help you a bit. It seems it may be some sort of error with upgrading (at least on my issue)


#3

Ok I’ll try reverting and see what up. Thanks


#4

I can confirm that going back from 1.3.2.2 to 1.3 has divided rebuild time by 2-3 for me


#5

I am glad it helped!

For me, the second attempt at updating worked perfectly fine.

It seems the updating process does indeed have some sort of bug! I wonder if there is a bug report on this issue already?


#6

Can we get any input from MDG on this? My rebuilds tend to take around 10 seconds when changing server code, and 4-8 seconds when just changing HTML templates (Blaze). Also, I didn’t saw any real improvements by downgrading to Meteor 1.3.


#7

Yes it’d be good to have official input rather than trying random things (downgrading and reupgrading, reseting meteor…) that may work, may work temporary, may not work… as people are doing here https://github.com/meteor/meteor/issues/4284


#8

Does Meteor builder have some “Log performance” option? Than it would be clear where the bottlenecks are. I am not sure if it is open-sourced but if it is than everyone would just look into that part of code which causes delay and would be able to offer optimizations on github.


#9

METEOR_PROFILE=1 meteor gives a cummulative time for some functions, but it’s hard to dig into it because the profile has no trace


#10

I guess we need METEOR_PROFILE=42 than :slight_smile: 42 meaning verbose log file.

Also, do you use SSD? I see the bottlenecks are in file operations. I wonder if these can be moved to memory.

And from other thread I remember that every Meteorite sooner or later gets 500Mb-1.5Gb files in their builder directory. So the main reason behind slowness is probably HUGE amount of PHYSICAL drive file operations, which surely can be improved.


#11

:slight_smile: What’s SSD? I don’t think I’m doing anything very specific, but I’m very surprise to always see so many file operations while I’m changing 1-2 files


#12

SSD is Solid State Drive every developer should use nowadays with slow builders :slight_smile:

And many file operations is sure way to not run into errors when architecture of builder is error-prone or not segmented in reusable parts :slight_smile: Still, with small apps builder behaves well, sometimes it even does not rebuild app at all. Just telling 'Refreshing client …"


#13

I also realized the same after moving to Meteor latest version 1.3.2.x


#14

I am using SSD but the build times are still a bit painful.

Not really, if you run meteor with METEOR_PROFILE=1 you’ll see that even when it’s only saying “client refreshed” its still doing some rebuilds, that in my opinion are not necessary. For example it shouldn’t be necessary to rebuild the Javascript and CSS and prepare source maps if you only changed one HTML file.


#15

I forgot to note that write speed of SSD is really important. Cheap SSD do not do well at this parameter. I still wonder however if it is possible to move that process to RAMDrive. Probably not because even when My TEMP was on RAMDrive Meteor 1.2.1 crashed on build.

Interesting. Anyway, if I would be author of linker, I would also go “better safe than sorry” way though maybe these algorithms (on what and when to rebuild) are would be better positioned outside of builder code, in some JSON configuration… but… if Meteor would go webpack or other builder way, these is already obsolete discussion :slight_smile:


#16

I’m running my dev app in my mac. Are you saying I should by a SSD, mount it on my mac and run my meteor code from there? That doesn’t sounds great…


#17

In no way I am pushing you into any hardware replacement scenario. Software tweaks are always safer.

And I am not a prophet or evangelist of SSDs :slight_smile:

P.S. Just out of curiosity, what read and write speed do standalone Mac SSDs have?


#18

Is this fast enough? :slight_smile:


#19

Sure. So, I guess, bottleneck is really on Meteor builder side.


#20

There’s also this old issue… https://github.com/meteor/meteor/issues/5269, you might find some help there. I have a comment in there somewhere about how the size of your private folder affects rebuild times. If any of you are doing React, highly encourage using react hotloading :slight_smile: