from a post 2 days ago on the Community Slack
Slack user jeremias writes:
Hello, I’ve a question about Meteor and its memory limits, I’ve been playing around running some tests to check if Meteor is using all the memory I assign to it (–max-old-space-size), but I get results that I’m not understanding properly or I’m missing some config or similar, because it dies at different times and memory usages every time, and I never could make it pass the 4G, even if I --max-old-space-size=7000 in example. I would appreciate if you can point me to any docs or tell me please where it’s better to ask this kind of questions. Thank you!!
I missed to say that when I run the same test code snippet without Meteor, it behaves as I expect, dying near to the memory limit I assign.
(this topic is cross-posted to the original post on Community Slack)
I think you’d be limited to around 4GB if you were running a 32bit node process. Could that be the case here?
There is no “Meteor runtime” so Meteor in runtime on the server is just Node.js.
What maybe be happening is that your NODE_OPTIONS variable is not getting at your Node.js process.
How are you running it?
@alimgafar I don’t know how Jeremias is going to see this answer here but maybe you posted this link on Slack as well, right? Thanks!
Slack link is at the top of the original post. And I posted this link to this thread in Slack when I created it. He acknowledged in Slack.
Conclusive replies should end up in this thread as it is more durable to future searches.
Hi, thank you all for your time, I really appreciate it.
I’m running the tests on 64bit machines and the NODE_OPTIONS are in place, I also tried with node --max-old-space-size directly without the NODE_OPTIONS.
That said, I started yesterday a new repo so I can provide you a proper report but I got some results I didn’t expect. Meaning that in my laptop everything works as expected, but when I ran the tests on the production environment it doesn’t work as expected.
Thanks again for your time and sorry for the delay, but once I get to the bottom of this I’ll let you know.
Hi, could you paste which error is your Node.js process ending with?
Also, did you try to get the statistics directly from v8? See this code for reference.
I don’t know where you run your production servers but you could install Prometheus + Grafana to have this dashboard available, it shows heap spaces in detail.
If you deploy to zCloud (there is $15 bonus for new accounts so you don’t need to spend any money) I could add this dashboard for you there but you also need to add some lines of code to your app in order to collect the metrics (if you want to investigate at this level send an email to email@example.com and we will help you to set up your metrics).
The code necessary is also here in Quave’s template if anyone wants to give it a try. In this specific case, I also added some Fibers’ metrics into the mix.
Hello, thanks you all again for your time.
Trying to be brief, it was all my fault, my apologies for bothering you. The script starts meteor in production was also loading (source) another file that was overwriting the NODE_OPTIONS env var… And I was also relying on the numbers from v8.getHeapStatistics(), which doesn’t seem to be the best approach.
And also too, I was setting the --max-old-space-size and the docker limits at the same values, and I think there was kind of a race condition there, as sometimes node itself died with a OOM stack trace, others it was just killed by docker (or the kernel?).
Once I removed that script which was overwriting NODE_OPTIONS and I set the docker mem limit a bit higher (100M) than the node limit. I got the results I was expecting: the process dying close to the mem limit.
In brief: it all works as expected!
To allocate memory for Meteor you just have to set the --max-old-space-size=XXX (in Megabytes) for node. In example:
Or just run:
node --max-old-space-size=7000 main.js