Please help me troubleshoot: Application exited with signal: killed (out of memory)

help
1

How can I troubleshoot this server error?

Application exited with signal: killed (out of memory)
Application process starting, version 172```

It is happening during a long process. It doesn't happen at the same time or place each time a try it.

I'm hosted with one compact container on Galaxy.

I'm not sure what else you may need to know to help me.
2

It could be related to this issue: https://github.com/meteor/meteor/issues/8630

My app in production has also been restarting automatically multiple times per day due to “out of memory”, so I’m waiting eagerly for version 1.4.4.2 to be released.

3

I’m thinking that the “out of memory” problem is caused by this:

Node (V8) uses a lazy and greedy garbage collector. With its default limit of about 1.5 GB, it sometimes waits until it absolutely has to before reclaiming unused memory. If your memory usage is increasing, it might not be a leak - but rather node’s usual lazy behavior.

You can try to pass some flags when starting the app with node. One example provided in that article is:

web: node --optimize_for_size --max_old_space_size=460 --gc_interval=100 server.js

1 Like
4

Thank you. I’d like to try that.

How do I pass a flag when starting my Meteor app in production on Galaxy?

5

Ok, I think I figured out how to pass flags like those.


{
  "galaxy.meteor.com": {
    "env": {
      "MONGO_URL": "...",
      "MONGO_OPLOG_URL": "...",
      "NODE_OPTIONS": "--optimize_for_size --max_old_space_size=460 --gc_interval=100"
    }
  }
}

However, I am getting this error:

Error: Cannot find module '/app/bundle/server.js’
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Function.Module.runMain (module.js:441:10)
at startup (node.js:140:18)

So, I am going to take off the “server.js” part and see if that works. (http://galaxy-guide.meteor.com/scaling.html)

6

For a Meteor app, it’s main.js.

7

http://galaxy-guide.meteor.com/scaling.html doesn’t mention putting main.js at the end.

Do you think it is necessary?

1 Like
8

The javascript file name at the end is just the name of the file that you want Node to execute to start the app.

In a Meteor app bundle, you want Node to execute main.js.

I don’t use Galaxy, so I can’t help with Galaxy-specific details.

9

It appears that “main.js” does need to be at the end of that line.

My app got to 97% memory usage and stayed there for a while, but it didn’t crash (although the galaxy’s green dot did turn red).

10

Never mind. I just ran the process again and it crashed again.

Hmm.

11

I managed to fix my “out of memory” problem in production using the Node flags and installing Node 4.8.2 instead of using Meteor’s Node.

12

I included --expose-gc in the NODE_OPTIONS flags and added the following code in one of the loops in my long running process:

            if (global.gc) {
				global.gc();
				console.log('collecting garbage')
			} else {
				console.log('Garbage collection unavailable.  Pass --expose-gc '
					+ 'when launching node to enable forced garbage collection.');
			}

This seems to be helping, but I’m not sure it is the best idea.

13

Sorry to bump - @webmagnets did you ever get to the bottom of this? Currently trying to troubleshoot out of memory issues on Galaxy.

14

No. Sorry. I’m no longer involved in Meteor or any other programming projects.

15

Recently we had a similar issue and it was hard to debug. But ultimately we’ve found it was because we had a wrong query with fetch that returned millions a documents and starter to process them and choked the server.

1 Like