[Solved] Meteor 1.8.1 deployed using MUP: JavaScript heap out of memory

#1

Hi everyone,

I know that similar threads exist - but none of them with a solution.

My production server suffers from out-of-memory exceptions which occur multiple times a day.

It is a 32GB machine and I have already set TOOL_NODE_FLAGS to 16GB - but no effect. It always crashes in mongo.js when processing a call from meteor-promise/fiber_pool.js

Does anyone know how to increase the memory limit when using mup? Or is this a known bug? (if yes, is it caused by meteor, mup or node?)

[127.0.0.1]FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
[127.0.0.1]
[127.0.0.1]<--- Last few GCs --->
[127.0.0.1]
[127.0.0.1][1:0x2b974c0] 29149496 ms: Mark-sweep 1315.3 (1461.5) -> 1315.2 (1462.5) MB, 1892.2 / 0.1 ms  allocation failure GC in old space requested
[127.0.0.1][1:0x2b974c0] 29151239 ms: Mark-sweep 1315.2 (1462.5) -> 1315.2 (1431.5) MB, 1743.1 / 0.1 ms  last resort GC in old space requested
[127.0.0.1][1:0x2b974c0] 29153030 ms: Mark-sweep 1315.2 (1431.5) -> 1315.2 (1431.5) MB, 1790.6 / 0.0 ms  last resort GC in old space requested
[127.0.0.1]
[127.0.0.1]
[127.0.0.1]<--- JS stacktrace --->
[127.0.0.1]
[127.0.0.1]==== JS stack trace =========================================
[127.0.0.1]
[127.0.0.1]Security context: 0x26cf01ea58b9 <JSObject>
[127.0.0.1]    1: /* anonymous */ [/built_app/programs/server/packages/mongo.js:~1051] [pc=0x2c7cc48fdcf5](this=0x191599c8c2f1 <JSGlobal Object>)
[127.0.0.1]    2: /* anonymous */ [/built_app/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:~14] [pc=0x2c7cc48fc894](this=0x191599c8c2f1 <JSGlobal Object>)
[127.0.0.1]    3: arguments adaptor frame: 1->0
[127.0.0.1]
[127.0.0.1]==== Details =======================...
[127.0.0.1]
[127.0.0.1] 1: node::Abort() [node]
[127.0.0.1] 2: 0x8ccf9c [node]
[127.0.0.1] 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
[127.0.0.1] 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
[127.0.0.1] 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
[127.0.0.1] 6: v8::internal::HashTable<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::New(v8::internal::Isolate*, int, v8::internal::PretenureFlag, v8::internal::MinimumCapacity) [node]
[127.0.0.1] 7: v8::internal::HashTable<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::EnsureCapacity(v8::internal::Handle<v8::internal::NameDictionary>, int, v8::internal::PretenureFlag) [node]
[127.0.0.1] 8: v8::internal::BaseNameDictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::EnsureCapacity(v8::internal::Handle<v8::internal::NameDictionary>, int) [node]
[127.0.0.1] 9: v8::internal::BaseNameDictionary<v8::internal::NameDictionary, v8::internal::NameDictionaryShape>::Add(v8::internal::Handle<v8::internal::NameDictionary>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyDetails, int*) [node]
[127.0.0.1]10: v8::internal::LookupIterator::ApplyTransitionToDataProperty(v8::internal::Handle<v8::internal::JSObject>) [node]
[127.0.0.1]11: v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::Object::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
[127.0.0.1]12: v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
[127.0.0.1]13: v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
[127.0.0.1]14: 0x2c7cc0c842fd

mup.js:

module.exports = {

  app: {

    env: {
      ...
      TOOL_NODE_FLAGS: '--max_old_space_size=16384',
    },

    docker: {
      image: 'abernix/meteord:node-8.15.1-base',
      buildInstructions: [
        'ENV TOOL_NODE_FLAGS="--max_old_space_size=16384"',
        'RUN export TOOL_NODE_FLAGS="--max_old_space_size=16384"',
      ],
    },
...

#2

There are two different environment variables for

  1. Building the app with Meteor on the computer mup is deploying from
  2. Running the app on the server

TOOL_NODE_FLAGS is used by the Meteor Tool and only applies to #1. You want to modify the --max-old-space-size for #2, which uses the NODE_OPTIONS environment variable instead.

#3

Finally :smile: