Process out of memory on long running task


#1

Hi,
I have an out of memory problem with meteor (current meteor 1.4), and I’m not sure about the cause.

I write a lot into mongo collections (query a third party service and process/store the data) and output a lot of lines to the server log. My UI uses collections as reactive sources and displays for example the current count or the last processed job.

But what part most likely caused this?

The last log output on server side is:

I20161104-08:23:13.733(1)? CLUSTER PROCESSING FOR ID FMBNSPB_qtXAQkPykD9Rqr IS FINISHED
W20161104-08:23:56.473(1)? (STDERR) FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - process out of memory
I20161104-08:23:56.475(1)? 
I20161104-08:23:56.475(1)? <--- Last few GCs --->
I20161104-08:23:56.475(1)? 
I20161104-08:23:56.475(1)? 51027283 ms: Mark-sweep 1378.2 (1453.7) -> 1386.9 (1453.7) MB, 1676.1 / 0 ms [allocation failure] [GC in old space requested].
I20161104-08:23:56.475(1)? 51028955 ms: Mark-sweep 1386.9 (1453.7) -> 1386.9 (1453.7) MB, 1672.8 / 0 ms [allocation failure] [GC in old space requested].
I20161104-08:23:56.475(1)? 51030714 ms: Mark-sweep 1386.9 (1453.7) -> 1376.9 (1453.7) MB, 1758.9 / 0 ms [last resort gc].
I20161104-08:23:56.476(1)? 51032474 ms: Mark-sweep 1376.9 (1453.7) -> 1378.2 (1453.7) MB, 1759.9 / 0 ms [last resort gc].
I20161104-08:23:56.476(1)? 
I20161104-08:23:56.476(1)? 
I20161104-08:23:56.476(1)? <--- JS stacktrace --->
I20161104-08:23:56.476(1)? 
I20161104-08:23:56.476(1)? ==== JS stack trace =========================================
I20161104-08:23:56.476(1)? 
I20161104-08:23:56.477(1)? Security context: 0x1bf8c6fb4629 <JS Object>
I20161104-08:23:56.477(1)?     2: /* anonymous */(aka /* anonymous */) [/home/phoenix/.meteor/packages/ddp-server/.1.2.10.1737qen++os+web.browser+web.cordova/npm/node_modules/sockjs/lib/transport.js:355] [pc=0x3f4dc5a2877c] (this=0x1bf8c6f041b9 <undefined>)
I20161104-08:23:56.477(1)?     3: doSendBulk [/home/phoenix/.meteor/packages/ddp-server/.1.2.10.1737qen++os+web.browser+web.cordova/npm/node_modules/sockjs/lib/transport.js:~351] [pc=0x3f4dc56b...
I20161104-08:23:56.477(1)? 
=> Exited from signal: SIGABRT

Because it happens in transport.js it looks to me like server-client communication (changes in collection that are pushed to the client?!?) caused that nodejs died. Any suggestions? What can I do about this?

best
Sebastian