Methods runs the server out of memory

help
1

Hi,

I wrote some server side code that analyses data. The code is rather heavy, with lots of loops.

The client is sending the data to the server, functions run in about 500ms. Server seems to handle that well, but then a few seconds after I get a SIGKILL (server has 1GB memory) and the log on the server show me it closed meteor because it was out of memory.

It seems that the action of returning the transformed data is the heavy part that makes the memory usage burst, not the calculations. The largest object I managed to return is 65 millions UTF8 characters. above that it seems to go wrong.
Is that something that is known to happen ? Is there any optimization I can do on meteor side ? Or is it just too big ?

my method below. Client side I simply do a Meteor.call and get the result in the callback.

Meteor.methods({
‘analyzeModelStructure’:function(data,lists,email,IDs){
try{
let result = analyzeModelStructure(data,lists);
console.log('analysis returned ’ + moment().unix())
return result;
}catch(err){
console.log(err)
}

}
})

2

We probably won’t be able to help a lot without seeing the code in analyzeModelStructure. The fact that it returns and then crashes might indicate that you have a promise you aren’t waiting for. Or some other async work going on which is where the memory bottleneck is.

3

You have two high-level possibilities for the error:

  1. Something going wrong inside analyzeModelStructure(), as suggested by @znewsham
  2. Memory limitations when trying to return large amounts of data to the client.

Standard debugging practice is to isolate each of these possibilities to test them independently:

  1. Try calling analyzeModelStructure() as normal but only return the first 100 or 1000 bytes of the result…
  2. Try skipping analyzeModelStructure() completely and instead return a random string of “65 millions UTF8 characters”

That should narrow it down a bit…

1 Like
4

I’m guessing when you return the result, Meteor is cloning it before sending it over DDP.

The easiest solution would be to upgrade the server to have more memory, Meteor recommends a minimum of 2GB anyway

Otherwise, you could create a HTTP endpoint with webapp to bypass the cloning

5

thanks all for reply.

There’s no promise within the analyseStructure function, I’m pretty confident that issue doesn’t come from there.
I drastically reduce the size of the object returned (I used to run this on the client so I didn’t care much).
No it seems to fit on my heroku dyno with 512 MB of ram…