Meteor server is going down randomly


#1

I just migrated my application from Meteor 1.2 to Meteor 1.8.

Now my server goes down randomly, sometimes after 3 hours, and sometimes after 6 hours, its totally random.

When the server is down, my mobile apps or web app stops working, so I restart my server and after that it remains slow for a minute or two, and it became faster again.

When the server is down, at that time the node uses 105% of CPU usage.

So how do I debug this?


#2

Have you allowed it to run in developer mode for that amount of time to see if you can catch the bug?

Also, I set up a special collection called error logs, that I didn’t write any errors to go whenever they are caught in the code. It’s not ideal when running in production mode but I set up a few extra parameters that I sent to the log to let me know which file method in line number the error happened on at least according to where the error is thrown.


#3

@bmcgonag1 How can I run in developer mode? I used simple meteor --port 6000 command though.


#4

Ok, then this is what I call developer mode. When the server stops, what error is shown in the terminal?


#5

Now this is the strange thing, I am not getting any error at that time, but i did get these two errors in a while, but they occurred only once:

And

W20181227-11:32:23.594(1)? (STDERR) /home/myuser/.meteor/packages/meteor-tool/.1.8.0_1.12cbugr++os.linux.x86_64+web.browser+web.browser.legacy+web.cordova/mt-os.linux.x86_64/dev_bundle/server-lib/node_modules/fibers/future.js:280
W20181227-11:32:23.596(1)? (STDERR)                                             throw(ex);
W20181227-11:32:23.596(1)? (STDERR)                                             ^
W20181227-11:32:23.597(1)? (STDERR)
W20181227-11:32:23.597(1)? (STDERR) RangeError: Out of memory
W20181227-11:32:23.598(1)? (STDERR)     at processNext (packages/ddp-server/livedata_server.js:563:10)
W20181227-11:32:23.598(1)? (STDERR)     at unblock (packages/ddp-server/livedata_server.js:550:11)
W20181227-11:32:23.599(1)? (STDERR)     at packages/ddp-server/livedata_server.js:562:9
=> Exited with code: 1
=> Meteor server restarted

I dont know if these are related or not


#6

Nothing above it about Babel runtime, or bcrypt? How much RAM on your machine?


#7

No nothing,
Here are my npm packages list:

"dependencies": {
    "@babel/runtime": "^7.2.0",
    "apn": "1.7.5",
    "babel-runtime": "^6.23.0",
    "bcrypt": "3.0.3",
    "esendex": "0.1.8",
    "gcm": "1.0.1",
    "node-gcm": "0.14.4",
    "nodemailer": "2.0.0",
    "nodemailer-smtp-transport": "2.5.0",
    "qr-image": "3.1.0",
    "stripe": "^5.2.0"
  },

#8

My server RAM is 32 GB


#9

Only thing I see is Babel runtime twice, but probably not that.

So, I want to understand, when you say the server stops, do you mean it crashes, or it’s still running in the terminal, but the pages stop responding, or become really slow?


#10

Yeah the meteor instance keeps running, but it doesnt load page, i dont get any response from it


#11

Here are the ram usage:


#12

Nice rig, so memory definitely shouldn’t be the issue. Guessing that out of memory error was a 1-off. I wish I had more ideas. After updating to 1.8, did you run `meteor update —all-packages as well?


#13

Yeah I did that as well


#14

@bmcgonag1 Few years back I integrated Kadira plugin, which would help check the performance statistics and crashes etc, since it has been discontinued, so is there any alternate for that?


#15

Sure, there is one called monti apm. montiapm.com

I would also suggest you to do a cpu profile, maybe you can find more info on what is causing the issue there.


#16

@pmogollon How can I do the CPU profile? Can you give me some directions?


#17

My nodejs version is 8.14.x your 0.10.x may cause the problem.


#18

Actually I saw montiapm has a cpu profiler, not sure how good it is. Also you can check this one https://github.com/qualialabs/profile.

But what @minhna says might be the issue, didnt saw that. But how are you even able to run meteor 1.8 with node 0.10 when using meteor --port NNNN. That should use the node version meteor comes with.

@hassansardarlecodeur if you run meteor node -vwhats the result?

Also you shouldnt run a production app like that, you should build the app and then serve it. You can find more info here. https://guide.meteor.com/deployment.html#custom-deployment


#19

Here is the meteor node version:

meteor node -v
v8.11.4

#20

I am also getting this error time to time while moving in paginations:

Executor error during find command :: caused by :: Sort operation used more than the maximum 33554432 bytes of RAM. Add an index, or specify a smaller limit.