[SOLVED] Meteor App doesn't respond to requests because memory usage increases without stopping

help
1

Hello, this morning (Central Time (CST)) my app it was working well, and some minutes ago it was increasing memory usage (at this moment has 1.011GB) and normally it was consuming 120MB. When I tried to call methods (with Meteorman DDP Client) or login, it takes like 7 seconds (returns with error because of timeout of the connection) and after that the session is disconnected.

image
image1799×792 63.1 KB

docker stats

CONTAINER ID   NAME      CPU %     MEM USAGE / LIMIT     MEM %     NET I/O         BLOCK I/O    PIDS
49e4eca38d44   crontab   0.00%     1.484MiB / 3.844GiB   0.04%     1.48kB / 0B     0B / 0B      1
1545bfa635cb   app       44.03%    1.103GiB / 3.844GiB   28.69%    132MB / 202MB   4.1kB / 0B   11

These are my meteor packages:

# Meteor packages used by this project, one per line.
# Check this file (and the other files in this directory) into your repository.
#
# 'meteor add' and 'meteor remove' will edit this file for you,
# but you can also edit it by hand.

meteor-base@1.5.0             # Packages every Meteor app needs to have
mobile-experience@1.1.0       # Packages for a great mobile UX
mongo@1.12.0                   # The database Meteor supports right now

standard-minifier-css@1.7.2   # CSS minifier run for production mode
standard-minifier-js@2.6.0    # JS minifier run for production mode
es5-shim@4.8.0                # ECMAScript 5 compatibility for older browsers
ecmascript@0.15.2              # Enable ECMAScript2015+ syntax in app code
typescript@4.3.2              # Enable TypeScript syntax in .ts and .tsx modules
shell-server@0.5.0            # Server-side component of the `meteor shell` command

tracker@1.2.0                 # Dependency tracker to allow reactive callbacks
static-html             # Define static page content in .html files
akryum:vue-component    # Vue-CLI template to publish components

meteortesting:mocha               # A package for writing and running your meteor app and package tests with mocha
mdg:validated-method
alanning:roles
accounts-base@2.0.0
accounts-password@2.0.0
socialize:user-presence
lacosta:method-hooks
peerlibrary:middleware
check@1.3.1
meteorhacks:async
xolvio:cleaner
meteorhacks:ssr
montiapm:agent
tunguska:reactive-aggregate
dburles:factory

versions

accounts-base@2.0.0
accounts-password@2.0.0
akryum:vue-component@0.15.2
akryum:vue-component-dev-client@0.4.7
akryum:vue-component-dev-server@0.1.4
alanning:roles@3.3.0
aldeed:collection2@3.4.1
aldeed:schema-deny@3.0.0
aldeed:schema-index@3.0.0
allow-deny@1.1.0
autoupdate@1.7.0
babel-compiler@7.6.2
babel-runtime@1.5.0
base64@1.0.12
binary-heap@1.0.11
blaze@2.5.0
blaze-tools@1.1.2
boilerplate-generator@1.7.1
caching-compiler@1.2.2
caching-html-compiler@1.2.1
callback-hook@1.3.0
check@1.3.1
coffeescript@2.4.1
coffeescript-compiler@2.4.1
dburles:factory@1.1.0
ddp@1.4.0
ddp-client@2.5.0
ddp-common@1.4.0
ddp-rate-limiter@1.1.0
ddp-server@2.4.0
diff-sequence@1.1.1
dynamic-import@0.7.1
ecmascript@0.15.2
ecmascript-runtime@0.7.0
ecmascript-runtime-client@0.11.1
ecmascript-runtime-server@0.10.1
ejson@1.1.1
email@2.1.0
es5-shim@4.8.0
fetch@0.1.1
geojson-utils@1.0.10
hot-code-push@1.0.4
html-tools@1.1.2
htmljs@1.1.1
http@2.0.0
id-map@1.1.1
inter-process-messaging@0.1.1
lacosta:method-hooks@1.5.4
launch-screen@1.3.0
livedata@1.0.18
localstorage@1.2.0
logging@1.2.0
matb33:collection-hooks@1.1.0
mdg:validated-method@1.2.0
meteor@1.9.3
meteor-base@1.5.1
meteorhacks:async@1.0.0
meteorhacks:ssr@2.2.0
meteortesting:browser-tests@1.3.4
meteortesting:mocha@2.0.2
meteortesting:mocha-core@8.0.1
minifier-css@1.5.4
minifier-js@2.6.1
minimongo@1.7.0
mobile-experience@1.1.0
mobile-status-bar@1.1.0
modern-browsers@0.1.5
modules@0.16.0
modules-runtime@0.12.0
mongo@1.12.0
mongo-decimal@0.1.2
mongo-dev-server@1.1.0
mongo-id@1.0.8
mongo-livedata@1.0.12
montiapm:agent@2.44.0
montiapm:meteorx@2.2.0
npm-mongo@3.9.0
observe-sequence@1.0.19
ordered-dict@1.1.0
peerlibrary:middleware@0.3.0
promise@0.12.0
raix:eventemitter@1.0.0
random@1.2.0
rate-limit@1.0.9
react-fast-refresh@0.1.1
reactive-var@1.0.11
reload@1.3.1
retry@1.1.0
routepolicy@1.1.1
service-configuration@1.1.0
sha@1.0.9
shell-server@0.5.0
socialize:base-model@1.1.7
socialize:linkable-model@1.0.6
socialize:server-presence@1.0.4
socialize:server-time@1.0.3
socialize:user-model@1.0.5
socialize:user-presence@1.0.4
socket-stream-client@0.4.0
spacebars@1.0.15
spacebars-compiler@1.3.0
standard-minifier-css@1.7.3
standard-minifier-js@2.6.1
static-html@1.3.2
templating-tools@1.2.1
tmeasday:check-npm-versions@1.0.2
tracker@1.2.0
tunguska:reactive-aggregate@1.3.6
typescript@4.3.2
underscore@1.0.10
url@1.3.2
webapp@1.11.1
webapp-hashing@1.1.0
xolvio:cleaner@0.4.0
zodern:meteor-package-versions@0.2.1

Anyelse have had the same problem?

Notes:

  • Server: Digital Ocean - Docker image: disney/meteor-base - 4GB , 2vCPUs
  • Node 14.17.1
  • Meteor 2.3.0
  • Mongo Atlas 4.4.6
2

Check for memory leaks. Get a memory snapshot feom your server and use chrome dev tools to check which variables are eating too much memory.

(search on how to check memory leaks from the server)

1 Like
3

It was very weird, yesterday at night I turned off the server and today I turned it on again and ran correctly (95MB of ram). Yesterday, I restarted the server and immediately memory usage was increasing very fast with only 3 connections which weren’t calling methods.

After some hours… I realized that maybe it was the montiapm-agent (meteor package), since I removed it and the app hasn’t had that issue again (so far). Before of this, I downgraded to Meteor 2.2 and it happened the same (memory usage increased), so that’s why I wanted to try by removing montiapm, since it was the only thing that runs at the starting of the application and maybe was the cause of memory usage. So, on the next days I’m going to upgrade again to Meteor 2.3.

2 Likes
4

After some days, unfortunately I got the same problem, however, this time was without the agent of Monti APM. So, I was finding out the reason why this happened and now I can suspect that it was Mongo Atlas. I was using the free tier which brings RAM shared and 512MB of storage. It was weird because some days works perfectly but one day can be very slow (guess is due to RAM shared) and this affect to my meteor app by having issues like logging in and response times. I am almost sure that the problem is caused by the database of Mongo Atlas because when I got that problem in the application and I access to the database (through Robo 3t), the connection to it is slow too. Unlike when the montiapm agent was present, this time only increase the memory until certain number (240MB) whereas with agent increased without stopping (exceeded 1.6GB of memory), so, maybe the Mongo Atlas problem also affected to the agent and it ended up affecting the application in a worse way.

Before of this, I used to have mongodb in a docker container in the same machine where it was also hosted the meteor app container, however, because I had to add transactions to my app, I had to acquire a Replica Set instance of MongoDB, and Mongo Atlas seemed to be a good option. So, some days ago I configured a Replica Set instance of MongoDB in docker containers and it has been working well so far. I use a droplet of Digital Ocean with 4GB of RAM, 2vCPUs and 80GB of SSD, and I think that is a good hardware to run a test server and pay 20 dollars per month for it.

Shared a docker compose file example with the configuration of a replica set instance for a meteor app:

docker-compose.yml

version: '3'

services:
  app:
    container_name: app
    image: meteorapp
    restart: on-failure
    build:
      context: .
      dockerfile: dockerfile
    env_file: docker.env
    ports:
      - 80:3000
    depends_on:
      - mongo-primary
      - mongo-secondary
      - mongo-arbiter
    volumes:
      - $PWD/app-files:/opt/app-files
    networks:
      - meteor-net
  ssl-proxy:
    container_name: ssl-proxy
    image: justsml/ssl-proxy:latest
    depends_on:
      - app
    environment:
      - HTTPS_PORT=443
      - SERVER_NAME=www.yourdomain.com
      - UPSTREAM_TARGET=app:3000
      - CERT_PUBLIC_PATH=/certs/fullchain.pem
      - CERT_PRIVATE_PATH=/certs/privkey.pem
    volumes:
      - /certs:/certs:ro
    ports:
      - 443:443
    networks:
      - meteor-net
  mongo-primary:
    container_name: mongo-primary
    image: bitnami/mongodb:4.4.7
    restart: always
    ports:
      - 27017:27017
    environment:
      MONGODB_ADVERTISED_HOSTNAME: mongo-primary
      MONGODB_REPLICA_SET_MODE: primary
      MONGODB_ROOT_PASSWORD: password123
      MONGODB_REPLICA_SET_KEY: replicasetkey123
    volumes:
      - $PWD/data:/bitnami
    networks:
      - meteor-net
  mongo-secondary:
    container_name: mongo-secondary
    image: bitnami/mongodb:4.4.7
    restart: always
    depends_on:
      - mongo-primary
    environment:
      MONGODB_ADVERTISED_HOSTNAME: mongo-secondary
      MONGODB_REPLICA_SET_MODE: secondary
      MONGODB_INITIAL_PRIMARY_HOST: mongo-primary
      MONGODB_INITIAL_PRIMARY_PORT_NUMBER: 27017
      MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD: password123
      MONGODB_REPLICA_SET_KEY: replicasetkey123
    networks:
      - meteor-net
  mongo-arbiter:
    container_name: mongo-arbiter
    image: bitnami/mongodb:4.4.7
    restart: always
    depends_on:
      - mongo-primary
    environment:
      MONGODB_ADVERTISED_HOSTNAME: mongo-arbiter
      MONGODB_REPLICA_SET_MODE: arbiter
      MONGODB_INITIAL_PRIMARY_HOST: mongo-primary
      MONGODB_INITIAL_PRIMARY_PORT_NUMBER: 27017
      MONGODB_INITIAL_PRIMARY_ROOT_PASSWORD: password123
      MONGODB_REPLICA_SET_KEY: replicasetkey123
    networks:
      - meteor-net
  crontab:
    container_name: crontab
    image: crontab
    privileged: true
    restart: always
    depends_on:
      - app
    build:
      context: .
      dockerfile: cron-dockerfile
    volumes:
      - ./backups:/opt/backups
    networks:
      - meteor-net
networks:
  meteor-net:

This docker-compose file covers the following aspects:

  • Cron container to do backups of the database.
  • Nginx container to add HTTPS (and consequently WSS) to node apps. (Remember to add force-ssl package to redirect automatically to HTTPS from HTTP)
  • Containers for a replica set instance of MongoDB (More info here).
  • App service is using a dockerfile from Disney.

If you need more details, do not hesitate to ask.

1 Like