Memory spikes as soon as site loads

[rogue_ai_0]

I’ve run into a problem where memory usage starts increasing nonstop as soon as I load the index page. Using Chrome DevTools I can see that when I start the server, the memory is fine sitting at around 90MB. However as soon as I load the site at http://127.0.0.1:3000/, the memory usage starts increasing to the node heap limit (which I’ve increased to 8GB) and it crashes.

To confirm this isn’t an issue specific to my app, I spun up a fresh meteor app, and populated the db when the server started inside Meteor.startup like so:

  for (let i = 0; i < 1000000; i++) {
    TasksCollection.insert({
      'test': '1231aoidwjdoiajwdoiajwd oaijdo awidja owidj aowidj aowidjaowidj aowidj aowidj aowidj aowij'.repeat(10)
    })
  }

Indeed I saw the same behavior. Memory was fine until I load the web page and it shot up to 1.8GB.

Can anyone provide insight as to why Meteor behaves like this and what I can do to prevent this? It seems unlikely the framework was be designed without this level of db traffic in mind. Thanks!

---

If it helps, this is from a snapshot of the heap, these strings are articles that my app processed in the past (not on this particular load) and you can see they’re getting loaded from some _IdMap object that rolls up to something called PollingObserverDriver then ObserverMultiplexer

[alawi]

Do you have pub/sub on your TasksCollection?

[rogue_ai_0]

I don’t - it’s a vanilla app and that loop I posted is basically the only code other than the TasksCollection and a boilerplate project.

[alawi]

Do you have any queries on that collection? you might want to share your project.

Meteor has oplog enabled by on the dev server, and it seems you have some query that is consuming a lot of memory. Also, as you might know, the code above should not be part of any app, you want to use bulk insert when you have such a large number of records. This code will also stress your MongoDB server.

[rogue_ai_0]

Ah ok, I’ve found the issue. Removing the autopublish package did the trick: meteor remove autopublish

Documented here and mentioned here: “Until now, we have worked assuming the entire database is present on the client, meaning if we call Tasks.find() we will get every task in the collection.”

So it sounds like this package, which is intended for development only, attempts to load the entire database onto the client, or something to that effect. That’s what I get for not finishing the tutorial .

[peterfkruger]

Oh well, things like that happen

[alawi]

Good mistake actually, now you have better understanding of pub/sub and its performance cost.