How to minimize server load for a mostly read only project?

help
1

Yes yes, Meteor is all about reactivity - but for various reasons, we’re using Meteor for a mostly read-only project, that does not need reactivity. What are the best practices to reduce the server load, and maximize # of concurrent users, for a mostly read only project?

Would not using the oplog put less strain on the server?

Is there a simple way to use subscriptions without reactivity? I could try to change everything to method calls, but that would involve quite a bit of work :).

Any other tricks?

1 Like
2

I see where your coming from: sometimes we use Meteor just because we can’t be bothered to set up the tools for a static site (e.g., writing a gulp file). Firstly determine that there is even a problem. Here’s what we do:

  1. Create a small Selenium program that trigger the parts of the site you’re profiling. E.g., post nonsense comments on something.

  2. Open ten instances on a different machine to the server and look at the CPU and memory usage on the server (I use htop). Note: look just at the node and mongo processes. Note 1: you’re looking for the difference between the usage with no clients and 10 (so you can estimate the incremental cost of another concurrent client).

  3. Look at the server you’ll be deploying to (for us a basic linode). Take the ratio between your dev server CPU and memory compared to production. Divide again to get a rough CPU/Memory% per concurrent user, multiply up for expected concurrent users. If the result is more than 100%, see below.

If your site can be partitioned into static and dynamic, could you have something like nginx serve the static part then hand over to Meteor once they enter the dynamic area?

Reactivity is client-side only, so it doesn’t stress the server. Subscriptions, methods calls, DDP ping, server-side setIntervals, server-side observers, DDP connections, and MongoDB connections are all the things that cause server-side work (that I know of).

An easy way to estimate how much work a client is making the server do is to (in Chrome) open up the developer console, select the Network tab, select the WebSocket and watch the DDP messages come and go. The more messages, the more work the server is probably doing.

  • Reduce the number of subscriptions and increase the specificity (only return the documents and fields you need client-side).

  • You can’t really get around method calls: if you need the server to do something, you need a method call. Throttle calls, don’t call again until the previous has returned. Don’t return a result that doesn’t get used.

  • The DDP ping uses almost nothing.

  • You shouldn’t need server-side setIntervals or observers if your content is static. Observers can sneak up on you: I borked our performance last week by switching from a custom callback system to observers (switched back now).

  • This is extreme but if your site is really really static, you could disconnect the DDP connection client-side. This will disable hotcode reload: Meteor.connection.disconnect()

  • If you’re not using MongoDB, set the MONGO_URL=nope.

2 Likes
3

Really helpful, thanks!

Looks like ditching subscriptions is indeed the way to go.

Good tips on the CPU profiling - we’re using Kadira - I think I remember them offering some sort of build in profiling. I’ll give that a look.

Thanks again!

4

And you believe that’s less work than writing a gulp file?

5

Sounds like use-case for a a static site generator. We’ve moved a lot of our sites over to Netlify.

Building a Static CMS >"Now we have a full fledged CMS and we didn’t even have to build most of it. "

6

Try using this package for any subscriptions you don’t want to be reactive https://github.com/andrejsm/meteor-static-subs

1 Like
7

This is exactly what I’m looking for - better than method calls because it works with minimongo, which gives us access to local querying.

I’ve submitted a pull request, check it out when you have time.

8

don’t use mongo, just hardcode everything, disable websockets