Tom,
we’re having a very database heavy app as well. We plan to use the $nearSphere in the future as well but for now we just paint locations on a world map using leaflet package.
I mentioned to you in another thread before, separate frontend and backend in separate apps first of all. That allows independent scaling which isn’t only very cost effective but let’s you react to heavy CPU or database processes that are unpredictable (meaning, user triggered).
For regular jobs use Lambda and monitor its performance so that you can increase CPU/memory when you need more performance. This can scale up to handle a lot of things.
Use MongoDb on Atlas, not only do they offer the tools but they also have the best knowledge to scale your MongoDb. It’s worth it.
Lastly, limit what data you send to the Frontend. By separating your apps, your backend or Lambda’s will do all the heavy database lifting together with MongoDb. Run aggregations whenever possible, so that most database work happens in database and not in your app (never a good idea). Then send only the needed results of your heavy database work to the end user, nothing less but nothing more.
We don’t use Redis Oplog, we have only a few pub/sub left and almost all our database queries are optimized. That results in performance improvements by several hundred percent and the app feels completely different to the end user, way more snappy.
Lastly, @radekmie as a performance consultant for Node.js apps already gave the best advices. Especially:
To be honest I never used any of the clustering solutions. For scaling, using Docker helps a lot - adding and removing containers is simple (both manually and automatically).
Check out AWS Fargate. Both our frontend and backend apps run as Fargate on AWS and we auto-scale them there. Cheap and works like a charm when performance is needed. Start with the smallest available configuration but ensure you have enough memory as you otherwise will see that you app just crashes with no clear indication what is wrong (it ran out of memory).
Hope this helps.