MongoDB hosting with oplog

Hi all,

what MongoDB hosting provider with oplog and scalability can you recommend? MongoDB Atlas dedicated instance is - atm - a bit too much for my needs, and the serverless plans do not provide oplog access.

Just run your own server it costs like $5 to $10 a month and you have full control, it’s not hard at all. Just log in with ssh and run a few commands. Or you can hire an admin to do it for you for under $100

1 Like

IIRC atlas also provide shared instances, M2 & M5 which are pretty cheap and provide oplog access.

2 Likes

You can run it in your own server or you can use something like Scalegrid
Check out their pricing page for MongoDB: Database Hosting Pricing (2022) | Choose your Plan - ScaleGrid

1 Like

Thanks all for your answers. @marklynch I‘m not sure whether oplog is possible; guess I‘ll just give it a shot and play around a little bit.

The most important thing to consider when selecting a DB host for your Meteor project is to have that host in the same data center with your Meteor server. You either get another server in the same VPN as your Meteor servers or, for the AWS case, select a region that makes sense for your business and find a DB host that allows you to bring your own cloud (but there are not too many). If you want something rather small/medium and cost efficient, Digital Ocean could be your partner for both Meteor and DB

2 Likes

You should run your db on the same server so it’s contacted via localhost with zero network latency. Unless you do 100k visitors a day you have no use for a seperate db server and it’s total overkill like putting a 6Lv8 on a small riding lawn mower. No point at all.

Meteor and DB server plus some custom proprietary software running a server for your company internal use, and other services like company email, voip, crm, task manager and game server or torrent downloader can all be run on one machine for totally standard $20 a month vps instance and have better performance than any subscription cloud crap

1 Like

Ok, right now I have a 2 core virtual machine running mongo and one node instance; things are going really well, but right now i have roundabout 1000 users, and this will eventually expand. Really liked the idea of scaling easily, but I guess with a local replica set and a bigger machine / several small machines this could work as well.

For “self-hosted” mongodbs: Have you any recommendations for a nice mongodb admin tool? Came across dbKoda, which looks nice but doesn’t seem to be maintained anymore…

Scaling is really easy with mongodb, you won’t need replicas and sharding until you reach millions of users to be honest. It can handle a high load easily. You must monitor ram overall because memory is the biggest constraint of mongo. It will automatically scale to the amount of ram you have installed on the machine and use it all up so to give the best throughput, unless you modify it to not do so. So by default it’s really easy.

Set a machine up, open top and watch it run. If memory is getting eaten up and hitting disk going into swap and the cpu is heating up - you need more power.

So increase the ram but the crucial caveat here is to make sure you always stop your mongo service do not just let the admin shut her down because when you bring it back up - mongo wont start without running repair on your dbs and it can be a bit of a stressful situation when you have your application down. So always make sure you have stopped the process via service mongod stop before issuing a shutdown -h now or similar command.

Once it’s back online with the added ram or cores / whatever you were doing, if you stopped mongo before shutdown it will come back online and be all good. Otherwise you must run something along the lines of sudo mongod --config /path/to/your/mongod.conf --repair mongod when the service is not started.

Mongodb doesn’t have great admin tools really the CLI is ok and that’s what I use. For a GUI you can try Robo3T although it’s lacking and no where near what you can get for MySQL sadly. There are many other tools including official ones, in my experience they all suck and are a hindrance. But maybe you get better usage.

The only tools I have ever needed are mongoexport, mongodump and mongoimport and occasionally mongotop - the whole collection are here:

Hope that helps!

1 Like

Hi @bratelefant, another thing to consider when operating in Europe, or with at least one European client / user or in other countries with similar legal standards. Consider that on a self hosted MongoDB on “some cheap Ubuntu”, you cannot make proof of compliancy with GDPR. Here is some straight to the point reading: GDPR Compliance — MongoDB | MongoDB

1 Like

Replica sets are not only about scaling, even though reading from secondaries etc. helps with that, too. They are about redundancy. If your $5 VPC goes down for whatever reason, you lose all database access. It won’t matter of course, if your app is also running on that same $5 VPC. But ideally, you’d use a multiple of those $5 VPCs and a load balancer to create a resilient, redundant setup.

This is FUD. You don’t need to have “proof” that you’re “GDPR compliant” AFAIK, even though the GDPR as a whole is a horrible mess. You do need to be mindful who has access to your users’ data though and where it resides, and to have good security & privacy measures in place. You can achieve all that using “cheap Ubuntu”, though.

It’s true that anyone can buy a $5 VPC and run a few commands to install MongoDB. It’s not very different from installing any package on Ubuntu for example. Setting up MongoDB in a robust way takes time & effort though. Creating a properly optimised, geographically distributed, encrypted, properly secured replica set with private, TLS-only connections, certificates and whatnot with proper monitoring & alerting is a slightly more demanding task.

It boils down to: Do you need a secure, high availability database setup that can withstand servers going down, or do you just need less secure storage for non-critical & non-personal (so not real peoples’ emails, names, ip’s etc) data that hopefully won’t die on you too often.

1 Like

So setup two database servers and have them run on round robin it’s cost $20 what’s the big deal here bro?

Securing your server is always an essential part of running a web service and is important regardless, if you are using secure data like that you should have it encrypted in the office LAN and not connected to the internet unless via an authorized API - the government forbids using a cloud service due to these reasons and payment gateways use token based authentication to mitigate it for PCIDSS

These may help you!

To help set up a MongoDB server…
https://jamesloper.com/mongodb-setup-guide-ubuntu-20

To create the right users & oplog read only user…
https://jamesloper.com/complete-users-setup-guide-meteor-mongodb

When your client wants you to make proof of compliancy you don’t say “FUD”. That only works here in the forum …

For sure, by all means, please your customers and make 'em happy. But you can definitely setup your own MongoDB cluster and remain fully “GDPR compliant”.

Btw, you can also use Atlas and successfully violate GDPR laws.

1 Like

Ok I think the better word is “certified”.

Thanks for the great responses to my questions. Currently I’m running mongo on a virtual linux machine, oplog (and, hence, replicaset) activated, which is fine for sure, but “hosted” mongo db solutions have some nice add ons, like getting automated backups, rollback, dashboard, auto scaling, all kind of handy stuff for profiling your queries, indices etc.

Hoped to be able to fill that gap between “self hosted” and “outsourced hosted + cool stuff” by using some nice admin tool, but dbcoda seems to be not actively maintained, and other tools like robo3t seem to be proprietary…

Check out Compass, they’ve been really active developing it of late. Lots of nice improvements.
Others have spoken highly of Mingo which is on my list to try out.

1 Like