Self hosting mongodb cluster?

deployment
1

Hi,

I have a bunch of AWS credits, and I’m looking to move severak small production dbs off of Compose.io to save some money.

Does anyone have a configuration for a mongo cluster to run on AWS servers to get me started please?

My needs are

  • Replica set (2 servers will be adequate)
  • Backups

I could probably work out how to do this myself (eventually), but I suspect I’m not the first person to want this.

Any help would be appreciated

Mike

2

I’ve not self-installed a complete AWS MongoDB cluster, but you may like to take a look at Amazon’s self-install quick-start or this guide to installing a MongoDB replica set on EC2.

I guess it comes down to how comfortable you are with doing your own devops.

3

I have managed mongodb in professional / enterprise environment, as much as I would like to say its easy, and it doesn’t take a lot of maintenance, I think managing servers / backups vs paying few dollars a month for hosted database is much better fit in my opinion.

However if you decided to go old school and host things yourself I would recommend to use AWS light sail, and maybe pick 3 x $5 nodes to create a replica set. (You will get cheaper pricing for EC2 quality VM). Lightsail block storage comes with 99.99% uptime.

As for backups you probably end up with a cron job type of backup, as I’m not aware of any decent open source backup / restore tool.

4

Make life easy on yourself and run these as single-node replicas, or host one cluster for all your databases.

5

It really is dead simple to setup. Just head over to MongoDB docs Replica Set Deployments and get on with it.

There’s a few gotchas or things worth mentioning:

  • Remember to setup proper authentication and firewall rules, don’t bind your mongod to a public interface!
  • Consider using a RAID HDD setup for the database files for speed and increased reliability
  • Only SSD drives!
  • Consider distributing replica set nodes across different data centers for reliability
  • Backups can be as easy as a CRON job, or you can use something like https://github.com/kontena/mongo-backup
  • Backups are the easiest to locate to a different data center, or even different continent, so you should do it
  • Create a system that alerts you if backups are not taken or if they are corrupt, make sure you can actually restore the backups!
  • Setup a free MongoDB Cloud account and install their monitoring agent. You get useful insights into your DB’s operation and alerts in case of nodes going down or resources going low.
4 Likes
6

https://jamesloper.com/mongodb-cluster-setup-with-ssl

While hosted mongodb solutions exist, they aren’t for everyone.

Set up your primary server

In this example, the primary is db.example.com. Start by creating a data directory on a mounted volume. Remember, WiredTiger has the best performance on XFS.

ssh root@db.example.com
apt update && apt upgrade
mkdir /mnt/db/mongodb
chown -R mongodb /mnt/db/mongodb

Install mongodb:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4
echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list
apt update
apt install -y mongodb-org

Create a bare bones configuration just to get it functioning. Edit /etc/mongod.conf.

storage.dbPath = /mnt/db/mongodb
net.bindIp = 0.0.0.0

replication:
  replSetName: rs0
  oplogSizeMB: 100

dbPath is now set to the path you made. 0.0.0.0 makes your server accessible via the web.

Restart mongod and register it with systemctl to start when your server boots:

service mongod restart
systemctl enable mongod

Create the admin user

In this section, you will configure your database to only accept username & password authentication. Create an admin user:

mongo
use local
rs.initiate()
use admin

db.createUser({
  user: "admin",
  pwd: "password",
  roles: [ { role: "root", db: "admin" } ]
})

Edit /etc/mongod.conf again! Specify a security policy. This one is absolutely required if your server is bound to 0.0.0.0! Once done, you must authenticate every time (even through localhost!)

security:
  authorization: "enabled"

Restart mongod:

systemctl restart mongod
systemctl status mongod

Try it out! You want to make sure you can log in!

mongo mongodb://admin:password@db.example.com

Enable SSL

First, you need to have a minimal web server running to connect to LetsEncrypt. Install NGINX and Certbot:

apt install nginx
add-apt-repository ppa:certbot/certbot
apt update
apt install software-properties-common
apt install python-certbot-nginx

Create the certificate for this host:

certbot --nginx -d secondary.example.com

Set up automatic creation of the SSL certificate:

cd ~
nano renew-mongo-cert.sh
chmod +x renew-mongo-cert.sh

Paste in this script:

#!/bin/bash
DOMAIN=secondary.example.com

certbot renew
newestFull=$(ls -v /etc/letsencrypt/archive/"$DOMAIN"/fullchain*.pem | tail -n 1)
newestPriv=$(ls -v /etc/letsencrypt/archive/"$DOMAIN"/privkey*.pem | tail -n 1)
cat {$newestFull,$newestPriv} | tee /etc/ssl/mongo.pem
chmod 600 /etc/ssl/mongo.pem
chown mongodb:mongodb /etc/ssl/mongo.pem
service mongod restart

Enable SSL in the mongo server configuration in /etc/mongod.conf:

net.ssl:
    mode: requireSSL
    PEMKeyFile: /etc/ssl/mongo.pem

Run the script for the first time:

./renew-mongo-cert.sh

Add the script as a cron job with crontab -e

0 0 1 * * renew-mongo-cert.sh

Reboot, and now you can connect through SSL. Note that the URL now contains the ssl param:

mongo mongodb://admin:password@db.example.com/?ssl=true

Adding a secondary

Follow the same process above to get mongo installed and configured, switching out db.example.com with your secondary’s hostname.

All nodes in a cluster must share the same secret key-- that’s how they authenticate with each other. We’ll make that file next. Do this step on both servers!

echo "your-secret-key" > /data/key

Edit /etc/mongod.conf

> security:
>  keyFile: /data/key

Then restart the mongo server again

service mongod restart

Next, log back in to the mongo shell.

mongo mongodb://admin:password@db.example.com/?ssl=true

The following mongo shell script is to have a secondary that your BI tools could beat up without affecting your primary. That’s why hidden is set to 1 and not 0. This ensures that the node can not become the primary in the event of a primary malfunction.

rs.add({host: "secondary.example.com", priority: 0, votes: 0, hidden: 1})
rs.slaveOk()

If you want to provide more hosts for your app to connect to, you instead want to make them visible. The minimum number of nodes in a cluster jumps from 1 straight to 3, otherwise a cluster of 2 can not perform a vote to decide which node will become primary.

3 Likes
7

FWIW the mgob project is pretty good for setting up backups for Mongo, with S3 backup support

More here: https://github.com/stefanprodan/mgob

1 Like