Hi,
Just sharing this here because I had quite some trouble setting this up.
Goal: use Digital Ocean droplets + load balancer, deploy with Meteor-up and get free Letsencrypt certificate.
Why? DO increased their 5USD droplets to 1GB, perfect for Meteor. And I like simple managed load balancers.
It turned out to be quite difficult because the order of steps really determines the success of getting Letsencrypt and your load balancer working. I have failed a lot of times so here’s the magic recipe:
Now set DNS of your domain to the IP of that droplet, make sure that the domain matches in mup.js
mup setup & mup deploy
Result
Should now have Letsencrypt and force ssl for the domain. Optionally check this with mup proxy logs-le
Step 2
Create a DO load balancer, USD20/month (but it’s managed, availability and easy to configure, imo better than nginx). Make sure it’s in the same region as your droplet!
Assign the droplet from step 1 to the load balancer
Set redirect https 443 to passthrough. This makes sure https traffic is redirected to your droplet, so encryption till your droplet, not till your load balancer. It also avoids needing to use DO’s DNS - I prefer Cloudflare.
Set health check to TCP port 443.
Now change the DNS of your domain to point to load balancer
Result
Load balancer should show droplet as healthy.
Your domain now points to load balancer and should redirect just fine, loading your app
Step 3
Now create a live snapshot of the droplet
Create a new droplet from the snapshot
Assign the droplet to the load balancer
(Here you can make as many droplets as you like)
(This snapshot step could potentially be skipped, I haven’t tried. But as Letsencrypt is already configured on the droplet of step 1, I feel this is a more safe approach)
Result
Load balancer should show all the droplets as healthy
Your domain now points to load balancer and should redirect just fine, loading your app
Step 4
Update mup.js to include the 2nd droplet
Now you’re ready to do mup deploy
Result
Your domain now points to load balancer and should redirect just fine, loading your app.
You now have a 40USD/month high-availability setup (hooray! ) that is easily scalable as you can just clone droplets and add them to your load balancer.
Oh and did I mention this is about half the price of Galaxy?
The only downside is that DO doesn’t support Ireland/Dublin as a region, so if you use mLab, that will decrease your performance a little as the closest is probably London. But then again, only AWS does Dublin (Galaxy is on AWS) so what can you do… For US your options are better.
Misc:
Check status for proxy ssl: mup status
Letsencrypt logs mup proxy logs-le
Clearing dns host cache in Chrome chrome://net-internals/#dns
@satya couple of questions about load balancer. Does it support sticky sessions? If yes, do you know which algorithm is used to distribute clients across app nodes (ip_hash or something else)?
No it doesn’t support multiple regions. Cloudflare’s load balancer can do it
Your nginx setup encrypts traffic until your LB right? So from nginx till your droplets it’s unencrypted, I’m not sure how secure that is even when using private ips?
And if your LB is a single droplet, then it’s a single point of failure right? If it goes down, your app goes down.
Yes, from nginx to app traffic not encrypted, but it is in the same private network. You can configure a firewall(it is free DO feature) to filter input/output traffic for your droplets, so only proxy can communicate with app servers.
Yes, it is single point of failure as well as have all infrastructure in one datacenter)
DO’s load balancers are managed, don’t depend on a single instance. But yes, a single data centre. I personally feel it’s worth the additional 15$, but it’s a personal decision I guess.
But, DO’s advice is to not decrypt SSL on your load balancer, but do pass-through SSL- also for internal traffic.
I’m curious how do you deploy if your firewall only allows traffic from your LB?
It does support sticky sessions based on LB’s own cookies which it stripes out during routing.
And it supports round robin and least connections routing strategies.
When balancing encrypted web traffic, there are two main configuration choices: SSL termination and SSL passthrough.
With SSL termination, SSL requests are decrypted at the Load Balancer and sent unencrypted to the backend. This places the slower and more CPU intensive work of decryption on the Load Balancer and simplifies the management of the SSL certificates. However, the traffic between the Load Balancer and the backend is unencrypted, which means it is visible to any neighbors on that network segment.
The other option is SSL passthrough, which sends SSL connections directly to the backend. Unlike SSL termination, the requests remain encrypted and the decryption load is distributed across the backend servers. However, the certificate information must be replicated on every server. In addition, you cannot add or modify the HTTP headers, so you may lose the client’s IP address, port, and other information contained in the X-forwarded-* headers.
We recommend SSL passthrough to secure traffic between the Load Balancers and backend Droplets.`
So what I read here is: don’t count on private networking for security!
As of July 18, 2018, DigitalOcean private networking isolates communication at the account or team level between Droplets located in the same datacenter region.
As I understood: only your(account or team) droplets can communicate with each other in private network.
Of course, you should enable firewall to prevent access from public. I allowed only ssh and proxy<->app communications.
But it is your decision to use SSL termination or passthrough.
Very nice write up of your setup, thank you very much.
I had a very similar setup about a year ago but ran into problems with the oplog features of Meteor. My servers had very high CPU usage and kept on crashing.
In the end I had to go with 1 server that had multiple cpu`s and high memory, after this the CPU usage was almost nothing and I had no further issues.
So my question…what did or are you doing to mitigate this? Do you have Oplog enabled?
if I want to use the droplet IP should I need to put it in place of example.com? I don’t have a domain yet and i’m not using ssl for now. I just want to change the default sites-enabled/default file to test my Meteor app. I guess that if I don’t have a domain name I need to use the default file, right?
I don’t understant how to config the nginx default file and the mup file. Can you please post your sample config files?
Hi I managed to setup correctly the load balancer on DO based on this great article but I’m having issues with the proxy_protocol and how I have to setup correctly nginx via mup so I can have the real IP forwarded on my droplets.
Anyone has faced this before and managed to do it?
Sorry to resurrect an old thread, but I’d really like to follow this approach as I am using exactly the same stack (DO + Cloudflare).
The only thing making me doubt is the documentation page on DO’s website. It has a warning message in the Passthrough section saying it is not compatible with Sticky Sessions. Any feedback on that from anyone using this setup?
) that is easily scalable as you can just clone droplets and add them to your load balancer.
