The Missing Link in Meteor's Rate Limiter


#1

Hey all,

Inspired by a recent topic in the forums, I spent some time poking at Meteor’s rate limiter.

I noticed that while method and subscription requests can be rate limited, WebSocket connection requests cannot. This means that a malicious user could potentially flood a server with WebSocket connections, effectively denying access for other users.

For more information, check out my recent blog post. The surefire fix for this issue is to move your application behind a proxy layer like nginx, which can be used to limit requests to your WebSocket endpoint.

Other potential fixes could possibly include adding a new type of rate limiter event, "connection", and letting developers define their own connection rate limits. I haven’t done this, so I’m not sure how effective it would be.


Any advantage to using CloudFlare in front of Galaxy-hosted app?
#2

I think this is something everyone should be doing in production anyway, for example if you are using SSL you need something to terminate the SSL connection in front of Meteor. I think rate-limiting connections should be the web server or hosting platform’s job.


#3

I completely agree. It’s definitely something to be aware of, though.


#4

Is there anything planned for galaxy to solve these kind of problems? Or what is the current recommended way of facing those attacks under galaxy?


#5

Sorry, I’m not a galaxy expert - I’ll see if someone else can answer!


#6

Now that it’s fully available (including free plans), you could always look into integrating CloudFlare’s WebSocket DDoS protection.


#7

Thanks @hwillson , already had a look at CloudFlare websocket options, but wans’t sure if advanced plans were needed for websockets DDoS protection (The link from your website takes to Advanced DDoS Protection Service which is not included in free and pro plans).
Also, since no one else knows better DDP than MDG themseIves I thought maybe they were planning on something for Galaxy in order to mitigate possible attacks to their infrastructure.


#8

Yes, their website is a bit of a mess when it comes to WebSocket details. Take a look at their Plans page and scroll down to the Free/Pro/Business/Enterprise comparison chart. WebSocket support is included with all plans.

Definitely - I’d love to hear about this as well!

EDIT: I just found a way better link about their WebSocket support for different plans, on their support site.


#9

Yes I also have seen that, but I think there are 2 different features, one is websocket support, which means that you can open a websocket connection through CloudFlare servers (which is great and helps saving a lot of space in communications by not replicating headers) and a second feature, which is limiting the amount of websockets connections opened (an anti DDoS feature for websockets).
In that sense, I think that the second is only available with the Advanced DDoS Protection Service, and has to be configured, because as you can see on your last reference:

If my site uses more concurrent WebSockets connections than CloudFlare expects, what will happen?
Immediately, nothing. Within reason, CloudFlare will allow occasional
spikes in usage beyond our guidelines, and we will not apply
unnecessary limits.


#10

@tcastelli Good points - I’ve opened a support request with CloudFlare to get more details. I’ll post back with their response.


#11

My question to CloudFlare support:

I’m reviewing your different plans and am having a hard time finding out how much DDoS protection you offer for WebSockets, at your different plans levels. Do you offer some form of WebSocket rate limiting on all plans for example? Your plans page just mentions that WebSockets are supported on all plans, but it doesn’t say what level of protection is offered at each level.

Reply from CloudFlare support:

All plans offer our DDoS protection, and the Business/Enterprise plan offers our Advanced DDoS Protection, detailed here:

https://www.cloudflare.com/ddos/

The websocket protection we offer is the same as our HTTP protection.

So Free and Pro plans get their limited DDoS protection, which consists of I’m Under Attack Mode. It has to be enabled/disabled manually but it does include WebSocket protection.

I’m working with a company that will likely be signing up for the Business plan shortly, so I’ll post back on my findings with regards to WebSocket protection at that level.


#12

Good stuff @hwillson. I haven’t used CloudFlare personally, but it sounds like it might be a solid fix for this problem too.

The interesting thing about the connection spamming problem is that it doesn’t require a Distributed-DOS attack to exploit. A single user on a single machine can generate enough connections through a simple node script to cause problems for an unprotected server.