Howdy all,
TL;DR - What’s the best way to serve an HTML template without impacting my website’s Meteor functionality?
Background:
I’ve created a simple HTML landing page product on my platform: SubmitHub
Unfortunately, a few nefarious actors have taken advantage of these, and in the latest incident, over 45 million requests were sent within a few minutes.
Unsurprisingly, this brought all my containers down.
@philippeoliveira did an excellent job helping bring everything back online, but I’m wondering if there’s a more efficient way of handling these types of requests that don’t actually need any of the Meteor functionality.
Current approach:
Here’s how I handle the requests:
const cookieParser = require('cookie-parser');
const Bottleneck = require('bottleneck')
const limiter = new Bottleneck({
minTime: 200, // minimum time between requests
maxConcurrent: 1, // only process one request at a time
highWater: 50, // maximum queue size
strategy: Bottleneck.strategy.BLOCK // blocks excess requests beyond highWater
})
WebApp.connectHandlers.use(cookieParser())
WebApp.connectHandlers.use('/', function(req, res, next) {
const host = req.headers.host
const redirect = (href) => {
res.writeHead(301,{
'Location':href,
'Cache-Control':'private, no-cache, no-store, must-revalidate',
'Expires':'-1',
'Pragma':'no-cache',
})
}
if (req.url.startsWith('/link/')) {
limiter.schedule(
Meteor.bindEnvironment(() => {
const html = linksRoute(req)
res.statusCode = 200
res.end(html)
})
).catch(err => {
// Send 429 if queue limit is exceeded
res.writeHead(429, { 'Content-Type': 'text/plain' })
res.end('Too Many Requests')
})
} else {
... // code continues
Then inside the linksRoute function, I’m getting my data like this (simplified version):
linksRoute = function(req,id) {
const cache = getLinksCache(id)
let link = false
if (cache) {
link = cache
} else {
link = Links.findOne({id})
if (link) setLinksCache(id,link)
}
let html = Assets.get('link.html')
html = html.replace('title_placeholder',link.title)
return html
}
To reduce hits on Mongodb, I cache the result of link:
const linksCache = {}
// 600,000 ms = 10 minutes
const LINKS_CACHE_TTL = 600000
getLinksCache = function(id) {
return linksCache[id] ? linksCache[id].data : null
}
setLinksCache = function(id, data) {
linksCache[id] = { data, timestamp: Date.now() }
}
// cache gets invalidated based on TTL
Question:
Is there a more efficient way to serve these kinds of requests to avoid my main Meteor app from getting wrecked when someone tries to spam me?
Many thanks for your time!