So I have been utilizing Prerender.io for several of my sites, I have it set based on the advised specs form Prerender where my NGINX server determines if it is a bot and sends it to its pre render.
The issue is this is causing a bunch of blank pages to be pre-rendered as it seems random bots keeps trying to access random non existent pages on my site. I’m assuming these are auto bots looking for data or holes in the security, but how do I stop this from happening?
I know there are ways to check if a route exists with SSR but how can I do this with NGINX? One thing I was thinking was only going to pre-render if it was trying to access certain uri’s but that doesn’t seem scallable or efficient.
Has anyone else tackled this?
There are two things you can do here to prevent those pages from being cached in Prerender.io:
If the URLs being accessed only have different query string parameters, you can log into your Prerender.io account and go to the URL Parameters section. From there, you can tell us to ignore certain query string parameters that don’t affect the rendering of the page. That way, we wouldn’t cache URLs with those query strings as separate URLs in Prerender.io.
That should solve things but let me know if you have any other questions!