Hosting a Meteor app plus scraping bots

help
1

Hi guys! I’m making an app that will read data from MongoDB , provided by ten scraping bots(Scrapy).
I’ve made a prototype locally and everything works but i’m facing some issues when i need to deploy the whole system in a production environment.

-Firstly i have no idea where to host the bots.They need to run 24/7, so using my computer wouldn’t be an optimal solution, so i was thinking about a VPS.Have you any suggestions?

-Secondly, right now the bots are updating the data on Meteor’s MongoDB(i’ve set 127.0.0.1:3001)locally,instead when i tried to deploy the app using -meteor deploy, the bots were using the url provided by meteor mongo -U(the one that you can try to connect for one minute, after that time it expires).
My question here is: Should the bots update a separated MongoDB instead of the Meteor’s one? because using everytime meteor mongo -u with ten different bots it’s not a viable option.

-Third i was thinking to deploy on DigitalOcean at the moment,waiting for Galaxy Developer’s Edition.What do you think?

Thanks for the help and forgive me if there are any comprehension mistakes about the deploying process but it’s my first app :smile:

2

I would deploy to DO using mupx
Bots would end up in separate docker container
And mupx can also setup mongo in container for your app.

Deployed mongo does not change access rules.

And for your usecase I would be thinking about running mongogb in docker container also for developing purposes so bots can always connect to it with set user/pass

3

Thanks! I’ll let you know if i find any issues :smile: