I am hitting a problem where I need from time to time to write jobs to change datas on my MongoDB.
Currently I need to redeploy on galaxy after testing the code locally.
I am looking for a way to have a separate instances where I can simply deploy my meteor app without annoying my users so that I can have my new jobs run immediately.
I am using wildhart/meteor.jobs ro run jobs so maybe I can set the serverId of the queue to be the one of the server I will be deploying.
I am pretty sure that everyone have this problem ( writing codes for chore purposes ) but I do not find any good solutions
Nosqlbooster has some scripting ability - you can basically write js code which manipulates your data without the need for deployment overhead. You can develop your script against a copy of your database
We also started doing something similar to @rjdavid recently. We had tried various other queues in the past but Bull is the one that feels really well thought out and is in active development. Bullboard is also nice for keeping tabs on them.
We kept a single app and then assign roles like webapp or jobs to different instances. Ideally we’d have separate apps which could be launched in parallel with rollbacks if any of them fail - but we run on elastic beanstalk and the options for orchestration are limited.
With the updates to Monti planned to automatically instrument job queues this should become even nicer.
Set up a separate staging environment on Galaxy could help - you can test job changes there before deploying to production. The wildhart/meteor.jobs package should let you use a different server ID for the staging queue