Hey guys!
I’m looking for a Celery alternative for my Meteor app. I’ve used job-collection in the past, but I had issues with jobs not running on time, or being skipped, or being run multiple times (on Galaxy when scaling up), etc. I moved out job processing to a separate Flask app because of this, but ideally I don’t want to keep doing that.
What solution are you using for this problem?
For what it’s worth:
Have a look a https://github.com/apache/incubator-airflow its not meteor based but it looks really impressive and can run python scripts. I’m still trying to get some time to spin it up so don’t have more details.
https://medium.com/airbnb-engineering/airflow-a-workflow-management-platform-46318b977fd8
This looks really neat! I’m also looking for failed job retries without restart, got any ideas for that?
Looks good, but will be overkill for my use case.