Hi, very often we find clients struggling with long-running tasks affecting their connected users.
And we have tried a few approaches internally and we decide the share the approach we believe is the best one to solve this problem.
Your feedback is welcome!
Or you could use worker threads/web workers with something like https://threads.js.org/
If only it could be supported by the meteor build tool… 
I use cron.
Hi,
Great to have such documentation on the subject. For my understanding as I’m far to be a specialist in the field, this works if you launch multiple instance of the same app on meteor cloud but this only uses one CPU per instance which is great with meteor cloud (most of them have only one CPU right ?) ?
On the other hand threadJs mentioned above or GitHub - nathanschwarz/meteor-cluster: worker pool for meteor using node js native `cluster` module use the multiple CPUs of the same instance ? (which is better for some other environment)
These would really help:
Cron is part of the operating system, it’s very easy to use, here’s a user guide below
As for severless functions, this is a cash cow for Bezos and cloudt*rds, it still runs on a computer which is a server just you pay for it instead of using the free technologies that are available under gnugpl so you can just setup your own server and get unlimited access, or resell it to CTs that don’t know how to RTFM. It’s a knowledge economy
Cron and background jobs are two different things.
We were using the same approach where crons were running in the user serving container and then decide to migrate all the corns to separate services.
Here’s the blog that I have written about it. Hope this helps.
Interesting article, thanks for sharing.
Been using something quite similar, but we are not satisfied with the basic scheduler provided by aws. What are you using to meet all the scheduler requirements (specially timezone)? Is that a custom service?
We are using meteor schedulers (this is a separate service with no UI) for generating events.
And we are using this package to satisfy our timezone needs.
I use cron too for about 5 years without any problem.
For cron jobs, I’m using @vlasky’s excellent percolate:synced-cron. This way I can set up the cron job from inside my Meteor server code.
We’re using job-collections see simonsimcity:job-collection which allows to control how we handle failing jobs, how many jobs of the same type we run and much more. Using it for 6 years now.
This is an old article but it compares packages mentioned here: https://forums.meteor.com/t/jobs-worker-in-meteor-steve-jobs-vs-meteor-jobs-vs-meteor-workers-vs-synced-cron/44578
we used the exact same approach in some meteor apps.
this has the benefit that you can use existing code of your app.
system cronjobs are a bit trickier in pure meteor apps, as its harder to share code between the cron-script and the webapp.
in general, avoid running cpu intense tasks on the same process that does answer web-requests and instead run as separate worker-process
Seems to me the best architecture depends on the job. Big, monolithic actions will need scheduling or maybe threads. For stuff that loops – like batch I/O – we use streams. That slows the job down, but it doesn’t crush the server either.
We also put the bigger jobs on the admin server, where there are very few users.