Steve Jobs: The Simple Jobs Queue That Just Works

michcoll · January 24, 2018, 6:16am

Hi.

This is great package. I just starred both your atmosphere and github pages.

With that said, I implemented this package but would like to change the polling interval to every six hours. I read the docs but am having trouble implementing
// Jobs.configure({
// timer: 21600 * 1000
// });

The code breaks when I include it with the Jobs.register and Jobs.run.

If you can provide insight on the best way to configure the polling interval it would be appreciated.

msavin · January 26, 2018, 9:13am

Try interval instead of timer

michcoll · January 30, 2018, 9:01pm

Thanks for the follow-up. I tested it at a shorter interval and it worked!

ivo · January 31, 2018, 7:19am

Dear Msavin,

thanks for the great package ! Some questions :

Jobs should be wrtten on server side ?
Do you think it is suitable to use with my problem :
I have a collection with a particular field “end_time” which is a timestamp. I would like to find the best way to trigger an action while this timestamps passes the now() time.

Can steve jobs act depending on a timestamp in a collection ?

or I have to make a job with an interval that runs through the collection (every minute ? 10 minutes ?) and check if timestamp is passed ? Is is ressource consuming process ?

Thanks for your help !

msavin · January 31, 2018, 7:37am

It runs the jobs based on the time you specify. Here’s one way to do it:

Jobs.run("sendEmail", "Ivo", "me@ivo.com", {
   date: function () {
    return date;
  }
})

You can also stack it with the in and on operators:

Jobs.run("sendEmail", "Ivo", "me@ivo.com", {
   date: function () {
    return date;
  },
  in: {
    minutes: 10
  }
})

The jobs cannot be set to run repeatedly, but you could write a job that would create a new job after it finishes. In whatever the case, the resources used should be negligible.

ivo · January 31, 2018, 7:46am

So each time I create one item in my collection I could set up a job to end “in” the duration time ? And therefor would not have to loop on anything ? sounds great.

You did not mention if code should be on server, client or both side ?

Thanks again !

msavin · January 31, 2018, 7:58am

The package is for server only - and you only deal with it’s API - it does the MongoDB stuff on its own. Check out the documentation and wiki, it should help you get a better understanding of it.

ivo · February 3, 2018, 10:01am

Hey Max,

starting using your package, seems great but I don’t understand something. I have the following code (combining it with collections hooks) :

Jobs.register({
    "logtransa": function () {
    	console.log("BADABOUM");
    	return
    }
});

Transactions.after.insert(function (doc) {
	Jobs.run("logtransa", {
	    in: {
	        minute: 0.1,
	    }, 
	});
});

What works : after an insert, it takes 6 seconds before jobs is started and I see “BADABOUM” in server console.
What doesn’t work (or at least not as i expected) : Jobs is running continously, the function is called hundreds of time until maximum stack exceeded and server shuts down

Didn’t find in the doc why, or how to call it only once. Did i miss the point ? For me the whole point was to call the function once, not to have it run continously.

— Actually jobs starting after server restart and 6 seconds… not even after insert

Thanks for reply,

Regards.

msavin · February 3, 2018, 5:19pm

In the new version, jobs have to marked completed (whereas before, it would happen automatically).

Try this instead:

Jobs.register({
    "logtransa": function () {
    	console.log("BADABOUM");
        this.success(); // <------------------
    	return;
    }
});

You can use this.success() to mark the job complete, this.failure() to have it run on a server restart, or this.reschedule() to postpone it to a later time.

bhanuprasad · February 6, 2018, 5:13am

How to run the job, for every minute?

Jobs.register({
testjob: function () {
log.info("calling test ");
this.success(“sucess”);
return;
}
});
// schedule the job
Jobs.run(“testjob”,{
in:{
minute: 1
}
});

I tried like above, but it’s not working for me.I want to run the jobs for every minute and every day.Could you please help me how to configure the jobs to run for every minute and every day.

ncfrak · February 6, 2018, 3:14pm

Hello @msavin ! Great package !
Going back on this idea, is there any plans on handing off jobs to other processes ?

We have 2 processes launched on a multi-core server and we want 1 process to handle the HTTP and the other to process heavy tasks.

Thanks !

msavin · February 6, 2018, 3:35pm

There is one way to re-run jobs:

Jobs.register({
	"jobName": function () {
		doSomething()

		this.reschedule({
			in: { 
      			minutes: 5 
      		}
    	})
  }
})

That job would run when its time comes, and as part of that, it reschedule itself.

It’s not ideal, however. The history field in each job document keeps track of each time the document runs. It could get pretty big if you are scheduling it to run frequently. Additionally, you would have to “kickstart” the job to run the first time.

msavin · February 6, 2018, 3:36pm

The way it would work here is, you would create two queues with Jobs.register, and add jobs as you wish. The package will only run those jobs on one server at a time.

Do you wish to run one queue on one server and the other queue on another server? Any particular reason?

ncfrak · February 6, 2018, 3:50pm

Thanks for your reply.
If I know which server is the one handling HTTP (front) and which is the one for heavy tasks (worker), I can set an activityDelay on the front server right ? And this would let the worker take the jobs as they come.

Yes my reason for this, are 100% CPU spikes that can last several seconds/minutes due to a lot of tasks to perform. So I wan to off-load the heavy tasks to the hidden process.

I did not quite get this

you would create two queues with Jobs.register, and add jobs as you wish. The package will only run those jobs on one server at a time

How is it done ?

ncfrak · February 6, 2018, 4:39pm

Adding the startupDelay seems to do what I wanted.

FYI I set the server role with an env variable, when Meteor startups the startupDelay is applied on the front server.

relicho · February 19, 2018, 1:05am

Would it be a possible solution to this problem to use the Jobs.run(…) function with Meteor.setInterval(func, delay) so that the job would get executed repeatedly?

msavin · February 19, 2018, 8:49am

Not quite unfortunately. Right now, the best I can suggest is to schedule jobs inside of jobs.

Jobs.register({
	recurringJob: function () {
		magic();

		Jobs.add("recurringJob", {
			in: { 
				minutes: 5
			}
		})
	}
})

ivo · July 9, 2018, 10:17am

Hi Max,

Do you know what could cause Jobs not to start ? I have a website using your package, users create a post that has a job to start after a define time, It works for a while but already stopped twice and the only way to launch those “delayed” jobs was to restart the server and the jobs started again. New jobs worked for a while and then stopped again…

Dunno if the error comes from me, the package, the host, etc… any general ideas ?

msavin · July 9, 2018, 11:38am

Hey - the queue starts automatically on start-up, and stops automatically if it detects that a job had not been resolved. This prevents the job from running over and over again. Even if the job does not run the way you want, you have to tell it to either resolve, or reschedule for later, etc.

The server logs should tell you… if not… maybe you are running an old version?

ivo · July 9, 2018, 12:12pm

Thanks a lot !

Shall I work with a try and catch then ?
Right now I have my function with no special error management and a this.success at the end. If I put it inside a try / catch would it work ?

try {
    myfunction()
    this.success()
} catch(e) {
    this.reschedule()
}