Server sided Database Reactivity - Run Tasks when new data is available

#1

First things first: Yes, I know, I am building a message queue and yes, I know there are better “Databases” for that than mongo. But the Prototype needs to go live soon-ish and I don’t want to introduce yet another server.

Is there an easy way to define a Collection that will hold Jobs and as soon (or with some small delay) a new one comes in, it will trigger a server sided procedure in meteor?

e.g. observe but not for publications.
The server that will run these tasks is a stripped down version of meteor where everything is removed that belongs to the client.

#2

You might want to examine https://atmospherejs.com/vsivsi/job-collection to get some inspiration, or perhaps even use this one as your main infrastructure.

Or if you want to begin barebones, take a look at http://docs.meteor.com/api/collections.html#Mongo-Cursor-observe to react to new data on a collection (query).

2 Likes
#3

+1 https://github.com/vsivsi/meteor-job-collection

or look at message queue as a service?


https://docs.aws.amazon.com/batch/latest/userguide/job_queues.html

1 Like
#4

maybe us collection hooks?

but still may want to use jobs for the task if it might fail.

@serkandurusoy what’s advantage of Mongo-Cursor-observe over collection hooks? same thing?

#5

Enjoy :slight_smile:

1 Like
#6

Well, practically, they can both be used for very similar things where the fundamental difference being collection hooks defined on collection’s instance methods whereas observe(changes) act reactively on a query cursor.

A rather important difference would be that;

A collection’s instance methods can only be tracked within the instance of an application (where the collection is defined and instantiated) whereas observe taps onto the database (oplog or poll) and can react to external changes.

1 Like