Async processing with Meteor.js

Hello. Let’s say I have a Meteor.js app displaying data collected by some IOT device pushing data to a MQTT broker.
I need to pick that data up, processing it and store it to MongoDB before Meteor can actually display it.
Is there a way to have some kind of worker in Meteor? I googled a bit about async workers but couldn’t find much. If you come from PHP, I mean something resembling Symfony or Laravel commands (so I can schedule them with systemd or supervisor or whatever).

The goal is to have this in the same app/codebase instead of somewhere else.

I can recommend @msavin s Sjobs.
https://atmospherejs.com/msavin/sjobs

1 Like

This might be fitting to the requirements:

meteor npm install mqtt. Then, use the example from https://www.npmjs.com/package/mqtt#example, except in your 'message' handler, wrap the callback in a Meteor.bindEnvironment call, like this:

client.on('message', Meteor.bindEnvironment(function (topic, message) {
  // message is Buffer
  console.log(message.toString())
  MyMongoCollection.update( ... )
}));

Don’t worry about async workers or stuff like that. It won’t improve performance, it won’t prevent things from “blocking,” the meteor execution model isn’t comparable to PHP. Don’t use node-webworker-threads, that’s really going to blow up in your face. Use exactly the example above, and check it out!

Sounds fair, but can Meteor keep running even when I am not on the page?
I think this should go in the server startup function?

Meteor (node) is always running, node’s execution model is quite different than PHP’s.

Yes, you can put these commands into a Meteor.startup call.

Alright, that’s great!
Thanks!

I’m going to throw a new library in there:
Microsoft Napa.js for Node.

What I like about this one is I didn’t have to modify my functions, it just worked by adding the npm module with a few lines of code.

const napa = require("napajs");
const NUMBER_OF_WORKERS = 4;
const zone = napa.zone.create('zone', { workers: NUMBER_OF_WORKERS });

then for a function:

const occurences = (str) => {
...code here
};

zone.broadcast(occurences.toString());

const occurencesResult = await zone.execute(occurences, [str]);

For the client, I recommend workly. Super easy to integrate.