Meteor method queue processing

help
21

I don’t know if I fully understand your target outcome but think I had to do something similar - the user experience I wanted was that the user would kick off some work, the user could see the progress of this work that was done on the server in the background while they did other stuff, and when the work was complete they would be notified that the work was done with something they could tap on to see the final result.

I simply did this with a background worker kicked off from a method call. The method call returned immediately letting the client code know the thing started.

The client subscribed to a publication returning a view of a collection keyed by their user id that the background worker populated.

I knew ahead of time how many things needed to be done and based on the progress of the background worker on the server I could display x of y that auto updated as the background job progressed through its work and updated the publication.

Once the number of expected things was reached I changed the display on the client to indicate complete status which we made tappable and when tapped would navigate the user to a view showing them the final result. I guess in your case a list of PDF links.

1 Like
22

@robfallows I tried using:

Promise.await(_promiseWrappedSetTimeout());

instead of

await _promiseWrappedSetTimeout();

but it has the same behavior.

My use-case is actually really simple, I have an ordered list of documents (e.g. Documents 1-10) and the user can view any one of those documents as the current active document (e.g. Document 3). This ordering is powered by a field order on the document and also a field activeDocument on a parent collection document that holds the current active document’s order. My use-case is a user can only insert a new document after the current active document.

So when a user clicks a button to insert a new document, the Meteor Method that gets called queries the current activeDocument and inserts the new document setting its order to the number one greater than the activeDocument and then increments the order of all the documents after the new document and then lastly increments the activeDocument to make the newly inserted document the active document - this is the important part.

I need to wait for that last update to activeDocument before firing my Meteor Method again to insert a new document or else the activeDocument won’t be updated and multiple newly inserted documents all get the same order - which causes… problems. :slight_smile:

So the code looks like this:

Meteor.methods({
   insertNewDocument: function(parentId, doc) {
      
      //  Query the parent collection to get the current activeDocument (versus trusting the client)
      let parentCollection = SomeParentCollection.findOne({parentId}, {fields: {activeDocument: 1}});

      //  Set the new documents order from the parentCollection
      doc.order = parentCollection.activeDocument + 1;

      //  Insert the new Document
      Documents.insert(doc, function(result, error) {
         //  Update the other documents after this one
         _someServerMethodThatUpdatesOtherDocumentsOrder(parentId, doc.order);
         //  Update the activeDocument
         SomeParentCollection.update({parentId}, {$inc: {activeDocument: 1}}, {...});
         
      });
   }
});

So I need to block the next Meteor.call("insertNewDocument", order) until the first one has completely finished all the asynchronous database updates. The problem in the use-case isn’t that I’m waiting for some return value, it’s that if it calls insertNewDocument repeatedly before waiting for it to return, it will always get the same parentCollection.activeDocument which messes things up. And the button, method, and functionality should be able to be called quickly, e.g “click”, “click”, “click” to quickly add three new documents.

For my UI, I am disabling the “Insert New Document” button and all of that - which helps, but I want my code to work so that if on the console someone types in:

Meteor.call("insertNewDocument", id, document1);
Meteor.call("insertNewDocument", id, document2);
Meteor.call("insertNewDocument", id, document3);

it works correctly. Even if no one will ever do this, the code needs to work correctly in case someone does or someone tries to screw with it in the console. Its seems like this should easily be possible with Promises versus using a worker or jobs-queue for something that will complete in 200ms.

If I make the Meteor Method async it doesn’t work because of the way I’m calling the above commands in the console. I’m not waiting on a result to pass into the next call, I’m just trying to block rapid-fire calls to the method.

23

Then you have to program it. For example, you create a collection named jobs or something. Then when a user calls the method insertNewDocument, you insert/update the jobs collection:

jobs.upsert({ userId: 'someId'}, { inserting: true, updatedAt: new Date() });

Then in the insertNewDocument method:

Meteor.methods({
   insertNewDocument: (parentId, doc) => {
    const user = Meteor.user();
    // check if user is inserting other document
    const job = jobs.findOne({ userId: user._id, inserting: true });
    if (job) {
      // you may want to check the method timeout
      // then refuse to insert the document
      throw new Meteor.Error('some code', 'some message');
    }
    jobs.upsert({ userId: user._id }, { inserting: true, updatedAt: new Date() });
    // do the job here

    // after that, update the jobs collection
    jobs.remove({ userId: user._id, inserting: true });
  },
});
1 Like
24

That’s a good reply, but that seems like overkill… I’m thinking there’s a Promise-based solution here. Isn’t this what Promises are supposed to do?

25

Promise has no power here.

26

What you want is a FIFO queue (which is basically what @minhna) has suggested.

However, “normal” Meteor methods (i.e. not async, not calling this.unblock()) behave exactly like a FIFO as far as any single client is concerned.

So, my confusion at the moment is why you need to use Promises at all. Any standard Meteor MongoDB call on the server, when used without a callback, waits until it’s complete before moving on. It is, to use your terminology, self-blocking.

2 Likes
27

Ah that was it. I completely forgot about calling Mongo operations synchronously on the server. That did the trick.

Is the move to then wrap these operations in try/catch for error handling? Thanks @robfallows! I knew you would know the answer! :clap:

1 Like
28

Yes. That’s the right approach with these sync-style methods.

You’re welcome. :slight_smile:

1 Like