Meteor Explained and methods mechanisms

1

As I was reading Meteor Explained, I got stuck at the methods part.
What’s the use of updated ddp message in methods mechanism? If the client had received the server response (which isn’t an error) why does it have to wait for the updated message? As I came to understand, when updated message is received local collection gets overwritten with the data in originals. But again why so? You already had received the server response with the changes!!

Thanks in advance!

5 Likes
2

I haven’t read the book, and I didn’t know that methods send an updated ddp message, but you got me curious.

First off I had a look at the websocket messages from the project I’m working on

⬆️ ["{\"msg\":\"method\",\"method\":\"logger.log\",\"params\":[\"debug\",\"(CLIENT-ID: kPAosM7sJcsuaXxm6) method modules.update called with args {\\\"id\\\":\\\"FSGyLdZ7YWEAo5ve8\\\",\\\"modifier\\\":{\\\"$set\\\":{\\\"dimensions\\\":{\\\"x\\\":41,\\\"y\\\":81,\\\"width\\\":763,\\\"height\\\":385}}}}\"],\"id\":\"40\"}"]
⬆️ ["{\"msg\":\"method\",\"method\":\"modules.update\",\"params\":[{\"id\":\"FSGyLdZ7YWEAo5ve8\",\"modifier\":{\"$set\":{\"dimensions\":{\"x\":41,\"y\":81,\"width\":763,\"height\":385}}}}],\"id\":\"41\"}"]
⬇️ a["{\"msg\":\"updated\",\"methods\":[\"40\"]}"]
⬇️ a["{\"msg\":\"result\",\"id\":\"40\"}"]
⬇️ a["{\"msg\":\"result\",\"id\":\"41\",\"result\":1}"]
⬇️ a["{\"msg\":\"changed\",\"collection\":\"modules\",\"id\":\"FSGyLdZ7YWEAo5ve8\",\"fields\":{\"dimensions\":{\"x\":41,\"y\":81,\"width\":763,\"height\":385},\"updatedAt\":{\"$date\":1581392648179}}}"]
⬇️ a["{\"msg\":\"updated\",\"methods\":[\"41\"]}"]

First we see two messages sent to the server:

  1. A log method call (id 40),
  2. Followed by a method call to modify the modules collection (id 41)

What’s interesting is the replies:

  1. We get an updated message for the log call (id 40)
  2. We get the result of the log call?
  3. We get the result of the update call (id 41, result is 1 - the number of changed documents returned from method code)
  4. We get a changed message with the changed fields of a subscribed document
  5. We get the updated message

Interesting that they seem to come in different orders!

  • I suspect the log sends updated first because I call this.unblock in the log method code
  • Also interesting that the database change comes through before the updated message for 41
  • I think in the case of multiple document changes as a result of a method, all the DDP events will be sent through before the updated message

Looking through the Meteor source, seems to validate these thoughts:

  • Lets start with the method protocol handler
  • There’s an interesting construct called a WriteFence DDPServer._WriteFence that keeps track of pending database operations, and the DDPServer's protocol handler for methods waits until all ‘writes’ are complete before sending updated. The comments are pretty explicit: 
    // set up to mark the method as satisfied once all observers
// (and subscriptions) have reacted to any writes that were
// done.
  • The mongo driver marks writes on the current fence for insert/update/remove/dropCollection/dropDatabase calls.
  • The oplog observer and polling observer both mark writes when a collection has changed and they need to wait to grab the result from the database and send the required DDP message

I couldn’t directly find why unblock would send updated immediately, but I assume it clears the current write, so more operations can continue out-of-order from the same session.

tl;dr: Updated lets the client know that all operations related to a (not-unblocked) method have completed and the next method can be executed

Thanks for the inspiration for a deep-dive

8 Likes
3

This is incredible. Something that made no sense, now explained! What does the client use the updated response for though? Does the callback trigger on result, or on updated?

1 Like
4

If you use Meteor.apply, you can pass an onResultReceived callback in options, which is called when result is received.
The updated response is the trigger to call the normal callback given to Meteor.call or apply

This means that you can expect that any database changes will have arrived on the client by the time the callback runs.
For example, say you’ve inserted a document with a method. If you’re subscribed to the relevant collection, you can be sure that when the callback runs, you can use findOne() to get the document.

This is explained pretty well in the guide: https://guide.meteor.com/methods.html#call-lifecycle

3 Likes
5

Thank you so much for the through answer @coagmano. I believe it’s important to examine our understanding of methods workflow is correct. So the scenario, in my own understanding, goes as follows:

  1. You click on a button that triggers a method (this method is defined on both the client and server which means latency compensation is involved)
  2. Meteor preps for invoking the simulation by starting to watch out for which documents are about to be changed. The changes in the simulation are applied to what’s called ‘local collection’ and as we watched which documents got changed we managed to identify them and store them in ‘original collection’.

Please note that we can later on use the original collection to track back the changes applied in the simulation if the method fails.

  1. The simulation runs gracefully, the UI gets updated splendidly and the user doesn’t know a thing!

  2. Meanwhile, Meteor sends out a DDP call to the server with the method name and parameters.

  3. The server does its thing and returns the result of the method computation. Simultaneously, the DB operations resulted from the method are to be notified back to the client by updated DDP message when done.

Again, It’s important to differentiate between the method ‘result’ and the write operations denoted by updated DDP message.

  1. When Meteor gets the ‘result’ from the method invocation, it applies the returned result to the ‘original collection’ but keeps the ‘local collection’ as is.

  2. We get the updated DDP message and we overwrite ‘local collection’ with ‘original collection’ .

There’re two problems with the previous scenario [If I understood methods correctly, lmao].

  1. updated is sent out to confirm that the DB operations resulted from the method are done, right? So, in the case of a method which inserts a new document for example, why would updated get sent out after the method return?! I mean for the method that inserts a document to return successfully that means that the insertion operation is already done!

  2. What’s the use of waiting for updated to overwrite ‘local collection’ with ‘original collection’? In the scenario listed above, you’d find out that awaiting and applying original collection to local collection is unnecessary because they were already the same!

Sorry for my failure to word the question properly, looking forward to your response, thanks!

6

Do you mean added instead of updated here?
My guess is that insert is special, since it returns the new ID, and it’s possible that the inserted event is sent before the result?

Yeah, just tested and an insert inside a method sends the added event before result.
In my test I added a document in collection A with a relation to a document in collection B. If we just look at the websocket, the order of events was:

  1. Client calls the create method - method sent to server
  2. Server inserts new document in A - added sent to client
  3. Method returns - result with new ID sent to client
  4. Collection B responds to event listener on A, updates a document in B - changed sent to client
  5. All writes complete, updated sent to client.

It’s possible there is extra logic that happens on the server, which would create a mismatch. As a rough example, a media collection that creates a thumbnail as part of the insert process.

If the two do turn out the same, you shouldn’t get a second render on the client because Tracker compares previous values to new ones to test if the dependency has actually changed.

1 Like
7

im really glad you mentioned this book, i got a copy and its great. thank you @harry97

1 Like
8

@coagmano I guess I need some time to digest this and fiddle with a meteor app to test things out :sweat_smile: Thank you so much for help, man.

@brianmulhall That’s really good to know, hope you learn a lot from it :muscle:

9

I thought this book/pdf was well made too

https://www.securemeteor.com/

if anyone wants to check it out

2 Likes
10

@brianmulhall
Yeah, it’s on my to-read list. BTW, since you seem to be into Meteor Books, have you tried these? What’s you take on them?

https://books.google.com.eg/books?id=lOeuCQAAQBAJ
https://books.google.dk/books?id=hp_cDgAAQBAJ
https://books.google.dk/books?id=6_l_CwAAQBAJ

11

I haven’t read any of those actually. I really liked Meteor In Action because i was so new to Meteor when i read it, i found it very helpful. Those looks nice tho

1 Like
12

So Meteor explained is worth the read ? I have been looking at it for a while now and couldn’t make up my mind.

13

Take a look at the chapters and sections if you’re a familiar with them and how they work internally, then maybe this book isn’t for you :wink:

14

I check the themes and these are always interesting subject to get acquainted or even read more about but was more talking if the content was clear and well explained.

15

It’s well explained and there’re references to code blocks where you get to know exactly how things are implemented

1 Like