Is this explanation of publish/subscribe OK?

I am not satisfied with how publish and subscribe are explained in the tutorials I am currently reading.
So I made my own explanation as shown below.
Since I have no idea what is actually going on behind the scene when using publish and subscribe I would like some feedback if this view is OK.
What I like about this explanation is that it decouples DB synchronization from reading data allowing us to talk about these two topics separately.

Publish and subscribe actually define which data from MongoDB will be cached on MiniMongoDB.
When you later use find() on the client you are actually reading from MiniMongoDB and therefore you can only read data that has been cached on MiniMongoDB which is controlled by publish and subscribe methods.
By changing subscribe parameters client can influence which data is being cached but only to the point as restricted by publication implementation on the server.

Publish and subscribe are parts of the same thing that has been split between client and server for security reasons.
Subscribe part on the client gives client certain freedom on what data is going to be cached for reading.
Publish part on the server restricts client on what data client can request to be cached.
This prevents client from being able to cache and therefore read the entire data base and therefore get access to restricted data, like data belonging to other users.

3 Likes

Sounds ok to me.

What is really lacking in the doc is a simpler example of subscribe arguments. The “counts-by-room” example is very difficult to grasp and should be replaced.

1 Like

That’s a great explanation, OP!

I attempted to explain some of this stuff in my post here: https://medium.com/@stubailo/data-flow-from-the-database-to-the-ui-three-layers-of-meteor-d5e208b466c3

@ivoronline, do you think this does a better job explaining?

Also, what do you think about the basic docs section on this topic: http://docs.meteor.com/#/basic/Meteor-publish

I agree that we should improve the docs about this, trying to collect more info about exactly what it should say.

1 Like

Most of your tutorial is about how to select how much data to show by combining publication and subscription.
But I think a bit more in depth explanation should be in order explaining what exactly is going on and at what time.

Definition “Publications are functions on the server that define a stream of documents” implies that publications define which documents will be pushed to the client. At least this is how I understood it before I found out that clients can send parameters to publications while subscribing.

Since this client originated parameters also define which documents server will push to client I would say that more proper definition would be “Publications and subscriptions are functions that together define a stream of documents from server to client”.

Also term “stream of documents” is a bit vague? When I execute find() from the client does this triggers stream creation and at that point documents are sent from server to client? Or is this streaming going on continually as a way of caching MongoDB into MiniMongoDB?

Is publishing/subscribing about caching MongoDB or about reading data?
If it controls caching of MongoDB into MiniMongoDB this should be clearly stated and in this way this process is completely decoupled and unrelated from reading data from MiniMongoDB.
Looking at your diagram it looks like publishing/subscribing really is about caching MongoDB but this fact is not explicitly stated anywhere and it should be if it is true rather than relying that people might get on their own to that conclusion.

A border line examples might give more insight of how all this works.

If we have two subscriptions based on the same collection but using different filter what happens when client executed find(), which documents will it get?

If user in china inserts new document in MongoDB will that document get cached in MiniMongoDB on my client as soon as possible if publish/subscribe methods say it should even if at that moment I am not actively using the app?

If I understood correctly publish/subscribe should have no influence on inserting records to MiniMongoDB. So what happens when I insert a record I am not supposed to read as defined by publish/subscribe? Will this document find its way to MongoDB? Will it dissapear from MiniMongoDB?

This is super important topic for Meteor and I think it should have both super simple and super detailed explanation of what it is and how it works.
It is also interesting as hell.

Concerning your reference to the “the basic docs section on this topic”. It talks about “this.userId” but it doesn’t say that in order to use “this.userId” you have to install accounts package. Also there is no reference to what “this” is and what “this.userId” is.
In another post I asked a question what is this “this”? Where does it come from? How do I put my stuff in it?

1 Like

Here’s an explain like I’m 5 description:

You’re a funny man that gives funny names to things. You call your grocery shopping trips Subscriptions. And at the grocery shop, you named Server, works the shop owner, you named mr. Publisher. He decides what you get to take home with you on your shopping trip. You can take home all the food you want but he won’t let you take home the cash register, only a bit of change out of it, and he won’t even show you the shop owners wife.

You’re a smart man so you decide you only want to take beefsteak home with you. Because let’s be honest, even if you brought vegtables home with you you wouldn’t eat them. The shop owner mr. Publisher is ok with that, but he does decide you only get to take home as much beefsteak as can fit in your car trunk, because mr Publisher is also smart, he knows that he also has other customers that will want steak and letting you take home all the steak would be idiotic, it’s so much steak you would clog up the checkout line for others, wouldn’t even be able to transport it all home nor eat it before it spoils.

So you take your car trunk full of steak to your home, you named Client, and put it all in the fridge, you named Mini-Mongo. That way for a couple days, whenever you’re hungry, you can just look in your fridge and pick a steak to eat without everytime having to go on another Subscription shopping trip.

Oh, and ofcourse now that you’ve shown your appetite for steak, the shop owner will make sure to send you a message to come pick up some Kobe steak when he has some newly available.

1 Like

I’m totally lost in translation.

@camiel
Yeah, this is how you explain core concepts in IT industry. Finally something I can understand.
Maybe few changes might be in order to get the complete picture.

Replacing “funny man” with mother.
Introducing son named “Client”.

Son Client puts subscription stickers on the fridge saying what he wants to eat.
So son Client is happily playing GTA 5 on his Playstation 4 and from time to time he goes to fridge to get some more coke and chees or whatever.
During this time mother makes constant roundtrips to grocery store to keep the fridge full using fridge subscription notes as shopping list.
Sometimes shop runs out of something like chips but as soon as new batch arrives shop owner Publisher calls mom to come pick it up and mom puts it in the fridge so that it is there waiting for her sweet son when he needs it.

1 Like

For probably too long, I conceptualized subscriptions like client side views and wasted a lot of time trying to force the client to accept a publication I have subscribed to as a parameter for CRUD operations. It might help to emphasize that the primary role of a subscription is to maintain the state of miniMongo in sync with the server for the cursor specified.

The use of “null” publications probably deserves a bit more emphasis in the documentation. These are mentioned in passing - an example where they make sense would be helpful.

1 Like