A simple question, but probably without an answer…
If you have two active subscriptions to the same collection, is it possible to determine which records in your minimongo are the result of which subscription?
Not easily. From the clients view the data is merged into a single cohesive view. The server maintains a list of subscription handles which you can inspect. But it won’t tell you which documents are associated with it or even which collections.
Depending on how your publications are structured it’s usually possible to figure it out manually though, e.g., subs x y and z all subscribe to these docs and fields so one if them is responsible.
I am thinking that a solution would be to append a marker, perhaps the subscription id, to a field on each record using a transform on the server. Any thoughts on this?
Add a field on each record with simple value of true with key corresponding to the subscription. In the subscription query include the field corresponding on the subscription.
The only negative here is that if you need three subscriptions, you need to add three keys on your documents with the true value
2 Likes
In this vein, I guess it would be possible to add the subscription IDs to the document - you could have an array of subscription IDs (or names). You’d need to pull the current value from the mergeBox (and there could be a race condition if you were’t careful).
But this would be a very manual subscription. On the whole, probably better to just track them as fields with true values!
This was my thought as well, to add a field to each document and then ship that out to the client. But I’m not following why you’d need three keys for three subscriptions.
In my view, if you have three publish functions from the same collection but for one of them you want to know where it came from, you could simply modify that pub to add the key with a property corresponding to that pub/sub.
There are two negatives with what you are thinking
-
You are doing DB writes (updates) on all documents you subscribe to. Those are unnecessary writes to the DB
-
You need extra handling if a document appears in more than one subscription
Both items are not an issue if you include the keys per subscription when you insert the document. No extra writes required after insert.
You can “manually” handle publication on the server by using mongodb cursor functions, it could help you in this situation.
For example: meteor - How to correctly use this.added - Stack Overflow
No DB writes needed. What I was proposing is what @minhna is referring to above (I thought you were proposing this too but perhaps not). Here’s psuedo code:
Meteor.publish('subscriptionOne', function () {
SubOneCollection.find({
// query collection based on whatever is needed, then specify fields to output for each document
}, {
fields: { '_id': 1, 'field_two': 1, 'field_three': 1 }
}).map((doc)=>{
doc.subscription = 'subscriptionOne'; // or whatever you want the key/value pair to be for this specific subscription
this.added('subscriptionOne', doc._id, doc);
});
this.ready();
});
@dpatte You can indeed do what you’re asking. I confirmed locally that this works and no DB writes occur. In my case I have three separate publications sent to the client, all of which come from the same collection. By adding the doc.subscription property and assigning it to a value for one specific publication, I could see that only documents from this subscription had the subscription property. All other subs lacked this property on the document.
1 Like
How will you know if one document is included to both subscriptions?
[addendum]
I forgot now what my exact problem before. But when I did this before, I had an issue with adding and removing documents for cases that a document can be subscribed multiple times. Seems I ended up with “orphaned docs” in the client (but cannot completely remember it now)
Although I abandoned that project but my solution was to write the values directly on the docs
I’m not sure I see a scenario where you’d want two subscriptions to the same collection that return the same documents. If that were the case a single subscription would probably be better.
In my case, my three subscriptions all query for different fields, such that the documents from Sub 1 will be different from Subs 2 and 3. I think the OP is after the same thing, to be able to have two subsets of documents from a single collection, and to be able to distinguish one subset from another via a property.
My approach above achieves what you and the OP are seeking, in that it writes values to the docs, but doesn’t do so in the DB. Instead the docs are fetched from Mongo and then each doc gets a property added on its way out to the client.
A lot of scenario:
E.g.
Games
- recent players
- players with highest scores
- players playing a specific game
Chat
- recently active rooms
- most popular rooms
- rooms with most online users
I agree the same collection can be queried multiple times for different scenarios, but the Find query I’d imagine would be different. i.e.
MyCollection.find({
// the specific query appropriate for the subscription
}, {
fields: { '_id': 1, 'field_two': 1, 'field_three': 1 }
}).map((doc)=>{ ...
So in your Chat example, I would think recently active rooms would have a find query different from most popular rooms.
If we follow that argument of having the “find query different” and having no overlaping documents, then why do you think there is a need to identify which docs came from which subscription?
Good question but I think that’s for the OP to answer. In my own use cases I don’t need to distinguish them (I have other properties on the documents for that). The solution I posted was to help @dpatte with his/her use-case.
I use console.log(theSubObject) to see whatever was in it.
I may not have made myself clear.
Assume I have a collection with 5000 docs or so. On the front end I want to page through the collection, seeing 12 at a time. So I would subscribe to the collection with a filter that gives me only 12 documents in minimongo (using skip & limit). When I want a new page of docs, i would start a new subscription for that new page. I would cache the subscriptions and let them time out after a few minutes each so I could quickly go back a few pages if necessary without re-subscribing.
So this way i get a few dozen documents in minimongo at a time, not all 5000 documents.
The problem is, I have no way of knowing which documents in minimongo are from which page subscription. So I want to automatically add the page number or subscription id to the docs i have subscribed to as I get them, then filter minimongo for docs for the page I want to display.
I don’t want the subscriptionId, or page# in the actual mongodb - only to appear in my minimongo collection after they get transfered to the client.
By the way, my solution, even if I do achieve the above, is still not totally complete, because I will also have to find a way to get a reactive count of the total number of docs in the mongoDB collection without having to download all the docs into minimongo as well.
You could still do this as suggested - with manually sending events:
// very rough - you'll want to be careful about modifying documents, security, etc.
Meteor.publish("stuff", function(...) {
const handle = Documents.find(...).observe({
added: (doc) => {
doc.subscriptionId = this.subscriptionId; // forget the exact name of this option
this.added("documents", doc);
},
// handle changed and removed.
});
this.ready();
this.onStop(() => handle.stop());
});
Or you could do it by returning a second document on a different fake collection with the IDs (this is how both my and aldeed’s table packages do it):
// very rough, will cause problems if the documents change to no longer be included in the set.
Meteor.publish("myPagedStuff", function(selector, skip, limit) {
const ids = Documents.find(selector, { skip, limit, fields: { _id: 1 } }).map(doc => doc._id);
// technically you should observe the above query so you can add and remove documents from this
this.added("documentsOnPage", { _id: "something-that-is-unique-on-client", ids });
return Documents.find({ _id: { $in: ids } });
});
on the client only (important that it is NOT in shared code) you declare a collection:
DocumentsOnPage = new Meteor.Collection("documentsOnPage");
DocumentsOnPage.findOne("something-that-is-unique-on-client").ids;
These implementations are quite rough - but if you want some more details you can look here:
https://bitbucket.org/znewsham/meteor-dynamic-tables/src/master/server/main.js
1 Like