Sorry to revive, so in your tests you found that reactive: false is essentially like using a Meteor method then? Does it eliminate all the server overhead with observeChanges and Tracker.autorun etc. etc.?
1 Like
A method would only be able to return the result in an array. With reactive: false you would be able to work with the collection on the client.
Using {reactive: false} on the client has nothing to do with the server overhead. It simply means that it will not create a Tracker dependency and the cursor will not be invalidated when new data is available (when used in a reactive context, such as a Blaze template or a Tracker.autorun.
The method suggested by @dburles should work. It sends the required messages to the client, the same way a normal, āreactiveā publication would, but will not keep processing the cursor after the initial fetch.
1 Like
A couple questions - forgive my inexperience. On Kadira, Iām having lots of delays on observeChanges on my subscriptions and Iām wanting to strip out reactivity on a lot of things. Whatās the best approach for eliminating as much observeChanges as possible? I want my actual reactive subscriptions to only push documents and fields that are required to be reactive for my app to function. Everything else can be static and/or updated on refresh.
A Meteor Method probably has the least overall overhead to the server. If instead I use {reactive: false} doesnāt this eliminate the observeChanges overhead on the server/DB which does in fact improve server overhead? Or on the server it still observes changes thus adding overhead, but not on the client?
Lastly, if I used the @dburles method, should I also throw in a this.stop() at the end just to close down the subscription?
1 Like
This has no influence on the server. It is a MiniMongo cursor. The server is not aware of those.
This solution is not a complete one. Adding onStop is likely the first enhancement you can make, although it adds a bit of memory and processing overhead (of course, this is needed in order to clean up after the publication). Other options are publishing to a dedicated client-only collection and tossing it away afterwards (this can also be done when using a method).
@alon, is it ok to use the dburles method as-is even though it is ānot a complete oneā? i.e. is it simply somewhat inefficient, or is it going to break randomly because of race conditions, etc?
That really depends on your use case.
It will send some documents to the client and thatās it. It will not clean up after itself when you unsubscribe, so in some use cases, you will have unwanted documents in your client-side collection. Using a method to non-reactively retrieve data may be more efficient in some cases.
If you can live with that, I see no real issue with using it.
1 Like
Sorry to revive this again, but I went down the long road of converting a lot of pub-sub to Meteor Methods to avoid the overhead of pub-sub only to realize that if you have 1000 users all requesting the same data via a Meteor Method (e.g. a chatroomās Chats for example) it will hit Mongo 1000 times for the exact same data which will (and did) create a bottleneck.
I found itās better to use pub-sub if you have heavy data/observer reuse. That way itās way lighter on Mongo by sending data to the clients thatās already on the server versus re-fetching it from Mongo every Method call. If you donāt need reactivity, the above āstatic publicationā patterns work well.
Thereās also cult-of-coders:grapher than can cache Meteor Method-like queries but this seems very similar to a nonreactive static publication.
Iām now understanding everything much more after a few years of being into Meteor.
4 Likes
Couldnāt you memoize your Meteor method function and invalidate the cache each second?
Then youāre essentially debouncing queries to the database and serving data that may be up to a second out of date to everyone else
1 Like
Will you share a nonreactive static publication example with us?
@coagmano Yes I guess you could memoize. I actually have never heard of that until right now. Iām not the savviest programmer in the world. This article on using lodash memoize in Meteor is interesting. I wonder if cult-of-coders:grapher uses this behind the scenes (checking itās package.js on Github it has a dependency on lodash). Very cool though. Wouldnāt this basically do what a static nonreactive publication would do if the query is the same? For example, 1000 users getting the same chat room history. Publications ādebounceā queries to the database for the same query. Iāll have to do some cloud tests to see which pattern is more performant under user load. The Meteor Method memoize route probably uses less memory than the pub-sub overhead. Maybe not if itās a static nonreactive publication and isnāt observing.
@aadams Read this thread, the code is above. Youāre basically doing a manual publication accessing the this functions available in it to control how documents get added to the cursor. So calling this.added(...) to add each document into the cursor, but then not calling a this.changed(...) or a this.removed(...) which strips out the reactivity because it only adds documents to the cursor. Then call this.ready() and this.stop() to stop new documents from being added. That would be a static nonreactive publication that acts like a cache for the same query. Then it probably doesnāt hurt to add reactive: false to any find(..) calls on the client after subscribing.
The point of doing this ānonreactive staticā publication though, for me at least, is to get a ācacheā functionality off-the-shelf with core Meteor.
1 Like
I was implementing the pattern described in this thread and I wanted to mention an issue with the code above: I said that publications can act like a cache and debounce subscribe calls off Mongo to the server because subscribing with the same query will use the data already on the server and not hit mongo again.
I told @aadams to look at the code above. But Iām thinking the code above is wrong if you want to achieve this because it has a fetch() in it, which will hit Mongo every time (I think - unless Meteor also handles fetch() calls with the same query within publications). To have observer reuse you have to use an observer.
So instead of fetching for the documents like this:
Meteor.publish('example', function() {
const docs = Foo.find().fetch();
docs.forEach(doc => this.added('foo', doc._id, doc));
this.ready();
this.stop();
});
It needs to use an observer:
Meteor.publish('example', function() {
var self = this;
var fooHandle = Foo.find().observe({
added: function (addedFoo) {
self.added('foo', addedFoo._id, addedFoo);
}
});
self.ready();
//self.stop(); // EDIT: Do not do this, will not deliver any data to client
// Instead do this
self.onStop(function() {
fooHandle.stop();
});
});
The one thing Iām wondering now is: I want to call self.stop() to stop any reactivity and overhead (because I just need a cached Meteor Method functionality). But Iām curious if self.stop() kills the observer reuse and subsequent calls from other clients to this publication will re-query Mongo?
Per the Meteor doc it says self.stop() only stops the clientās subscription, but Iām wondering if itās cleaned up on the server too. So having 1000 simultaneous calls to this publication would then hit Mongo 1000 times?
If it did, I could always just not call this.stop() in the publication and either benefit from new documents that get added, or use reactive: false on the client to stop any added document updates if I truly donāt want them and/or immediately call stop() on the client to kill any reactivity.
EDIT: After some experimentation, I answered my own question about calling this.stop(). The answer is you cannot call this.stop() in the publication like I did above. If you do this, the data never reaches the client. Instead, you can listen for the subscriptionHandle.stop() and then call this.stop() (I modified the code above).
However, even if you try to stop the subscription on the client immediately after calling subscribe() by calling stop() on the client subscription handle like this:
subscriptionHandle.stop() or instance.subscriptions[x].stop()
ā¦or by allowing the template to be destroyed, the data is also removed from the client. So you really canāt call stop() how I was thinking (and how itās mentioned above in this thread). Any call to stop() kills the data. So, you either deal with reactivity of new documents being added or call reactive: false on the client Foo.find().
EDIT2: If you were dead-set on calling subscriptionHandle.stop() to somehow end reactivity in a pub-sub, you could probably subscribe to the data on the client, then in the callback of the subscribe do a var foos = Foos.fetch() on the data and save it locally to use, then immediately call subscriptionHandle.stop(). No reactivity and you have the data. This would effectively be very much like a Meteor Method.
The problem is Iām not totally sure what stop() does to a publicationās observer on the server. If it kills that observer on the server right away, then it might be a bad idea to stop() like this as it may never leave you with a cached publication observer.
Imagine 1000+ users all subscribing, fetching, then stopping at the same time. If Meteor is somehow smart enough to always let one observer linger for a while on the server, then they would all hit the cache. But if theyāre all calling stop() as fast as they can and stop() kills the publicationās observer then there would likely never be an existing cached observer in existence (or at least not all the time). So you would have a lot of Mongo hits because no existing observer is in existence. Probably best to not stop() until your app does it naturally (e.g. user navigates away from a template and its subscription is torn down).
Sorry this got so meta. This is worth having an MDG dev chime in.
2 Likes