Simply put, subscription caching is a must-have feature for anyone with an app that uses a non-trivial amount of data. If you tend to agree, please make your comment below, even if it is just a “yes”
I whipped up a basic implementation which changes: Blaze.View.prototype.subscribe and Meteor.connection.subscribe to check if a subscription was marked as “cached” by the developer. Then we can check self._subscriptions[id],isCached && … before destroying the handle in the Tracker or the viewDestroyed.
Let the developer handle clearing and editing what subs are cached and what ones are not and how many, etc, because that behavior is quite app-specific. Expose Meteor.connections._subscriptions as something like Meteor.subscriptions to give the developer full access to the cache. It is not hard to write Meteor.subscriptions.clear(/*id or {} for all*/), .pop(), .forget(id), .forgetAndStop(id), .expireAfter(id, time), .setLimit(), etc. functions.
By default, the params.cache == false, so current Meteor behavior is achieved.
Caching subscription might be useful.
There is a known package for this, which I didn’t try yet: https://github.com/meteorhacks/subs-manager (from @arunoda).
The issue is that controlling the cache size in terms of number of subscriptions is very limited. What I would like is a fully automatic system where I can set the cache size in megabytes (or even better: the system could set the size for me, depending on the platform).
Although your suggestion for direct memory size control seems less limited, it isn’t because you’e still dealing with a static number on a per client basis. I feel that UX can be more easily mapped to documents than bytes.
Specifying a static number would only work if, internally, there was a more complex scheme that added dynamic weights to various subscriptions, or else you have the same problem as just specifying a limit.
The idea below is to add real-time weights to different subscriptions based on how often the user has been there, if they’ve been there before, how the page relates to them (maybe in a friend graph, or some other relation graph), how important the document is, how many clicks they did on it, etc.
At least, there should be a N(t)_i for all subscriptions_i for all connections_k, where N is the number of requested subscriptions, and t is time, this function could be used in the setMass_i function for subscription_i. As an educated guess, you could pre-assign seed weights to subscriptions, and have the algorithm evolve the value on a per user basis.
//In Meteor Core
...
else {
//New Sub! Generate an id, save it locally, send message.
id = Random.id(),
self.subscriptions[id] = {
...
isCached: true //hypotehtical
mass: getMass([name].concat(params)) //hypothetical
//mass used to determine the probability to remove this sub
//lower mass = lower probability.
uncache: f //function to uncache the sub
}
self._send({msg: 'sub', id: id, name: name, params: params});
}
...
function getMass(argsToSubscription){
if (name == "detail-view-post-my-profile"){
//this user's profile - gonna be there a lot.
return .01
} else if (name=="list-view-page-following"){
//users followings, gonna be therea lot.
return .1
} else {
//other pages that arent frquented by user
return .5
}
}
function setMass(subId){
//if user recently went to a sub very often,
//decrease the mass of the sub, giving it higher
//chance to "make it" when popCache is called.
Meteor.connection._subscriptions[subId].mass = ...
}
Cache = {subs: Meteor.connection._subscriptions}
CACHE_MAXIMUM_MASS = 10;
function popCache(){
if (Cache.mass > CACHE_MAXIMUM_MASS){
var subToRemove = pickOneRandomlyWeightedBy(Cache.subs, 'mass');
if (currentPageContains(subToRemove)){
var mass = subToRemove.mass
subToRemove.uncache() //but don't stop the sub.
Cache.mass -= mass
} else {
subToRemove.stop();
}
popCache();
}
}