Multiple subscriptions to the same collection blocks change notifications


#1

Can anyone explain the purpose of elt === precedenceList[0] in the function at the bottom? I pulled the code from livedata_server.js

I’ve found interesting problems with having multiple subscriptions returning different subsets of data for a while - but until now its always been work-aroundable.

However, I’ve recently come across a problem that I can’t get around. It involves a dynamic set of fields being displayed in a table, if the user changes those fields, I resubscribe to the same publication with new options, then after the subscription has completed, I kill the old subscription - the purpose of doing it in this order is to avoid UI flicker when one subscription stopping wipes the data, and the second subscription running re-fills it. Something like this (very simplified):

let sub;
let subToStop;
Tracker.autorun(() => {
  const fields  = someReactiveContext(); //
  subToStop = sub;
  sub = Meteor.subscribe("myCollection", fields);
});

Tracker.autorun(() => {
  if (sub.ready() && subToStop) {
    subToStop.stop();
  }
});

This ensures we only have one subscription running at a time (ignoring the obvious race condition), which always has the latest requested fields, but avoids draining and refilling the collection by stopping/starting (as a regular autorun subscription would). This works great if the fields being changed are top level fields on the document, however if you’re adding fields within the same object, e.g., profile.firstName, profile.lastName becomes profile.firstName, profile.lastName, profile.phone - it does NOT. The cause appears to be elt === precedenceList[0] - this check only sends the change if the subscription handle is the first subscription and I can’t think why this would be desirable.

  changeField: function (subscriptionHandle, key, value,
                         changeCollector, isAdd) {
    var self = this;
    // Publish API ignores _id if present in fields
    if (key === "_id")
      return;

    // Don't share state with the data passed in by the user.
    value = EJSON.clone(value);

    if (!_.has(self.dataByKey, key)) {
      self.dataByKey[key] = [{subscriptionHandle: subscriptionHandle,
                              value: value}];
      changeCollector[key] = value;
      return;
    }
    var precedenceList = self.dataByKey[key];
    var elt;
    if (!isAdd) {
      elt = _.find(precedenceList, function (precedence) {
        return precedence.subscriptionHandle === subscriptionHandle;
      });
    }

    if (elt) {
      if (elt === precedenceList[0] && !EJSON.equals(value, elt.value)) {
        // this subscription is changing the value of this field.
        changeCollector[key] = value;
      }
      elt.value = value;
    } else {
      // this subscription is newly caring about this field
      precedenceList.push({subscriptionHandle: subscriptionHandle, value: value});
    }

  }

#2

That sounds like a recipe for scaling issues. Pub/sub starting and stopping is CPU expensive and imagine having many users rapidly changing those fields in the table…

Why do you need a subscription per field why not for the whole table?

Also it’s clear what what the problem statement is, maybe you can elaborate more non what exactly is the desired behaviour…


#3

I’ve not explained this very well. There is one subscription per table - however, the user can dynamically change the fields in the table:

When this happens, we issue the new subscription before killing the old one - to prevent UI flicker. We have to re-issue the subscription as the fields have changed.

The issue is, it seems Meteor only tracks field changes at the top level, so if I subscribe to profile then run a second subscription for emails - it works. But if I subscribe to profile.firstName and run a second subscription for profile.lastName (this is a contrived example) - it wont work.

In my case - most of the fields the user can enable/disable exist within an object: flex (e.g., they have defined their own schema for the data, and to ensure their schema doesn’t conflict with the data we already track, we stick it in an object called flex, within the document). Because of this, virtually all the subscription changes caused by adding columns are within the same top level object flex.


#4

Ah, yeah more clear now thanks. That code snippet you shared is for Meteor’s merge box I believe. If I understood the code correctly, I think it’s only diffing the keys of two objects.

So I guess you’re passing the toggle fields on the left as a subscription params to observe a subset of the collection, but when the toggle field is not a top field, the changes are no longer reactive since the the merge box is only diffing top fields of the docs…


#5

Exactly! It turns out that it was partly my code causing problems - I had a custom observer that called this.added this.removed, etc - and I was also returning the cursor, so when I stopped the subscription, it left behind one of the “registrations” with the same ID.

I fixed my code and it got me 99% of the way to working correctly, back to the same point that I’ve been at with other subscriptions. So I can work around it. Thanks for your help!