Incorrect result set in template data context

[massimosgrelli]

I’m having a strage behavior that I cannot explain or solve in any way.

In summary: in a parent child template relationship – messages and message – the child template helpers receive a larger dataset than the one the parent selected for them. In particular the child – message – receives both previous and current iteration datasets.

The flow is like this:

1. I have Channels and Messages collection

2. Click on a channel and get its messages

3. Messages template has a child template named message to process the display phase of each message posted
   .

   {{#if Template.subscriptionsReady}} {{#each messages}} {{> message}} {{/each}} {{else}} loading... {{/if}}

The problem:

1. Click on channel A and I got the messages and everything is correct. Messages template processes 3 records and message template receives 3 records to process.
2. Then I click on channel B and I select the correct record set in the Messages template. But when I start to process each message in message template, I observe that it receives much more records than the ones contained in channel B. Using the log to trace what happens, it looks like message template receives channel A and channel B records.

Did anyone experience a similar behavior?

[arythiafan]

In your publish method, are you appending the results to the data that’s published by chance? If so, that would explain the data problem. That’s the first place I would check, as the template is just processing the data that’s available to it. If you wanted to share that code I’d be happy to help you try to resolve the issue.

[massimosgrelli]

No, the publish method for Messages is trivial at the moment:

  Meteor.publish('messages', function() {
      return Messages.find();
  });

I tried to comment out all the possible causes of that anomaly, but still nothing. It’s really weird.
It will probably be my fault, but there a chance that it can be a bug in template level subscription mechanism. I should try to debug meteor source code, by I don’t know where to start.
This is my call stack and I tried to look into the code, but it’s really complex without knowing Meteor internal architecture:

Template.message.helpers.getSeparator (message.js?3c71…ae0913e08b9:42)
(anonymous function) (lookup.js:23)
(anonymous function) (exceptions.js:51)
(anonymous function) (lookup.js:71)
Template._withTemplateInstanceFunc (template.js:437)
(anonymous function) (lookup.js:70)
Spacebars.call (spacebars-runtime.js:130)
Spacebars.mustacheImpl (spacebars-runtime.js:67)
Spacebars.mustache (spacebars-runtime.js:71)
(anonymous function) (template.messag…5b58788c567e:8)
doRender (view.js:337)
(anonymous function) (view.js:191)
Template._withTemplateInstanceFunc (template.js:437)
(anonymous function) (view.js:190)
Blaze._withCurrentView (view.js:523)
lookup:getSeparator:materialize (view.js:189)
Tracker.Computation._compute (tracker.js:294)
Tracker.Computation (tracker.js:210)
Tracker.autorun (tracker.js:533)
Blaze.View.autorun (view.js:201)
(anonymous function) (view.js:331)
Tracker.nonreactive (tracker.js:560)
Blaze._materializeView (view.js:330)
materializeDOMInner (materializer.js:81)
Blaze._materializeDOM (materializer.js:33)
(anonymous function) (view.js:366)
Tracker.nonreactive (tracker.js:560)
Blaze._materializeView (view.js:330)
(anonymous function) (builtins.js:117)
Tracker.nonreactive (tracker.js:560)
eachView.stopHandle.ObserveSequence.observe.addedAt (builtins.js:105)
diffFn.addedBefore (observe_sequence.js:191)
(anonymous function) (diff.js:171)
_.each._.forEach (underscore.js:105)
LocalCollection._diffQueryOrderedChanges (diff.js:163)
diffArray (observe_sequence.js:175)
(anonymous function) (observe_sequence.js:108)
Tracker.nonreactive (tracker.js:560)
(anonymous function) (observe_sequence.js:82)
Tracker.Computation._compute (tracker.js:294)
Tracker.Computation._recompute (tracker.js:313)
Tracker._runFlush (tracker.js:452)
onGlobalMessage (setimmediate.js:102)

P.S. The source code is closed, but thanks anyway

[shock]

I think that resubscribe is one of these DDP operations where u receive new records first and than old ones are removed.
Never deeper observed in which part the Ready() part is thrown.

good luck

[Steve]

@massimosgrelli, can we please see your subscribe code?
If channel A data are received but immediately removed, what your are experiencing is the famous “ready is not what you think it is” effect.

[massimosgrelli]

I have to admit that when it comes to subscriptions and their readiness I still have a little bit of a mess in my brain. I would need a subscription therapy session maybe…

However, your description seems to fit my case: I can log the extra records I receive, but not load them to the screen in the message list. So the only way to avoid the mess in my message.helpers code was to check for each record whether it belongs to the channel I selected or not before processing it. That’s pretty weird because when I create the recordset in the parent template – messages – everything works well.

About my code, the template’s structure is something like this:

layout
 | -- channels
 |------ channel
 |-- messages 
 |------ message

plus some modal dialogs – bootstrap based – used around the code and loaded at the bottom of layout template – all in one place.

I have structure my subscriptions like this:

Template.layout.onCreated(function(){
	var that = this;
	this.autorun(function(){
		that.subscribe('invitations');
		that.subscribe('channels');
		that.subscribe('companies');
		that.subscribe('workjournals');
		if (libProfilesHandle.ready() && that.subscriptionsReady()) {
                  //init data structures
                 ...

Profiles is the only collection subscribed in router.js:

libProfilesHandle = null;

Router.configure({
	layoutTemplate: 'layout',
	loadingTemplate: 'loading',
	waitOn: function () {
		libProfilesHandle = Meteor.subscribe('profiles');
		return libProfilesHandle;
	}
});

And messages template:

Template.messages.onCreated(function(){
	this.subscribe('messages');
});

I resubscribe channels in a couple of dialog templates that probably could be avoided.

There a lot of other stuff, but I think this is the point.

[Steve]

I don’t see anything wrong in your subscribe code. Notice that you can improve it by removing the subscribe calls from the autorun :

Template.layout.onCreated(function(){
  	var that = this;
  		that.subscribe('invitations');
  		that.subscribe('channels');
  		that.subscribe('companies');
  		that.subscribe('workjournals');
  	that.autorun(function(){
    		if (libProfilesHandle.ready() && that.subscriptionsReady()) {
      //init data structures
      ...
    }
  });
});

I see two areas you can investigate:

1. You use subscriptionsReady in two places in your code. Are you sure they are called in the right order? Anyway, this is a source of bug and, IMO, should be avoided (i.e. //init data structures should be placed in a subtemplate).
2. We don’t have the code of your messages template helper. Maybe there is something wrong in there.

[jamgold]

You might have stumbled onto one of the weaknesses of the current pub/sub implementation of Meteor. When you change your subscriptions, the mini-mongo on the client still has the content of the previous subscription. When you return a cursor in the helper for your template, you can actually see reactivity at work when the results of the new subscription are added and the old subscription is being removed. Since you don’t see that I assume your helper returns and array instead?

Anyways, check out this forum discussion

[shock]

Meteor.publish(‘messages’, function() {
return Messages.find();
});

well, this cant be true - cause you will never receive new set of documents, as there is no variable.

So more expected scenario is that you have variable publication with variable selector etc…
What bring us to simple solution which you have to implement every time you are working with sorted sets - and that is sort on client too.
Or in your case, use selector on client, not just find() but find({channel: Session.get("channel")}) or where you store it.

[massimosgrelli]

@Steve I followed your suggestions, moving all the subscribe(..) outside autorun. Thanks, it makes a lot of sense avoiding to subscribe publications again every time the ready flag changes. However It didn’t change the misbehavior I have in the message template (trash records from previous channel).

@Steve

(i.e. //init data structures should be placed in a subtemplate).

I don’t understand this. I need to init data structures before any other template gets rendered, that’s why I put it in the top template within onCreated.

My code in the messages helper does not access Messages collection directly, but it’s mediated by a class which adds an observer on message add event. In this way, I can have counters on user’s unread messages (on a per channel bases). Messages template helper looks like this:

Template.messages.helpers({
	messages: function() {
		var query;
		libRemoveAlert();
		var ch = Session.get('current-channel');

		Session.set('first-message-creation-date', Messages.findOne({channelId : ch._id}, {fields : {createdAt : 1}, sort : {createdAt : 1}}).createdAt);
		Messages.gPrevMessageDate = Session.get('first-message-creation-date');
		query = libChannelMsgsHash[ch._id].query;

		return query;
	}
});

where libChannelMsgsHash is a data structure that keeps memory of read/unread messages and keeps a poster to the observer on Messages collection, for each channel the current user can access.

libChannelMsgsHash is initialized when I create channels template:

libChannelMsgsHash[thisChannel._id] = new ChannelMsgsHandler(thisChannel._id, thisChannel.name);

ChannelMsgsHandler class, among the other things, has the following members/methods:

ChannelMsgsHandler = function (channelIdParam, channelNameParam)  {
    this.unreadMsgs = 0;
    this.channelId = channelIdParam;
    this.channelName = channelNameParam;
    this.query = Messages.find({channelId: channelIdParam}, {sort : {createdAt : 1}});
    ...
    this.observer = this.query.observeChanges({
        added: function(id, msg) {
        ...

BTW, I already tried to comment everything out in the messages helper:

Template.messages.helpers({
messages: function() {
…

replacing its code simply with:

return Messages.find({channelId : ch._id}, {sort : {createdAt : 1}});

in order to understand if the extra records in the recordset were related to the observer I created, but it wasn’t the case.

PS: @jamgold you can find here the code of my messages helper. It returns a cursor, not an array.

[Steve]

I don’t understand this. I need to init data structures before any other template gets rendered, that’s why I put it in the top template within onCreated.

Please forget about what I said about double calling subscriptionsReady. You call subscriptionsReady on two different templates, so they are not concurrent.

Maybe you could create a github repo with the simplified version of your code (i.e. without the libChannelMsgsHash structure).

[massimosgrelli]

You might have stumbled onto one of the weaknesses of the current pub/sub implementation of Meteor. When you change your subscriptions, the mini-mongo on the client still has the content of the previous subscription. When you return a cursor in the helper for your template, you can actually see reactivity at work when the results of the new subscription are added and the old subscription is being removed.

I can see what you say using my log actually. I can trace previous channel records on the console, but not on the screen. And yes, I’m returning a cursor.