Imperfection in reactivity (client)

Problem: Sometimes (~30%), I see that the changes in the database are not reflected on the client through subscriptions.
Explanation: While Collection.find({...}) gives me the updated doc at the server, at the client it gives me an old doc.

Reproduction: Happens only on production, for some time (2-4 hours), after which the changes are shown.

Using:

I have run a few tests, but without any satisfactory results. The guess is that the docs are getting cached but how and where? An important point is that when a particular doc is not gettting updated, it does not get updated at any client, suggesting that the problem is at the server, or cloudflare, and not the client/service-worker. I do not think cloudflare can cache ddp responses, so server seems to be the cause. But a Meteor method call gives me the correct data. Extremely puzzling behavior, and since I can not reproduce it on development, I am at quite a loss.
The only window to debug is those couple of hours in which that data is not updated.

Additional Info:
There is a list view and a single view. While the doc is outdated in the list view, when I open the single view, it is updated (it has a different subscription). Going back to the list view, again show outdated doc. Even a refresh does not fix it.

I am sure that without reproduction, it will be very difficult for people to help. What I need are ideas to debug and find out the problem?

I was thinking this might be a react-meteor-data issue (only because I am out of ideas), so I am also trying a Blaze version for testing. I will report the findings here.
Update: It is not, as expected. The subscription fails to update immediately in case of Blaze too.

Summary of findings till now

  • Switching routes, refresh, clearing app data - nothing at client seems to work.
  • Meteor method works - it’s the subscription that is the problem
  • The problem is at multiple clients - the outdated docs are outdated at other clients too
  • Even if I resubscribe the publication, there is no update
  • The only way to make it work is a server restart

Are you passing reactive data in your subscription without wrapping it in an autorun?

I was using a simple Meteor.subscribe in createContainer. In the Blaze version, I was using a templateInstance.subscribe. No autorun.

I would like to analyze the DDP connection, as that is the only thing that I can now think of.
I tried using ddp-analyzer, but on local it simply refuses to connect citing cors issues. Is anybody using it with latest Meteor?

Is there another way to hook into DDP connections?

Does the collection update in the reactive container when you console log it?

No.
On the list route, the documents are not updated. But when I move to a single doc route (different subscription), that single document is updated. When I move back, the document and entire collection is still not updated.

It feels like the server has cached the response for that list, and does not send me the new one, even after a hard refresh. Which is why I want to analyze the ddp responses now.

Additionally, I have also seen that Meteor.logout, login, and method calls also fail to work these times, i.e., I do not get any response from the server (I use call backs but I get neither an error nor a response). May be these two issues are related, but I can’t prove them.

I have 2 other complex apps working fine, while this simple app has been baffling me! And on local/testing I can’t reproduce the issues.

I use the Chrome Meteor DevTools.

However, most issues in this behaviour are to do with not handling reactive parameters in subscriptions properly and/or not handling reactive data in publications properly. I would look at these areas first.

Ok Thanks. I will try them.

  1. Not handling reactive parameters: I am using createContainer and the subscription is based on its props. So I don’t think I need any autorun, as the function itself must re-run whenever a prop changes (and it does get rerun as I can verify from console logs).
    Client code: https://gist.github.com/s7dhansh/721cf0bff6f9e34784c9594b16b30639

  2. Reactive data in publications: The publication is very simple, and does not need any reactivity as it is purely based on its arguments. https://gist.github.com/s7dhansh/bf0231e319b1c607d30bf435f072b1f5

That being said, there definitely is a problem in the code somewhere, but I am not able to figure it out. Mostly because only a few docs do not get updated, while others do (without any recognisable pattern), and everything works perfectly in dev environment.

Thank you for the extension. It shows that sometimes, there is no DDP response being sent at all. Probably that is the cause for the issues I am facing.

To check (it is happening currently), the site is https://visits.settlin.in
When I run,

Meteor.call('visits', {date: new Date('Feb 18, 2017')}, function(err, res) {console.log(err, res);})

on console, I do not get any error/result. The DDP tab too shows me nothing.

The same code on https://visits.testing.settlin.in, yields response.

Any ideas?

If I restart the prod app, it will work. I am leaving it like that for a few hours, so that some of you may check it out and hopefully help.
EDIT: Actually, you might not be able to see the behavior, because it fails only once I have tried login. After the first try, the calls do not reach the server. Not to mention that the login does not succeed.