Strange behavior with a long delay before container recognizes reactive data change

I’m not quite sure how to even describe this, but I’ve built a lazy load mechanism that makes a request when the user is approaching the bottom of the page. It goes like this:

  1. React component detects scroll reaching end of page.
  2. Call made to this.props.requestMore().
  3. This goes into my Paginator class, which is keeping track of our current subscription limit, if we already have a request being processed (to avoid hundreds of calls made from scroll events). See requestMore code below.
  4. The new limit is set (ReactiveVar).
  5. In the TrackManagerContainer, I have a subscribe call that uses limit.get().
  6. Publish function returns a new set of data.
  7. All is right in the world.

And this works fine on the first page load. But if I navigate away and then come back to that page, randomly something weird happens in between steps 4 and 5 above. limit is set to a new value, but it takes about 2-3 seconds before I see my console.info statement tell me that it’s about to call Meteor.subscribe. I can’t figure out why on earth there would be such a massive delay between those two things.

requestMore

    requestMore = (pagerName) => {
      const pager = this.data[pagerName];

      if (pager.requested || pager.limit.get() === -1) return;

      console.info('Paginator: increasing limit to',
        pager.limit.get() + pager.increment);
      pager.requested = true;
      pager.limit.set(pager.limit.get() + pager.increment);
    };

And in my Meteor container:

const TrackManagerContainer = connect()(createContainer((props) => {
  const { limit, reportCount, requestMore } = props.trackPager;
  console.info('TrackManagerContainer: subscribe to tracks, limit', limit.get());
  const handle = Meteor.subscribe('tracks', limit.get());
  const tracks = Track.find({ snippet: false },
    { sort: { createdAt: -1 } }).fetch();
  [...]

I know it’s probably nearly impossible to tell without an accessible repro and code base. But if this behavior sounds familiar to anyone, please let me know!

I managed to capture a screencap. The first lazy load works great. Watch the second one, and notice how in the client, it says “Paginator: increasing limit to 30” and it gets stuck before the “TrackManagerContainer: subscribe to tracks, limit 30” message (and the “publish tracks” message on the server side).

I checked the Chrome Timeline. Below is the highlighted window of time where the subscribe and publish should be occurring, but nothing is happening:

This seems to indicate that something wrong is going on with the server.

@robfallows - you have a good track record of saving my ass when I’m stuck. :wink: Any ideas?

I’ll give it a shot :stuck_out_tongue:

It’s safe to assume then this could stop the current subscription and then re-subscribe. So what you might be seeing here is the time taken to (re)generate the cursor on the server. That time is unlikely to be your Meteor code, but (especially if your collection has many documents) could well be in Mongo. Long delays on queries with sorts (and limits) says indexing to me. I would suggest a compound index on snippet and createdAt to optimise that find. You could do this through the Mongo shell:

db.collectionName.createIndex({ snippet:1, createdAt:-1 })

or in your server-side Meteor.startup.

Track._ensureIndex({ snippet:1, createdAt:-1 });
1 Like

Is pager.limit a ReactiveVar? How are you calling requestMore() method?

Unfortunately the index didn’t help. :frowning:

Reactive data change: 3338.147ms

It is indeed.

As far as requestMore, that’s called when the bottom of the content is reached:

  handleScroll = () => {
    if (!this.rootNode) return;

    const frameBottom = this.rootNode.getBoundingClientRect().bottom;
    const approachingEnd = frameBottom - window.innerHeight < 260;

    if (approachingEnd) this.props.requestMore();
  };

And this is requestMore being passed from the Paginator to its wrapped component:

    render() {
      let ourProps = {};

      for (const pager in this.data) {
        ourProps = {
          ...ourProps,
          [pager]: {
            limit: this.data[pager].limit,
            reportCount: this.reportCount.bind(null, pager),
            requestMore: this.requestMore.bind(null, pager),
          },
        };
      }

      return <Wrapped { ...this.props } { ...ourProps } />
    }

@robfallows Hang on… should the index be for the client-side DB search or the publication? They’re different.

Meteor.publish('tracks', function ({ limit, search }) {
  const selector = { artistId: this.userId };
  const options = {
    sort: { createdAt: -1 },
  };
  const partialMatch = new RegExp(search, 'i');

  const tagIds = Tags.find({
    artistId: this.userId,
    label: partialMatch,
  }).fetch().map((tag) => tag._id);

  if (search !== '') {
    selector.$or = [
      { tagIds: { $in: tagIds } },
      { name: partialMatch },
    ];
  }
  if (limit !== -1) options.limit = limit;
  return Tracks.find(selector, options);
});

Ah well, it was worth a shot. Indexing on artistId and createdAt didn’t help either. The time isn’t being spent in the publication anyway, it seems. The publication function is entered and exited in a fraction of a second. I feel like something is weird with the Tracker, because handle.ready() remains true (otherwise there would be a spinner that shows up).

Wow, so I removed publications totally from the picture. In other words, I’m subscribing to all data globally. I modified my paginator so all it does is just increase the Reactive Var and then it’s passed to the client-side find call:

  const { limit, requestMore } = props.trackPager;
  const tracks = Track.find({ snippet: false }, {
    limit: limit.get() !== -1 ? limit.get() : undefined,
    sort: { createdAt: -1 },
  }).fetch();

Amazingly, there’s still the delay! Could this possibly be a Tracker bug?

@sashko, @benjamn ?

This is still an issue after an upgrade to 1.4.2. See how much idle time passes between setting the limit ReactiveVar and the time it takes for the Meteor container to redraw. I’ll try to put together a repro.

The repro is perfect, so it’s specific to my project.

Maybe you want to use https://github.com/peerlibrary/meteor-subscription-scope to pass limits on, instead of resubscribing.

That looks interesting, I’ll check it out! But I’m already subscribed to all the data. This appears to be an issue with Tracker.

I’ve looked this over and I think I’m misunderstanding something. Wouldn’t this count as a resubscribe?

var subscription = Meteor.subscribe('search-documents', 'foobar');

You’ve hardcoded a search here for “foobar” - but if a user changes their search to “baz123”, you would have to resubscribe, no?