DDP and rendering: improving performance

[woniesong]

Hi! I need a quick elaboration on DDP connection and rendering.

Situation:
Let’s say when I first open a website, there are 150 posts to be rendered. In this case, a DDP connection should be established first. Then, all the published data that I subscribed to get cloned to DB and rendered. I noticed the subscription turns to a ready state only when all the 150 posts are cloned to local DB. In other words it seems to work like this:

1. DDP connection established.
2. Start cloning data from server to client.
3. When all the data is cloned, subscription.ready() is true.

Between steps 2 and 3, users see the posts are rendered as they are being cloned to the DB. Instead of showing the user an incomplete list of posts, I hide all the posts at step 2 and show all of them at once at step 3.

Question
Let’s say I navigate away to /posts/1 and then come back to /. It looks like there’s still some loading time; in other words, it takes a few seconds for subscription.ready() to be true. My guess for the reason is that when you navigate to /posts/1 all the posts that we had inside local DB get wiped, and we have to clone them again. Is it true? If not, what’s really happening?

More Context
In my app, I expect the user to go back and forth the /posts/ page and /posts/:id page pretty often. Right now, everytime the user goes back to the /posts page he needs to wait for a couple of seconds for the posts to be loaded. I feel like there must be a smart solution for this because it’s the same data that he already viewed, and should be able to be cached.

[nottheaveragecoder]

I believe this is what you are looking for https://atmospherejs.com/meteorhacks/subs-manager

[woniesong]

I am already using subs manager but I still need to wait a few seconds between /posts/ and /posts/1. Should it be instant? Am I doing something wrong?

[Steve]

I would be glad to help, but I would first need to know when you subscribe and when you stop the subscription…

[nottheaveragecoder]

Okey, a bit odd could still be other things ofc
You are also using Kadira Debug? If not would make it easier to pinpoint why this is happening

[woniesong]

Let me post code.

/posts/ subscription:

RecentPostsSub = new SubsManager();

Template.recent.created = function () {
  this.initialLoaded = false;
  this.loaded = new ReactiveVar(0);
  this.numPostsFetched = new ReactiveVar(NUM_POSTS_IN_BATCH);

  this.autorun(function () {
    var limit = this.numPostsFetched.get();
    this.postsSub = RecentPostsSub.subscribe('recentPostsAndComments');

    if (this.postsSub.ready()) {
      this.loaded.set(limit);
    }
  }.bind(this));
};

Template.recent.onRendered(function() {
  var limit = this.numPostsFetched.get();
  var instance = this;
  var numPosts = Posts.find().count();

  this.autorun(function () {
    if (!this.postsSub.ready()) {
      this.$('.posts-container').hide();
    } else {
      this.$('.posts-container').fadeIn();
    }
  }.bind(this));

  var fetchMorePosts = _.throttle(function() {
    limit += NUM_POSTS_IN_BATCH;
    instance.numPostsFetched.set(limit);
  }, 200, {
    trailing: false
  });

});

/posts/ publication:

Meteor.publishComposite('recentPostsAndComments', function () {
  var user = Meteor.users.findOne({_id: this.userId});
  return {
    find: function() {
      if (!this.userId) {
        return;
      }

      return Posts.find({
        networkId: user.networkId
      }, {
        sort: {createdAt: -1},
        // limit: limit
      });
    },
    children: [
      {
        find: function (post) {
          return Comments.find({postId: post._id});
        }
      }
    ]
  }
});

/posts/:id subscription:

Template.postsShow.created = function () {
  this.autorun(function () {
    this.postSub = this.subscribe('post', Router.current().params._id);
    this.commentSub = this.subscribe('comments');
  }.bind(this));
};

Template.postsShow.onRendered(function() {
  this.autorun(function() {
    if (!this.postSub.ready()) {
      $('.product-detail').hide();
    } else {
      $('.product-detail').show();
    }

    if (!this.commentSub.ready()) {
      $('.comments-list').hide();
    } else {
      $('.comments-list').show();
    }

/posts/:id publication:

Meteor.publishComposite('post', function(_id) {
  var user = Meteor.users.findOne({_id: this.userId});

  return {
    find: function() {
      if (!this.userId) {
        return;
      }

      return Posts.find({
        _id: _id,
        networkId: user.networkId
      });
    },
    children: [
      {
        find: function(post) {
          return Meteor.users.find({_id: post.userId});
        }
      },
      {
        find: function(post) {
          return Comments.find({postId: post._id});
        },
        children: [
          {
            find: function(comment) {
              return Meteor.users.find({_id: comment.userId}, {fields: {
                '_id': true,
                'emails': true,
                'networkId': true,
                'color': true,
                'icon': true,
                'rep': true,
              }});
            }
          }
        ]
      }
    ]
  };
});

Also, when the user navigates, he does Router.go("/posts/") or Router.go("/posts/POST_ID"). I am using Iron Router. I don’t think it’s doing a full reload everytime I navigate, but some loading. What I was wondering was where this little bit of loading comes from, and whether there is a way to not do any loading.

[shock]

well, and where exactly you are using that subscription manager?
everywhere you call this.subscribe and never mention RecentPostsSub

At least from my point of view it seems you never use it.
But I am not submanager user.

[woniesong]

@woniesong

this.postsSub = RecentPostsSub.subscribe(‘recentPostsAndComments’);

It’s used in the above line for /posts/. Inside /posts/:id, I don’t use subsManager because it’s just one post rather than 150 posts.

[shock]

I would check in browser network websockets what is being done on DDP side.
To identify if it is data based delay, or you are causing it by something else, for example that animation etc.

[woniesong]

Thanks! I will try it now.

One thing that confuses me is that the loading that I am referring to is the time subscription.ready() needs to become true. It’s possible that there might be some loading caused by animations and rendering, but I don’t know why subscription.ready() doesn’t immediately return true when you go back to /posts/ from /posts/SOME_POST_ID.

I think the above sentence better captures my confusion in the original post.

[shock]

But something feels strange about your use of “this” in that autorun, I always assign instance = this or self = this and use that instance/self, as this inside tracker function feels for me as bad scoping. But maybe it is correctly used by you and I am paranoid

[woniesong]

@arunoda would love to have your insight on this matter.

[arunoda]

Hmm.

For the initial load you can use FastRender: https://github.com/meteorhacks/fast-render
SubsManager use it pretty legit to me.

We’ve a new feature in Kadira Debug called DDP timeline. Show us a screenshot of it. https://kadira.io/blog/product/kadira-debug-with-traces-and-ddp-timeline

[arunoda]

Actually issue is here because of this:

Template.postsShow.created = function () {
  this.autorun(function () {
    this.postSub = this.subscribe('post', Router.current().params._id);
    this.commentSub = this.subscribe('comments');
  }.bind(this));
};

You are using Meteor’s subscriptions.

That’s why re-caching again and again. It’s always a good idea to use SubsManager for this as well.

[woniesong]

Thank you!

It looks like the subscription wasn’t the problem. After the initial load, the subscription for /posts/ page is always ready. I also checked that all the data were available even after I navigated to /posts/:id. The problem is that when I call Route.go('posts') it takes about a second before that page is rendered. I thought this delay was coming from the data subscription, but I was wrong.

Do you know where this extra second is coming from? I am running a webview inside a mobile environment.

To put it in another way, when all the data are available in the local Minimongo storage, it still takes a second to render a page after Router.go('posts') is called on a mobile phone. Where does that second come from, and is there a way that I can get rid of it? Could this performance lack come from using iron router instead of flow router? @martijnwalraven do you think you can chime in for one last time?

Edit: After more investigation, the delay happens in between onCreated and onRendered. What could be the reason for a long delay between onCreated and onRendered?