Excess rerendering in recursive templates with "children" list


#1

I’m having trouble with templates rerendering (I believe) too often, which is causing slowdown on my page.

Here’s the setting: I have a recursive template for rendering a message, which recursively renders children of the message. Messages are documents in a MongoDB. Each message document has a children field that lists the _id fields for all children messages. Naturally, I use a helper children that looks up the children in the Messages database:

children() { return Messages.find({_id: {$in: this.children}}) }

This seems to cause excess rendering trouble, though, which looks fine but can cause severe slowdowns for large message trees. Whenever a message document changes (not just the children field, but any field), all the children templates get rerendered (or, at least, all the helpers get called again). I assume this is because the find is executing a fresh search each time. Even adding a .fetch() at the end of the children helper doesn’t help. This surprises me, as the Reactivity Model for Each suggests that it should realize that the data is the same so not rerender… Is this intended behavior?

Here is a more complete example, where an “edit” button enables artificial edits, and a template variable renders keeps track of how many times the template gets rendered.

<h1>Messages</h1>
{{#with root}}
  {{> message}}
{{/with}}

<template name="message">
  <p>Message {{_id}} with <b>{{edits}} edits</b> and <b>{{renders}} renders</b> <button class="edit">Edit</button></p>
  <ul>
    {{#each children}}
      <li> {{> message}} </li>
    {{/each}}
  </ul>
</template>
Template.message.onCreated ->
  @renders = 0
Template.registerHelper 'root', ->
  Messages.findOne
    root: true
Template.message.helpers
  renders: ->
    Template.instance().renders += 1
  children: ->
    Messages.find _id: $in: @children ? []
Template.message.events
  'click button.edit': (event, instance) ->
    Messages.update instance.data._id,
      $inc: edits: 1
    event.preventDefault()
    event.stopPropagation()

In writing this post, I did discover one way to avoid the rerendering, which is to lookup inside the each instead of during the each helper:

<template name="message">
  <p>Message {{_id}} with <b>{{edits}} edits</b> and <b>{{renders}} renders</b> <button class="edit">Edit</button></p>
  <ul>
    {{#each children}}
      {{#with lookup}}
        <li> {{> message}} </li>
      {{/with}}
    {{/each}}
  </ul>
</template>
Template.message.helpers
  renders: ->
    Template.instance().renders += 1
  lookup: ->
    Messages.findOne @valueOf()

#2

I think your example by default creates what I believe to be one of the worst case performance scenarios. :face_with_raised_eyebrow: Lucky you!

Essentially, you are creating a tree of templates. Any node of that tree that updates will cause all children templates of the updated node to re-render completely. Re-rendering templates is pretty heavy weight. Your objective should be to modify data without having to re-render any templates.

Suggested Option

  1. Move your message template to only output the message, not traverse your list of messages.
  2. Then modify your helper to traverse your tree and produce a linear list of every message. This will do the heavy lifting re: ordering rather than nested templates. And this is pretty light weight in comparison to template re-renders.

That should improve things pretty significantly.

Other

Another thing that may be plaguing you is the sheer amount of data updating. How many messages do you have before you see a measurable slowdown? If it is in the 100’s then the data should be no problem.

If it is more data then you may want to figure out how to publish subsets of the data to the client. In that case you would need to put some work into handling the publication of the data in chunks. I can provide some ideas if you think it is warranted.


#3

Thanks for your response! The number of messages is in the 100s, and each message is generally fairly complex to render, so I think it’s indeed the template rendering, not the data.

I like your idea of a hierarchy traversal. I worry a bit that everything will get rerendered whenever the traversal changes (e.g. a new message is added, or a message is moved). But at least nothing will change when an individual message changes. Perhaps an even more efficient way along these lines would be to compute the traversal in an autorun, attempt to only change documents that change, store in a local collection, and then use an #each iterator over a find query on that collection – ideally so that only changed messages get rendered. I’ll have to experiment…

Meanwhile, the {{#with lookup}} solution (at the end of my message) seems to isolate things enough, at least in my toy code example. I’ve yet to get it to properly isolate in my full system, but I’m hoping it’s another dependency issue…


#4

One more followup here: while the lookup technique above works, the following does not work (dependencies leak again):

Template.message.helpers
  renders: ->
    Template.instance().renders += 1
  children: ->
    for child in @children
      _id: child
  lookup: ->
    Messages.findOne @_id

It seems that each is well-behaved (doesn’t trigger unnecessary template rerenders) when the argument is an array of strings, but does not work well (triggers unnecessary template rerenders even when the value doesn’t change) when the argument is an array of objects with string _id fields. I’m a little surprised by this, given the wording of Reactivity Model for Each which explicitly mentions _id support. Worth submitting as a bug report?


#5

Hi @edemaine,

It seems that each is well-behaved (doesn’t trigger unnecessary template rerenders) when the argument is an array of strings, but does not work well (triggers unnecessary template rerenders even when the value doesn’t change) when the argument is an array of objects with string _id fields. I’m a little surprised by this, given the wording of Reactivity Model for Each which explicitly mentions _id support. Worth submitting as a bug report?

Hazarding a guess:

  • If you are still using the template structure to traverse the hierarchy. Then it is your enclosing template that is re-rendering. The #each within the enclosing template would be required to re-render because its parent is re-rendering.

Given your problem, I would not approach it with a recursive solution for performance. Although, it is interesting that the templating logic supports recursive calling I think that is your real problem.

Plus you are making a mini mongo calls to find each element (e.g. findOne).

lookup: -> Messages.findOne @_id


I like your idea of a hierarchy traversal. I worry a bit that everything will get rerendered whenever the traversal changes (e.g. a new message is added, or a message is moved).

Indeed, the entire list may re-render on changes for the option I described above. If the #each can reason as described in the Reactivity Model then it will be able to determine which ones have changed and only render those (or close to it).

If you can do the following, it would provide your best solution:

  • Build a sort parameter into your messages that gives you traversal order. On insertion/deletion modify the appropriate node sort parameters.
  • Then your helper could just return a cursor sorted by that parameter. Meteor is very good at re-rendering lists directly from cursors and being smart about the changes.

This may be hard to implement given what is happening in your tree?