Meteor Pub/Sub Data Challenge - need architecture/schema insights


#1

Let’s say I have a Todos app with two new “advanced” features:

  1. Labels, which allow a user to organize todos by giving them one or more labels.
  2. Child todos, which allow me to move normal todos under other todos (parent/child relationship)

In our advanced Todos app, have a view that has 2 columns - each one represents one “Label” (We’ll say “Bug” and “Story”).

Each column displays all Todos that match the column’s label. To do this, we have a subscription for each column that passes in the column label and a limit (We’ll say 10) to enable infinite scrolling.

The server returns a cursor along the lines of:

// Return all Todos matching passed label that are "Top-level"
// Todos (e.g. they are a parent, not a child).
// The "todo.parent" field is the string _id of another todo.
return Todos.find({ labels: label, parent: { $exists: false } }, { limit });

On scroll bottom, we increase the limit and update the subscription.

This works well and is simple. Wow, very cool!


We now have a requirement where we’d like to show “Child” Todos in a column if their parent is not present in the column. For example, if we have 2 Todos:

todo1 = {
  title: "Re-style login screen",
  labels: ["Story"]
};

todo2 = {
  title: "Fix email validation",
  labels: ["Bug"],
  parent: todo1._id
};

Then we would want the “Story” column to show todo1 and the “Bug” column to show todo2.

To achieve this, given our current data model, we want the server to only return Todos that are “Top-level”, which are now defined by the following condition:

Todos that are not children && Todos who’s parent does not match the column label

So server publication (using Publish Composite) looks like:

  return {
    find() {
      return Todos.find({ labels: label },{ limit });
    },
    children: [
      {
        find(childTodo) {
          if (childTodo.parent) {
            return Todos.find({ _id: childTodo.parent });
          }
        },
      },
    ],
  };

Note: We return todo.parent doc in this publication so that on the client we don’t display a child Todo in a column if it has a parent in the same column. This was an issue caused by our “limit” returning a child before a parent.

This solution works okay for the most part…

We start to run into issues though, when the publication returns 9 child Todos (We’ll say label “Bug”) and 1 parent Todo (also label “Bug”). This means that in our “Bug” column, only 1 item (the parent) should display.

The 1 Todo displaying does not match the limit we intended from the server.

We could wait until the subscription is ready, check if the amount we choose to render matches our “real” limit, and then increase limit if it does not. But at this point the whole thing seems overly complex for what we’re aiming to achieve.

Is there a better pub/sub model you can think of here to solve this issue? Is there a tweak to the data model that can improve reduce complexity without requiring a massive increase in db writes?