Query regarding server bottleneck optimisations


#1

I’m trying to debug my application to find the cause of a huge bottleneck in the server code and was hoping you might be able to clarify something for me. I use a package called collection-hooks and I wondered if it might be the cause of this…

First off, the info, here is the Kadira profile:
https://ui.kadira.io/mt/86d5a109-6eff-4aa7-bf22-06a5ff556ee0/q56rPZsBwrGWWcykj

Note, I have no hooks at all on Users and no PrintLists hooks for find or findOne.

As such the expected behaviour would be:

  1. fetch on printLists
  2. fetch on user
  3. update on printLists
  4. update on quantityLists

Obviously there is a LOT more going on than just that… So I wondered if the collection-hooks might be responsible for the extra find/fetch’s even when there is no hook defined to utilise the result?? Or am I clutching at straws?

And now the method, followed by the one hook that relates to PrintLists.

PrintLists.methods.addPrintItems = new ValidatedMethod({
  name: 'PrintLists.methods.addPrintItems',

  validate: new SimpleSchema({
    printListId: { type: String },
    items: { type: [Object] },
    'items.$.product': { type: String },
    'items.$.template': { type: String },
    'items.$.quantity': { type: Number },
  }).validator(),

  run({ printListId, items }) {
    const printList = PrintLists.findOne(printListId);
    if (!printList) {
      return throwError('not-found', 'The specified print list does not exist');
    }
    userIsOwner(printList);

    // For each item that is to be added to the print list
    _.each(items, (printListItem) => {
      // Check to see if there is already an entry in the
      // print list for this exact product & template combo
      const existingItem = _.findWhere(printList.items, {
        product: printListItem.product,
        template: printListItem.template,
      });
      if (existingItem) {
        // There's an existing entry, so just add the quantity
        existingItem.quantity += printListItem.quantity;
      } else {
        // There's no item for this combo so add one
        printListItem._id = Random.id();
        printListItem.overrides = [];
        printList.items.push(printListItem);
      }
    });

    // Overwrite the old items array with the new one
    PrintLists.update(printList._id, {
      $set: { items: printList.items },
    });
  }
});
PrintLists.after.update(function (userId, doc) {
  // Get a list of valid print list item ids
  var validPrintListIds = _.pluck(doc.items, '_id');

  // Remove all quantity list items that reference non-valid print list items
  QuantityLists.update(
    { printList: doc._id },
    { $pull: { items: { printListItem: { $nin: validPrintListIds } } } },
    { multi: true }
  );
}, { fetchPrevious: false });

#2

Looking at the profile, I would try some simple optimisations first (mainly because I’m not sure whether collection-hooks is your problem here):

  • Unless you need to run the method on the client and server, I would be inclined to either use a standard call/method or wrap the workings of the run method in an if (Meteor.isServer) block. Other than the validation, the rest of the code may as well run only on the server.
  • I would also consider using this.unblock() inside the method to allow multiple methods to run in parallel.

#3

I need the method to run on the client as well as the server in order to provide optimistic updates, otherwise they have to wait for the response from the server, which is executed in parallel (to my understanding?).

I agree that I could unblock the method, and this is something I will try.

However, if anyone else finds themselves in a similar situation, I believe I may have found the culprit, or at least a decent chunk of it…

I have custom validation and autoValue functions set on the PrintList schema (simpleSchema) which do a reasonable amount of logic. Now these should only be run if this.isSet is true, but I’d somehow managed to forgo that bit of logic, meaning that all the validations were running every time any update was performed, even if the update didn’t touch the value being validated.

I’ve since corrected this mistake in the latest code and am waiting to deploy it and test it.


#4

Correct - that is the obvious use case. I just wasn’t sure how essential it was for you.