Update of several items kills my app

help
1

In my app I have 2 collections linked like this:

collections

// {name:"name of group",member_ids:[member_id1,member_id2]}
groups = new Mongo.Collection("groups")
//{name:"item name",group_id:my_group_id,group:groupObject}
items=new Mongo.Collection("items")//

hooks (/matb33/meteor-collection-hooks)

groups.after.update(function(userId,doc,fields){
   if(Meteor.isServer && (_.contains(fields,"member_ids")) 
         items.direct.update({group_id:doc._id},{$set:{groupObject:doc}},function(err){if(err)console.error(err.stack)})
})

publish

Meteor.publish("items",function(limit){
    return items.find({"groupObject.member_ids":this.userId},{limit:limit||100,fields:{groupObject:0}})
})

This does work and doesn’t bother in a fresh app.

But when I do it in my app where I have more code, more collections and more items, items.direct.update causes a huge load on node.js.

  • meteor 1.1.0.3 : jumps from 0-2% to 100% of UC with 1 update and 90 items
  • meteor 1.2-rc.3, 1.2-rc.4 : jumps from 0-2% to 30-40% of UC with 1 update and 90 items

What could/should I actually do ?

This are some screnshots from CPU profiler with meteor 1.2-rc.4

1499×596 88.7 KB
1618×470 79.8 KB
1533×481 101 KB

2

What does direct.update do?

3

it bypasses the collection’s hooks so that they aren’t triggered

4

I see. How many rows are you updating at the same time?

5

Exactly 91 items. ()

6

And if you run these same operations on mongo shell itself, it does not produce such load?
With and without meteor running ?

7

And if you run these same operations on mongo shell itself, it does not produce such load?

For 1 update, it goes to 4%. That is acceptable.

With and without meteor running ?

Yes

I lightened the load through this :
I removed all fields from groupObject that weren’t related to a “member_ids” field using SimpleSchema and Collection2. It dropped the load from 30-40 to 4-8%.

However this is not satisfying.

I wasn’t sure that it’s related to other posts. Now I know that it’s related to live query but I can’t use “Processing High-Velocity Time-Series Data” nor a case where I can use joins or reactive joins .