Server Performance issue, Wait time getting high on methods, Getting slow


#1

Hi, I’ve built a sample chat application with Meteor 1.4.1. Here I’m trying to do a stress test by sending 10 messages per second. to monitor this I’m using Kadira APM.

Click here for the Trace for one send method.
In case link got expire here, I’m attaching the screenshot as well.

I can’t even user Meteor.defer or this.block here because I need to send the response to the client confirming whether the message is sent to the server or not.

Any help or tip will be helpful, thanks in advance.


#2

Any workaround solution to this?


#3

Could you post your code? That does look like it was implemented wrong.


#4

You need to use this.unblock() in your methods and publications. Performance will always suck until you do this.


#5

@rjakobsson here below is my send method

    send:function(msg){
      let st = process.hrtime();
      if (! this.userId) {
                throw new Meteor.Error('error-not-authorized','User need to login', {method: "send"})
            }
      //check for the validation
      NonEmptyString = Match.Where(function (x) {
        check(x, String);
        return x.length > 0;
      });
      check(msg, {
        id: String,
        msg: NonEmptyString,
        rid: String,
        lts: String
      });

      let u = Meteor.user()
      let r = Rooms.findOne({_id: msg.rid})
      
      if(! r){
        throw new Meteor.Error("error-room-invalid", "Invalid Room id", msg.id);
      }

      //find the recepient user and check if he is online or not, if not send a push notification.
      let ru = Meteor.users.findOne({$and: [{username: {$in: r.users}},{username: {$ne: u.username}}]})

      if(! ru){
        throw new Meteor.Error("error-invalid-recipient","No recipent user found", msg.id);
      }
      

      // TODO: Check for message limit
      let record = {
                _id: msg.id,
                msg: msg.msg,
                rid: r._id,
        u: u.username,
                ts: new Date(),
        lts: msg.lts
            }
      let result = null;
      try {
        result = Chats.insert(record);
        if(result){
          //increase the msg counter
          Rooms.update({_id: r._id}, {$set: {mc: r.mc+1}})

          if((ru && ru.status) ? !ru.status.online : true){
            Meteor.defer(function() {
              Meteor.call('sendNotification',record,ru._id)
            })
          }
        }
      } catch (e) {
        if (e.name === 'MongoError' && e.code === 11000)
          result = record._id;
        else
          throw new Meteor.Error('error-500','Something went wrong', e)
      }
      let et = process.hrtime(st);
      return {_id: result};
    }

#6

@veered I can’t use this.unblock() here because I need a response back from the method. Though I’ve used this.unblck() in my subscriptions now.

This issue arises only when I try to send messages like 10 messages per second.


#7

this.unblock() doesn’t stop you from getting a response back from a method, it just makes it so that other methods/publications can be processed while this one is running (with the magic of fibers).


#8

In the above send function, How can I restructure it with this.unblock ? should I use with every DB operation?


#9

Just call this.unblock() on the first line


#10

Thanks, It helps, I had the wrong assumption about the this.unblock() function.


#11

You need to use this.unblock() in your methods and publications. Performance will always suck until you do this.

Always wondered why this isn’t the default behavior …


#12

isn’t this could be overhead to use in all the methods and publications. There could be some drawback of using this all the times.
Here I can use this for methods like send , read, and delivery report.


#13

Because it can cause very hard to debug situations. The nice thing about the meteor calls now is that they work in serial. Unless you unblock off course.

A simplistic example here: https://meteorhacks.com/understanding-meteor-wait-time-and-this-unblock/


#14

@lucfranken
It is clear now. Thanks


#15

I think that it’s best to use this.unblock in almost all situations, for both methods and publications. Qualia has a pretty sophisticated wrapper around all methods and publications, that makes this the default (and automatically does other things, like permissioning, rate limiting, debouncing, logging). We have yet to encounter a situation where this has caused issues.

If you don’t use this.unblock you will struggle to achieve acceptable performance. Unless you call this.unblock literally ZERO other methods or publications will be run until the current one completely finishes. For long running methods/publications that’s a total disaster.