[SOLVED] high cpu usage on relatively small db -- cfs:gridfs

help
1

We are having issues with our application. It gives a spikes of high CPU usage on every page load. We took a profile with Kadira and most of the time is taken by these:

  • receiving data from mongodb
  • parsing incoming mongodb documents
  • package:underscore

We added indexes, messed with some “big” collections (like 50k rows)… and then, all of sudden, understood what the problem is! It was cfs:gridfs. If you store images with it, especially big images, very fast it brings MiniMongo down. Never ever use it for uploaded images. Only for some small user files. Have a good luck )

2

I don’t think that is the problem. cfs:gridfs does not publish the images themselves to minimongo, it serves the images/files over http so it has no way to bring minimongo down.

In fact I am using cfs:gridfs on an educational game project that stores learning material (huge pdf files) and test questions (with lots of images) with hundreds of students subscribing to those collections simultaneously during the course of a live quiz/test and I have not experienced any slow downs.

Perhaps I can help you pinpoint the actual bottleneck if you share more about your set up.

1 Like
3

Hmm… but after i switched to cfs:filesystem, the problem is gone. may be the case is with the custom observers we have put on image upload? We did it like this:

observer = Images.find({_id: fileObj._id}).observe
    changed: (newDocument, oldDocument) ->
        if newDocument.copies && newDocument.uploadedAt && newDocument.copies.thumb && newDocument.copies.images
            #thumb and images are names of collections
            #file is uploaded.. do some stuff
            observer.stop()  #this line was added recently

and for some time we didn’t do the observer.stop(). Can it be the case? Is it possible for them still to live?

4

we didn’t do the observer.stop(). Can it be the case?

Definitely yes! The official docs about observe states that explicitely:

observe returns a live query handle, which is an object with a stop method. Call stop with no arguments to stop calling the callback functions and tear down the query. The query will run forever until you call this.

Furthermore, observeChanges is much more efficient than an observe since it only deals with actual changes.

1 Like
5

so, how to see the list of currently running observes and stop them?

6

Take a look at percolate:server-info, facts and of course kadira

1 Like
7

OK, here’s a piece of magic! Put it to meteor shell as one-liner or run somehow in other way on server-side. Only if you really need to stop all active observers and you know what you are doing.

_.each(MongoInternals.defaultRemoteCollectionDriver().mongo._observeMultiplexers, function(muxer) {
    _.each(muxer._handles, function(handle) {
        handle.stop();
    })
})
1 Like
8

With great power comes great responsibility :wink:

1 Like