Large client collection slows down browser


#1

Any one run in to this before?

I’m loading 225 documents with 20 fields each

My browser begins to behave very laggy. Anything for optimization here?


#2

Duh, never mind. I was returning all fields for like, 225 x 80. Limited fields and bam, faster.

Still I suppose the question remains, how does one efficiently manage say, 1000 documents in a client collection?


#3

You don’t load all the documents at once.


#4

They’re not loading all at once, they come in a que.

But at 250 collections it starts to lag out. Moving to something like 3000 collections just locks up the client.

This is an exercise is maximums I suppose. Maybe 250 is about it.


#5

I think with this kind of stuff, you want to be really conservative. If you look at Facebook, Amazon, etc, they usually load like 20-30 items at a time. Maybe pre-load for the page ahead to make it seem instant, and that’s it


#6

Yup, loading too much data is a clear no-no.
Though you can go redis way if you are :chipmunk:


#7

Links? What is redis?


#8

Here’s an example of my live application. I’ve got 129 documents, and the client is eating snails. Is this really the max?

The whole browsers about to explode! I have a fix I can build but, damn.


#9

are you also rendering 129 components into the view? that’s probably what’s doing it


#10

Yes I’m rendering, but when I disconnect the database from Meteor, I can render 3000+ objects with no issues.

When I load up Meteor and simply put 300 documents on MiniMongo it dies.

It just seems like yeah, that’s the limit.


#11

Have you tried profiling with dev tools to find out what’s actually eating up your resources?


#12

129 documents like the one you showed in the screenshot should be NOTHING, unless they’re all changing every second?
My app subscribes to like 10 or more collections often with a couple of hundred items each.


#13

that’s because it has to re-render each time a bit of new data goes in. you need to figure out a way to manage that :slight_smile:


#14

Again, I can take the Canvas rendering engine aside, and put through a MASSIVE amount of data before it chokes.

On just a regular Meteor project, I can’t load that much data. I’ve got some work arounds, and the more I optimize stuff, the more it behaves.


#15

They do ALL mostly change on 1 second intervals. Maybe I’ll need to que that data…

Thanks for sharing your numbers. How many fields in each document do you have?


#16

Agree with @herteby, that should be nothing.

Did you try optimizing mongo via index and oplog?


#17

Index and oplog doesn’t affect the client though.

@SkyRooms That could be the problem. I don’t have any experience with data that changes so frequently though, so I really don’t know what the limits are. Still sounds weird that it would freeze the browser I think. What happens if you remove your own code that uses the data and just subscribe?

Also, Meteor DevTools is pretty poorly optimized and can definitely hang the browser, so try without that maybe :stuck_out_tongue:

My admin>customers page subscribes to all 2300 customer users, with up to 40 fields each! Terrible optimization I know, I’m in the process of overhauling the whole app. But it still runs okay!


#18

I assume they might affect client subscription if the published collection changes so much? One thing I would check…


#19

If it’s not a hard requirement that the data changes be reactive (user sees active changes every second) have you thought about making the subscription non-reactive, then on specific interval manually update the view - say every 10 seconds or something - you’d have to handle the scroll positioning if that is an issue though.


#20

Yeah but where it’s a game, it’s gota be reactive. That was the whole point.

I actually found that changing the limit on a data pull had a great effect. I opened it from 10 to 30, and it seems to have helped greatly.