Unsubscribe - subscribe issue with large users collection


#1

Hi,

Running into an issue when I route from one Route to another. Both routes have a subscription to a user publisher.
However, the first Route subscribes to a Client side only collection, with around 2300 users.
When I route to another Route, the contact list, which has a subscription to all users with a limit of 20, Chrome browser hangs for 10 seconds and then starts rendering the list with a lot of undefines. Then it repaints and the list is correct.

So what happens is this: when I move away from the first page, the subscription should automatically be removed. (I even called the .stop() in the template.destroyed handler).
However, when I directly route to the Contact list it subscribes to users again (different subscription though). Apparently it cleaning the current subscription runs into a conflict with adding the new users to mini mongo.

Interesting enough, when I go from Route 1 (with the big client side collection only) to a Route 3, one that has a Composite subscription of classes with a child subscription of users (only 8), this loads immediately.
This is probably because the subscription is small, however Route 2 with the Contact list subscription has a limit of 20. So the client should also receive a small collection.

Am I running into a Meteor issue here?


#2

Hi @mspi

It’s pretty tough to diagnose a code-related issue from a textual description.

What would help explain your issue & get better responses -

Create a minimal reproduction on github that I can clone, examine and write a pull-request/ fix.

When I do this for my own issues, I find that in the process of creating a reproduction, I discover a way to solve the issue.


#3

Hi @nathan_muir

Yeah, you are right. However there is a good chance that in a small reproduction the issue doesn’t occur :frowning:

For now I can only describe the essence of the issues:
Route 1: big mini mongo user collection with Search Source (shouldnt affect it).
then we go to Route 2 which has a normal subscription of all users with a limit of 20.

It really looks like Chrome is just frozen because the mini mongo collection is emptied and at the same time filled by the new subscription of Route 2.
Probably one of those process is not blocking and causes an issue, or somekind of loop. But not infinite…it does recover after a while.

I can look into writing a little test case when I have some time off.


#4

A typical case where the browser hangs is when Blaze receives too many update orders. So if you remove the rendering part (i.e. you don’t display anything on the page), things will probably come back to normal.
But then you need to understand why Blaze goes into panic, which usually relates to too many reactive stuff depending on a bad subscription scheme.
You definitely need to test on a small app, without Iron Router.


#5

@mspi in Route 2 are you filtering what is displayed by the same limit for the subscription? What might be happening is when you initially start to render the page, it starts to render items that are then removed immediately after that. If this is the case, blaze is likely having to instantiate 2300 template partials , start to render them to the DOM, and then tear down all of them. If you filter in you template helper or data context it will eliminate this issue.


#6

check out this discussion. It might help you.


#7

@copleykj, this is the code. It is just subscribing to the first 20 users and then returns those 20 with a helper.
So before it hits this code, in Route 1 there was a huge mini mongo collection with 2400 users. Then I Iron Route to Route 2 where to code below is executed. It just subscribes to 20 users.

UPDATE
@copleykj I can confirm that when I comment the each loop in the template that prints the contacts, that loading of the page is instantly. So it is related to the fact the Blaze probably tries to render everything in minimongo while that is being updated to a new subscription.
The helper returns the records with the find() already has the limit… SO not sure how to resolve this one yet.

Template.contacts.created = ->
  instance = @

  instance.sort =
    'profile.firstName': 1
    'profile.lastName': 1

  instance.increment      = 10
  instance.limit          = new ReactiveVar 20
  instance.filter         = new ReactiveVar null

  # if right drawer is already open, reload the filter template
  if !Session.get( 'rightDrawer-panel-narrow')
    Session.set( "rightDrawerTemplate", 'filter_contacts' )

  instance.autorun ->
    filter = Session.get( 'filter.contacts' )
    instance.filter.set( filter )

    # if filter
    instance.contacts =
      subscription: instance.subscribe('contactList', instance.limit.get(), filter )

  instance.infiniteScroll = (e) ->
    scrollElement = $(e.currentTarget)
    scrollHeight = scrollElement.prop('scrollHeight') - scrollElement.prop('offsetHeight')

    if instance.contacts.subscription.ready()
      # Load more when the user scrolls to the last 10 contacts, which is about one page height
      if scrollElement.scrollTop() > scrollHeight - 70*5
        instance.limit.set( instance.limit.get() + instance.increment )

Template.contacts.rendered = ->
  instance = Template.instance()
  scrollElement = $('#contacts.infiniteScroller')
  scrollElement.scroll _.throttle(instance.infiniteScroll,100)

Template.contacts.helpers
  isReady: ->
    instance = Template.instance()
    return instance.contacts.subscription.ready()

  contacts: ->
    instance = Template.instance()
    return Meteor.users.find({},{sort: instance.sort, limit: instance.limit.get()})

#8

FIXED, thanks to @copleykj and @steve insight on the rendering part :slight_smile:

contacts: ->
  instance = Template.instance()

  # only return users when subscription is ready. This case applies especially for when you directly navigate from email or sms compose with the large minimongo collection of users
  if instance.contacts.subscription.ready()
    return Meteor.users.find({},{sort: instance.sort, limit: instance.limit.get()})
  else
    return []

Adding the check if the subscription is ready before returning the users in the helper fixed it. Now the template doesnt start rendering the users that where still in the collection during the Routing.

Is this still a blaze error? Shouldn’t Blaze separate the scopes in this case?


#9

The fact is that, right now, you have to put Blaze “on hold” while manipulating large subscriptions. In certain cases, you can even crash the browser. This has been acknowledged by MDG as an area of improvement:

dgreensp commented on Apr 1
Just to get this out of the way up front: Our overall performance definitely needs some work when it comes to receiving data over the network, storing it in Minimongo, pulling it out, and rendering it via Blaze.

(source: here)


#10

okay, great to know if has their focus.
I also notice that mobile performance could improve, especially when I include SearchSource and on keyUp refresh the list of results. It works smoother on a new iphone 6, but on my not so old HTC One (M7) the browsers starts to get sluggish.
On my macBook is super fast.


#11

to make navigating smoothly try

if you navigate a lot back in forth in your app and you do not want your user list always pulled from the server (bandwidth)

I see no reason to load the full list and than incrementaly load 10 users and the next again when the full list is already at the client. and the 10 is only a display issue.

just thoughts i do not know how your app is structured


#12

I pass the limit to the subscriber, If I remember well. So I am not loading everything and then displaying small portion only. Just displaying anything that is offered to the subscriber.