How to get count for pagination in meteor publication and subscribe?

help
1

I am stuck trying to find a way to get count. It seems if i subscribe to a particular collection, it will send all the collection and if i put skip and limit by 100 it sends only 100.

here is my Meteor function in client

  let defaultPageSize = 100;
  let page = 0;
  let params = {}
  let ctx = {
    sort: { createdAt: -1 },
    limit: defaultPageSize || Number(props.queryParams.pageSize),
    skip: (Number(props.queryParams.page) || 0) * (defaultPageSize || Number(props.queryParams.pageSize))
  }
  if (props.queryParams.orderId) { params['orderId'] = new RegExp(props.queryParams.orderId, "i"); ctx = {}; }
  let subscribe = Meteor.subscribe('orders', params, ctx);

The Following is for the server

// Stocks publish
Meteor.publish('stocks', (params = {}, ctx = {},) => {
  return Stocks.find(params, ctx);
});

Now since by default i want 100 items, and when i do the count in the frontend , the total will be 100.
Any idea or suggestion to get total and still be able to do pagination.

Regards,
Sijan

2

A few ways to do this. You want to do count on the server and then send the total to the client (method, aggregation) and limit data retrieval only to the the page you are requesting.

One way is to do a method that returns the count. Another option is to use performant count or similar approach.

1 Like
3

I use the count example from the pub-sub documentation of Meteor

1 Like
5

If in client i do let subscribe = Meteor.subscribe(‘orders’, {}, {limit:100}); and i do the count, it returns 100 items only , but in DB there is more than 200. Am i missing something from the documentation ?

6

You need to combine skip with limit:

  • skip: 0, limit: 100 returns docs 1-100
  • skip: 100, limit: 100 returns docs 101-200
  • skip: 200, limit: 100 returns docs 201-300

and so on.

Note that you also need to apply a sort, to ensure docs are always selected in a consistent, repeatable order. For example, if you sort by the document create date, you will always get the oldest doc first and the newest doc last. Obviously, you’d need to add a suitable create date field to each doc for this to work :wink:

2 Likes
7

I am using the skip and limit,

let ctx = {
    sort: { createdAt: -1 },
    limit: defaultPageSize || Number(props.queryParams.pageSize),
    skip: (Number(props.queryParams.page) || 0) * (defaultPageSize || Number(props.queryParams.pageSize))
  }

The issue i am facing is with getting the total count. i think i will need to have a server method send me that info on mount. :confused:

8

The easiest solution (at least what I’m doing) is to use publish-counts. That way you can publish the total count within your publication and then use client helpers to get the total number. The documentation for the package is pretty good.

1 Like
9

If you are getting the count, then why do you have a limit in your subscription?

If you are doing a paginated pub-sub, there must be two separate subscriptions:

  1. For the actual data with filters, limit and skip
  2. For the count with just the filters
10

I copied this from one of my pages requiring subscription for paginated data.

You can see the second subscription just for the count.

export default withTracker(props => {
    let paginatedInputs = {
        itemsPerPage: props.itemsExpectedCount,
        offset: props.itemsOffset,
        filter: props.filter
    };
    let handle = Meteor.subscribe(
        'errors.paginated',
        secureInputs(paginatedInputs)
    );

    let countInputs = {
        filter: props.filter
    };
    let handleCount = Meteor.subscribe(
        'errors.count',
        secureInputs(countInputs)
    );

    let result = ErrorLogs.find(
        {},
        {
            sort: {
                date: -1
            }
        }
    ).fetch();

    let errorsCount = Counts.findOne('errors');

    return {
        errorsCount: errorsCount ? errorsCount.count : 0,
        errorLogs: result,
        handle,
        handleCount
    };
})(ErrorsList);
11

Let me please explain my situation and discovery,

I have around 10000 document in a collection, and when i subscribe without limit, it is taking a long time to display 100 records, (approx 20 secs) . // (I was still using limit in client side after subscribe without limit )

I used the meteor dev tools to see what was the issue, and found that the process waits for all the documents to be downloaded then only displays.
Now i find this is really unnecessary as i could just limit to 100 documents at a time and display the data instantly (less than few 100ms).
Now to achieve this, i need to paginate, i need to show the number of pages. ex if i want 100 per page , 100000/100 = 100 pages). To show to the users that there is 100 pages, i need to know total count.

I hope i could explain you my situation clearly. Please let me know if i could still use the count and achieve or any alternate solutions for this.

Thanks in advance

12

We are talking about two (2) separate subscriptions here, just to be clear. One to query the limited data (e.g. 100 records of 10,000) and another to get the total count (e.g 10,000 as a number)

Have you tried this example from the pub/sub documentation?

// Publish the current size of a collection.
Meteor.publish('countsByRoom', function (roomId) {
  check(roomId, String);

  let count = 0;
  let initializing = true;

  // `observeChanges` only returns after the initial `added` callbacks have run.
  // Until then, we don't want to send a lot of `changed` messages—hence
  // tracking the `initializing` state.
  const handle = Messages.find({ roomId }).observeChanges({
    added: (id) => {
      count += 1;

      if (!initializing) {
        this.changed('counts', roomId, { count });
      }
    },

    removed: (id) => {
      count -= 1;
      this.changed('counts', roomId, { count });
    }

    // We don't care about `changed` events.
  });

  // Instead, we'll send one `added` message right after `observeChanges` has
  // returned, and mark the subscription as ready.
  initializing = false;
  this.added('counts', roomId, { count });
  this.ready();

  // Stop observing the cursor when the client unsubscribes. Stopping a
  // subscription automatically takes care of sending the client any `removed`
  // messages.
  this.onStop(() => handle.stop());
});

You cannot accomplish what you are trying to do with just one (1) subscription.

1 Like
13

If you have two subscription it will not work, you already pass on to client all the data if you sub to everything for the count, then whats the point of server side paginagtion? the whole point is to pass as little as possible data to have client store in the memory as less as possible
so there has to be one subscription which brings the needed data only

as for the total, we need to have a way for the client to listen to the collection and when its updated to bring the total, i have no idea how to do so