Best and optimized way to implement subscriptions (Server & Client)

help
1

I want know the best possible implementation of subscription or reactivity for both Server and Client side.
I am using IOS Native client app using SwiftDDP.

I have a collection called: Deliveries
Here is the schema for Deliveries collection:

{ _id: '3F3q2sw', title: 'Parcel delivery', price: 230, flag: 1, trackingStatus: 2, loadingAddress: 'some address and coordinates', deliveryAddress: 'some address and coordinates'... }

Note: Deliveries is a heavy collection, it has more than 100 keys in each record.

  • I am developing delivery native app on IOS using Meteor server.
  • I want to show all the latest related deliveries to that delivery man on his IOS device.
  • He will see the list of these deliveries.
  • Showing very latest 10 per page but he can scroll and load more.
  • So when he will swipe to load more he will get 10 older jobs appended in that screen.

Here is the server side subscription:

Meteor.publish("subscribeRelatedDeliveries", function (params) {

  params = params || {};

  if(!this.userId){return Deliveries.find({_id:''});}
  
  var perPage = 10;       
  var pageNumber = parseInt(params.pageNumber) || 1;

  condition.flag = { $lt: 2 };
  condition.trackingStatus = { $lt: 3 };
  
  return Deliveries.find(
    
    condition, 
    { 
      limit: perPage,
      skip: (pageNumber - 1) * perPage,
      sort: {'cDate.timeStamp' : -1},
      fields: { 
          'title': 1, 
          'flag': 1, 
          'loadingAddress': 1,
          'deliveryAddress': 1,
          'trackingStatus': 1, 
          'price': 1, 
          'cDate': 1                    
      } 
    }
  );
});

Now when IOS app swipes down, it sends page number to server subscription like: 1 2 3 4 … and so on.

On each swipe server subscription provide next 10 older deliveries from server side.

The issue here is that the very first latest 10 deliveries are no more reactive, if anything changes like trackingStatus, I dont get ping on onDataChange Event. It means that only the last 10 called Deliveries records are reactive, and those deliveries which were called before that, are no more reactive.
Also I am not getting the ping of new latest delivery which is currently posted by some user.

What should be the best and optimized way to handle this situation?

2

You will need to do some changes to the subscription to return the documents on all pages, so you will limit the query like this and remove the skip. That will make all the documents reactive, currently is not because you are “excluding” (skiping) all the documents that are not in the current page from the query.

limit: pageNumber * perPage

Meteor.publish("subscribeRelatedDeliveries", function (params) {

  params = params || {};

  if(!this.userId){return Deliveries.find({_id:''});}
  
  var perPage = 10;       
  var pageNumber = parseInt(params.pageNumber) || 1;

  condition.flag = { $lt: 2 };
  condition.trackingStatus = { $lt: 3 };
  
  return Deliveries.find(
    
    condition, 
    { 
      limit: pageNumber * perPage,
      // skip: (pageNumber - 1) * perPage,
      sort: {'cDate.timeStamp' : -1},
      fields: { 
          'title': 1, 
          'flag': 1, 
          'loadingAddress': 1,
          'deliveryAddress': 1,
          'trackingStatus': 1, 
          'price': 1, 
          'cDate': 1                    
      } 
    }
  );
});

Hope this helps

3

@pmogollon Thanks for the response. Actually I have already removed the skip part and getting all the documents, but as I mentioned above that I need optimized solution. In case I have 1000+ records so in that case a user can miss use my app by calling all those 1000 records by spamming when he will be on the last page.

Also one more thing to ask, I have 20 records, and I am at the last 20th records, so if I keep swiping load more again and again, it will keep fetching 20 records over and over again. This will put a heavy load on my server.

So how can I decide that there are no more records? Since I am getting all the records in onDataEvent over and over again

4

The issue is that if you want every document on that query reactive, you will need to publish all docs. What I usually do for what you want is not use infinite scroll but pagination. Or you could do something with methods and refetching every X ms.

For the count I think the best way would be a method call that returns the count, there is a package that allows you to return the count in a subscription but is not very performant. I usually prefer to use methods for counts even if is not reactive.

Also if you are looking to improve the performance when using publications you can check out redis oplog.