How do I filter my publisher and also how do I limit my collection items

1

I have a collection of Weather stations sensor data. So each station sends in a data array that is inserted into the collection.

`Nodes.insert({name: "name",humidity: #, temp: #,longitude: #,latitdue: #});`

This works and I have been able to move along through this by grabbing all the unique names and finding my specific collection items based on that.

However I have run into a snag because I when I load a page that needs to sift through thousands of data points just to grab the latest 20 points and then plot them the page takes forever to load. So I need a way to Publish only the latest 20 points of each uniquely named collection array. There is a createdAt time for each of these collection items based on the Schema I have made.

Also I have not figured out how to limit the number of collection items per name.

2

Not sure if I am getting you right.
Check out the docs for limiting a publication with limit options. And the metoer guide for pagination it shows how to sort and limit.

Make sure you have an index on the createdAt field and the name field too.

3

There is no easy way to create a publication that will publish last 20 items of every station grouped by name. Meteor supports Mongo aggregation (via 3rd party package), but it IIRC it will not be reactive.

What you can do is cache your station names in a separate collection, and then get the client to subscribe to them separately, limiting each to 20 results.

4

Thank you @M4v3R I ended up doing that and had it work. Although I do need to use the entire collection at one point to download the data. If I make a button event and in this event subscribe to the entire publication.Can I have multiple publications for one collection? One publishing the entire collection and one just for the last 20 points graphing?

5

Thank you @paul_bo this helped me to create the right publication so I was not sending the entire collection.Can you have 2 publications for one collection?

6

Yes. When you subscribe to both publications, they’ll be combined when accessing the collection, though.

7

If you are looking for aggregations check out this aricle from the meteor chef.

1 Like
8

Amazing. I have seen very many references to aggregation but I never knew what it was. Thank you for the link. I did not read it completely because I have an exam tomorrow.

However I do have a question. Essentially with aggregation I could just have different id’s to ask for in my subscriber/publisher? Also I have no idea what any of the names are. They are sent to me so I have to find them in the collection, I use the name as an ID, because each collection item has a name they are not always the same but the ones that are are going to be used together to create graphs.

Although the graphs only use 20 data points for the graph so I only subscribe to 20 points. However I need to download the entire thing into excel.

Here is my real question off of that info. Can I with aggregation limit the length of each unique name.So example name: “chicago station”, there can only be 7500 “chicago station” colleciton items. And as new ones come in I need to dynamically delete the oldest one so it stays at 7500 collection items.