I did some googling and changed the code to this, which seems to do the trick. 5000 items loading at once isn’t great, but baby steps…
How could this be improved?
Template.wordsList.helpers({
words: function(){
var words = Words.find();
return words;
}
});
Tracker.autorun(function () {
var query = Session.get('query');
Meteor.subscribe('filteredWords', query);
});
Template.body.events({
'submit .the-form': function (e) {
e.preventDefault();
var query = ...;
Session.set('query', query);
}
});
Hi,
New to Meteor a couple of days ago, but liking very much what I see.
I have a test app that I’m building. It has a large number of records which will be searchable by query by the UI.
I can get the subscription working, but the next time I query, the new data is being appended to the existing data, whereas I need it to overwrite to become fresh data.
Meteor.publish('filteredWords', function publishFilteredWords(query) {
var pattern = ..; // omitted for brevity
var search = new RegExp(pattern, 'i');
var words = Words.find({word: search});
return words;
});
Template.body.events({
'submit .new-task': function (e) {
e.preventDefault();
var query = ...;
Meteor.subscribe('filteredWords', query);
}
});
Not sure what you mean by “overwrite”. If you overwrite data on the server, it will (or at least “should”) overwrite in the client. I don’t think this is what you mean.
I think what you are trying to do is pagination, or some form of pagination at least. In this case, the data is still there, you are simply display only a part of it.
There are several ways to accomplish this. One would be to simply publish everything and limit what you show on the client. This would work fine if you only have a few records, but if you are dealing with any significant amounts of data you are going to have bad performance in your client.
Another solution would be to send in a offset and a limit, but I have read you end up with more problems due to how MongoDB handles the offset/limit. You could fix this by having an indexible field to server as your offset, which would make MongoDB skip ahead as one would expect. I think the key here would be how you sorted the data, as how you jump the offset would depend on how you sorted the data. There is a video I found that demonstrates this problem and at least one possible solution to fixing it.
Unfortunately there is no official Meteor way to handle pagination yet (that I know of), but I believe it is on the roadmap. In the meantime, there are several packages out there that can help.
The UI will allow the user to enter criteria to search, which needs to return a fresh list of items from the collection. The next time they do a (different) search, they need a new set of items.
What I’m finding is that both result sets are remaining in the local collection instead of the first set being cleared and replaced with the second set.
There are some known issues: [1], [2] of overlapping subscriptions, but these generally resolve after a short period of overlap. If you are seeing persistent overlap it sounds more like an issue with how you have defined your pub/sub.
Do you have a minimum reproduction repo to look at?
I’ve just updated my post. I’ve found trackers, and when I changed to using it, even though I’m returning large recordsets, it seems to change ok, albeit slowly.
I don’t know if this is best practice or not though?
Goog to know I wasn’t completely on the wrong track. Maybe I’ll have to limit and use infinite scrolling so the data minimal data changes promptly.
Could have more data than you expect if autopublish package is in there, but no matter, your query should limit the results.
Setup the publication to return the items you want. Then setup a subscription to use the publication, finally setup your query to define the data for your template. If the collection has more data that you need, no matter, that’s why the query is for.
I made sure to turn off autopublish before I posted.
The problem with returning more data than I need is that there are over 200,000 records and the users’ query is done over the entire collection. I don’t want the entire collection sent to each client.
There cant be more data than you are subscribing to.
BUT if you dont sort on client (what you are not doing based on code provided), it will look unsorted, cause resubscribe only remove records you dont need in new subscription and the ones you still need stay as 1st, than add new data to fill that limit for records which are sorted from server.
So it seems unsorted, but the documents in list should be there as they are result of query, they just are not sorted on client.
Kinda hard to debug when you did not show us query tho…