Fast in-memory search with Meteor


#1

Hi all!

At Qualia we’ve been working on improving search in our Meteor app. We were using EasySearch to construct MongoDB queries based on regex selectors, and we were having trouble getting search to behave the way we wanted. In particular, it was hard to prioritize exact matches for the search queries, and there wasn’t an easy way to use one search query to look across multiple fields on the same document in the collection.

We decided to maintain a projection of our collection in memory on the Meteor server, and then use plain JavaScript to search over it. The result is pretty fast and really flexible!

Check out our blog post about it, and the new libraries, MappedCollection for projecting a collection into memory and lasr for search!


#2

This looks great! Thanks for sharing this :smiley:


#3

Cool strategy! I’m curious about the scalability? The in-memory cache seems like it would be pretty scalable but I’m also wondering about the expense of that cursor.observe and how closely that is tied to the size of the collection.

Currently I’m implementing something very similar to this: https://web.archive.org/web/20170609122132/http://jam.sg:80/blog/efficient-partial-keyword-searches/ as a solution for bigger scaling and cross-collection searches.

I’m also wondering how tightly coupled you think your strategy is with your specific use case?

Good to see you guys always pushing the boundaries! =)