Hey all,
I’m working on a project that requires a large set of data to be searchable/filterable by various fields. Obviously, Meteor works quite well for filtering the data as it comes out of the Collection, however reporting on that data is a little more interesting…
What I’m trying to replicate is Searchkit, albeit without the reliance on ElasticSearch. Is this where I’m going wrong?
What I’m struggling with is the counts of the available results per field in the filters…
For example, I’m searching for widgets, and 12 are green, 3 are blue and 5 are red. How would I display the available number of widgets that match that filter field next to the checkbox? (See Searchkit above if that makes no sense…)
The only ways I can think of would be really pub/sub inefficient and be a massive drain on CPU.
Any ideas?
If you don’t want to use a search engine like ElasticSearch or Solr (that has built in facet handling capabilities), then maybe take a look at using Mongo aggregation. You could have your publication return matching widget counts, grouped by their color, then use these results to build your checkbox with counts list. There are a few great packages available to help you integrate Mongo aggregation into Meteor, like meteorhacks:aggregate (or you can also use [Mongo.Collection’s rawCollection]
(http://docs.meteor.com/#/full/Mongo-Collection-rawCollection) to build out your own aggregation queries).
Excellent, thanks for your help! I’ll take a look at aggregations and see what I can come up with.
Given that something like ElasticSearch or Solr seems to be the “proper” way of doing this, and my use-case is far less trivial than my widget example; would you say adding ES (or other) to my setup to be worthwhile? As something very unfamiliar with it, what are the headline pros/cons of ES in conjunction with Meteor/Mongo?
I haven’t worked with ElasticSearch directly, but I have worked with Apache’s Solr on a handful of projects (both are similar since they’re based on Apache Lucene). Search engines like ES and Solr are really great at handling faceted navigation; they (usually) give you functionality for this right out of the box and make it easy to refine/unrefine your search by selected facets. That being said introducing a search engine to your Meteor app will definitely add some overheard, since you’re adding a new piece to your existing technology pie, and you’ll have to manage it. I’ve heard ElasticSearch is easier to manage than Solr, but you’ll still be sinking time into getting your search environment setup/configured/running smoothly.
If you’re interested, I put together a Meteor + React + Solr demo app a little while ago. It uses the meteorhacks:search-source package to make searching nice and reactive, and includes a few examples of handling faceted searching (with single and multi-level facets). It uses Solr, but swapping it out and using ES instead should be pretty straightforward. It doesn’t show how to work with mongo though since it was put together as a small POC for indexing external data sources only.
Thank you, you’ve been really helpful! I taken a proper look at Elastic Search, had a play, and think it would be the best solution for the project; despite the additional requirement of clusters, etc…
Cheers!
1 Like