Search Engine : MongoDB / ElasticSearch

help
1

Hello,
I would like to integrate a pretty good reactive search engine in my app. So i started to follow examples (instant-search…) and learn lessons with BulletProof Meteor (ElasticSearch).
ElasticSearch seems to be very interesting, but it makes the app more complicated…
I have some questions :

  1. Can MongoDB rank by relevance, scoring, beyond the keyword search : for example, if the query is “House black”, I want first results that count the two words in order, then the two words in the disorder, then a word … etc (anyway, i guess i need elasticSearch for this king of query),

  2. ElasticSearch is actually storing data. What is the best solution ? Store data in mongo + elastic or in mongo, or in elastic ? It is possible to synchronize elastic with mongo (without storing anything in elastic) ?

  3. By using elastic, should necessarily start the elastic instance in parallel, or is there a way to launch automatically the elastic instance with the app? (for the moment i downloaded elasticSearch somewhere on my computer, and I launched the instance before to launch the app).

  4. In production, i guess it will be necessary to install ElasticSearch on the server and, launch the instance. And i will need Ngnix for security ?.

Thank you very much !

1 Like
2

I have used Elasticsearch in parallel with MongoDB and it worked well. I’m not an expert at either, but hopefully this helps:

  1. Mongo appears to have a “textScore” relevance score. I’d bet it’s also possible to sort by arbitrary JS functions using MongoDB’s aggregation framework. However, Elasticsearch makes it really easy to tune the score in all sorts of ways.

  2. In my case, I reindex my data in Elasticsearch every night. I’m not sure what I would do if I needed to keep things updated in realtime. Elasticsearch used to have something called “rivers” for that, but they have been deprecated. If I needed realtime updates, I would look to see what people are doing instead of rivers.

  3. You will need to start the Elastic instance in parallel. This is less convenient than if it was built-in, but it’s not a big deal IMHO.

  4. You don’t necessarily need Nginx. Just make sure that Elasticsearch ports are not externally accessible. I’m not sure how to do this in general, but on AWS you just don’t open those ports to the outside world. Your meteor app should access Elasticsearch on the internal IP, not the public one.

Hope that helps!

3

you can try compose’s transporter to keep ES up2date to mongodb, it has oplog tailing too, but removing records is not working as long as I know
mongodb have basic per word scoring, but nothing like combination and 2 words going after each other.

4

Thanks !
@shilman
Ok, i guess i’ll keep datas in Mongo AND elasticSearch and launch the elastic instance in parallel.
When i’ll change the Mongo database I’ll try to interact with elastisearch with meteor HTTP requests. Maybe this is a way to update elasticSearch in real time. Anyway, I don’t need to update elastic in real time, i just need to display some projects in real time.