Full text search status for MongoDB 2.6

core
1

I’m making a big Q&A website on Meteor and I really need a fast and comprehensive search capability there. So far I’ve found a few options for searching:

  • Using RegEx
  • Using Spomet
  • Using MongoDB 2.4 executeCommand
  • Using MongoDB 2.6 $text

RegEx is an option I currently use but it has limitations.
I don’t like Spomet cause it has examples on CoffieScript which I don’t know. Also I’m not sure how it works and how well it supports non-english texts.
I don’t like using MongoDB 2.4 version of full text search cause it introduces a lot of strange code in my project and I’m not sure if it supports reactivity.

I’ve been trying to upgrade my MongoDB to 2.6 and create text index there. It had worked in one project I made before. All I needed is to create text index via mongo console and then in Meteor.publish do something like this:

Meteor.publish('search-records', function(term) {
    check(term, String);
    return Records.find({ $text: { $search: term, $language: 'ru' } });
}

It worked like a charm but I’m not sure if the same search query will work on client using minimongo. Will it work?

In that app it didn’t matter cause it was a simple app with only one publication. So I’d just used Records.find({}) on the client and it was good enough.

But in my new app I can have a multiple publications of the same collections available on the client at time so I need to use queries on client to get required subset of data.

Another question is how do I sort results by score? MongoDB 2.6 supports that using score projection. Here’s an example from their docs:

db.articles.find(
   { $text: { $search: "cake" } },
   { score: { $meta: "textScore" } }
).sort( { score: { $meta: "textScore" } } )

Does Meteor support that in any way? Is there any workaround?

2 Likes
2

read this https://blog.compose.io/full-text-search-with-mongodb-and-node-js/ it will help

2 Likes
3

We like to use Elastic Search or Solr at Findwise, I’ve used @arunoda’s https://github.com/meteorhacks/search-source, it fulfills all our needs!

Cheers,
// Tim

1 Like
4

Here’s a more detailed post about search-source: https://meteorhacks.com/implementing-an-instant-search-solution-with-meteor.html

1 Like
5

Thanks for that link. I’ve tried that example in mongo console and it worked. But I can’t fully use it in Meteor for some reason. Here’s the code:

Meteor.publish('search-records', function(term) {
    check(term, String);
    return Records.find({
            $text: { $search: term, $language: 'ru' }
        }, {
            fields: { score: { $meta: 'textScore' } },
            sort: { score: { $meta: 'textScore' } }
        });
 }

Using that code I’m getting an error Exception in queued task: MongoError: Can't canonicalize query: BadValue must have $meta projection for all $meta sort keys

Without sort modifier it works just fine and even sends score field to the client which is great. But I need sorting ability on the server or else searching relevant records with pagination won’t work.

Does anybody know why Meteor’s implementation with sort modifier containing $meta doesn’t work while the same query with .sort({ score: { $meta: 'textScore' }}) in mongo console works just fine? What’s so special in Meteor MongoDB driver’s implementation preventing that from working?

D.

2 Likes
6

Hello,
Thanks for your interesting messages.
I am trying to do the same but also without success.
When I run int the meteor mongo shell:
db.docs.find( { $text: { $search: "something" , $language: "en" } }, { score: { $meta: "textScore" } } ).sort( { score: { $meta: "textScore" } } );
It works perfectly.
But no way to make it run from within my meteor application.
Do you know how to do it ?
Thanks a lot.
Thomas

7

I"m looking into doing something similar.

Did you get search + sort to work on the Server?

I haven’t tried this yet, but my first thought was, why not just sort on the Client instead; is this an option in your case?

8

Also, we have to put a ‘text index’ on the column we’re searching on in Mongo, can we do this in Meteor? Will you post that example?

9

On server you can use .rawCollection() to have access to raw Mongo collection.

10

I was able to get this working with the following setup:

  1. Within Meteor,

    var search_results = collection_name.find(
    { $text: { $search: search_criteria } },
    { score: { $meta: ‘textScore’ } }
    );

  2. MongoDB shell,

    db.collection_name.ensureIndex({filed_name: “text”})

– Note, the call to create the text index is not within Meteor.

– Also, note that I sort on the Client for now.

Can I just write a collection_name.ensureIndex({filed_name: “text”}) on Meteor startup or something to add the necessary "tex"t index?

Also, right now for example, the “text” index in the collection I’m working from is called [Display_Name]. This [field] contains both ‘FirstName + ’ ’ + LastName’.

While performing this search, I only get a “score” : 0.75, and it only matches if I input in the search box either the ‘FirstName’ or ‘LastName’ of a user.

Can I change the matching percentage or sensitivity so I get a match on partials?

[UPDATE]
After reading the docs it seems you can only match on a whole word, multiple words, or even not on the presence of words, within a [field] – but not parts of a word.

For example, if the target [field] contains the name, ‘don jon’, if I $search ‘don’, that document will be found (probably with a score of 0.75). If I $search ‘jon’ the same thing will happen. but if I $search ‘do’ or $search ‘jo’ nothing will $match.

12

I was able to sort on score in Meteor like so:

var search_results = collection_name.find(
        { active: true, $text: { $search: search_criteria } }, 
        { score: { $meta: 'textScore' } } ,
        { sort: { display_name: 1, score: { $meta: 'textScore' } } }
      );
13

After getting the $search working, i’m looking at $regex, what are its limitations?

14

@aadams Are you sure that is sorting properly with the score for you? I am trying to do the exact same thing in Meteor but it is still ignoring the score while sorting.

In fact I can’t even see the score in the returned fields.

UPDATE:

I got it working like this:

collection_name.find(
        { active: true, $text: { $search: search_criteria } }, 
	{
	        fields: {title: 1, textScore: { $meta: "textScore" } }, 
		sort: { textScore: { $meta: "textScore" } } 
	}
      );
16

Does your example work, adding another sort parameter to the mix, like display_name: 1 in my case?

17

You can also ensure the index from the Meteor app with:

if (Meteor.isServer) {
  Meteor.startup(function () {
   myCollection._ensureIndex(
      {'myField': 'text'},
      {background: 1}
    );
  });
}
1 Like
19

This is what I was looking for, thanks!

This is what I was looking for, thanks!

20

Are there any advantages of using $search over $regex or vis-versa?

21

I’ve been using this package and it’s done great for plain searches. I’m sure it could be extended for full text searches.

https://atmospherejs.com/matteodem/easy-search

Alternatively, I think in order to use full text search you basically can’t use the version of mongo that’s packaged with meteor (this may have changed) but you can create a new mongo database, connect to it, and enable full-text search.

1 Like
22

It works with the Meteor 1.1.0.2 installed version of Mongo 2.6.7, and text index, and the last query I posted above.

It was straight forward, with reactive search input, etc.

23

The $text operator uses the text index. The $regex don’t use the text index.
Also the query syntax is different. The $text query is like when you search something in Google.

See MongoDB documentation for details:

1 Like