Elasticsearch and query.observe()

1

Hello,
I recently decided to use elasticSearch in my app. Which has changed a lot of things at the server and the client.
In order to display “projects” , I get an array of objects, which is no longer a mongo cursor. So I 'm not able to use query.observe () on my result like before.

Server, before elasticSearch :

Meteor.publish('projects', function(slider, search) {
    // console.log(slider + ' ' + search);
    var options = {
        //something
    };
    var regExp = buildRegExp(search);
    var selector = {
        //something
    };
    return Projects.find(selector, options);
});

Then I used query.observe () to display items on a map :

        Tracker.autorun(function() {
            var query = Projects.find();
            query.observe({
                added: function(document) {
                    //something
                },
                removed: function(oldDocument) {
                    //something
                }
            });
        });

Now, with elasticSearch, i create a SearchSource on the server (i’m using search-source extension) :

SearchSource.defineSource('projects', function(searchText, options) {
  Meteor._sleepForMs(1000);
  var words = searchText.trim().split(" ");
  var lastWord = words[words.length -1];
  var slider = options.slider;
  var query = {
    "bool": {
      "must": [
        {
          "bool": {
            "should": [
              {"match": {"name": {"query": searchText}}},
              {"prefix": {"name": lastWord}},
              {"match": {"architect": {"query": searchText}}},
              {"prefix": {"architect": lastWord}},
              {"fuzzy": { "name" : searchText }},
              {"fuzzy": { "architect" : searchText }}
            ]
          }
        }
      ],
      "should": [
        {"match_phrase_prefix": {"name": {"query": searchText, slop: 5}}},
        {"match_phrase_prefix": {"architect": {"query": searchText, slop: 5}}}
      ]
    }
  };

  var result =  EsClient.search({
    index: "meteor",
    type: "projects",
    body: {
      query: query
    }
  });

  var data = result.hits.hits.map(function(doc) {
      var source = _.clone(doc._source);
      source._score = doc._score;
      source._id = doc._id; // <- add this line
      return source;
  });

  // getting the metadata
  var metadata = {
    total: result.hits.total,
    took: result.took
  };

  // return both data and metadata
  return {
    data: data,
    metadata: metadata
  };
});

Then, now… how can I display my projects reactivly on a map ?
With search-source, i can get the projects on the client with :

var projectsArray = ProjectSearch.getData();

Which returns this kind of array :
http://img15.hostingpics.net/pics/625588objects.jpg

Is there a way to continue using query.observe () on this type of result ? How else do ?

2

Save current map data, compare it with that method results and apply just differences.
But than there is nothing automatic.

3

Ok, thanks.
I am looking for a early example.
ProjectSearch.getData(); give me an array which is updated when i use ProjectSearch.search(‘search’, options);
How to retrieve objects that have been deleted?
For now, I delete all markers on the map, and I make a new loop for re-display. But I would like only remove projects that are no longer in the array, and display the new ones.
When you talk about saving current data, you mean for example in session ?
Another idea might be to give the old array to the server like an option, when I use ProjectSearch.search(‘search’, options).
I dont’t now, maybe this is silly… Maybe someone can guide me to the right path ? :smile:

edit : this is how i get the projects in order to display a reactive list (and i try to have the same effect for makers on a map)

Template.searchResult.helpers({
  getProjects: function() {
    return PackageSearch.getData(});
  }
});
4

I was thinking saving it somewhere, as current state what is in the map - even normal array, dont need to be reactive.

Than compute difference - for example using the standard underscore
_.difference and _.intersection come to my mind - for adding/updating/removing items.
So you dont have to empty or replace whole array to map, but just maintain consistent state there.

5

Oh thank you, I didn’t know it was possible. You are talking about underscorejs functionalities ?
I’ll look at it !

6

Hello, I tried something that works, but I don’t think the code is fine…
If someone have a better idea…

I don’t like to use a second loop to remove projects that are no longer in the search. And this loop is inside the first, the difference is recalculated for all projects, that is not good. But it doesn’t work when I put it outside the first loop.

        markers = {}; //object that stores google markers (objects)
        projectsOnMap = []; //array that stores projects currently on the map
        diff = [];
        var projectId;
        var indexOnMap;

        Tracker.autorun(function() {
            //get projects : results of the current search
            var projectsSearch = ProjectSearch.getData();

            for (var i = 0; i < projectsSearch.length; i++) {
                //copy to another array that stores all the projects currently on the map
                projectsOnMap.push(projectsSearch[i]);
                //difference between the "search" array, and the "onMap" array
                diff = _.difference(projectsOnMap, projectsSearch);
                //remove projects that are no longer in the search
                for (var e = 0; e < diff.length; e++) {
                    projectId = diff[e]._id;
                    indexOnMap = projectsOnMap.indexOf(diff[e]);

                    if (markers[projectId]) {
                        markers[projectId].setMap(null);
                        google.maps.event.clearInstanceListeners(markers[projectId]);
                        projectsOnMap.splice(indexOnMap, 1);
                        delete markers[markers[projectId]];
                    };
                };

                //display markers on the map
                if (projectsSearch[i]._id && projectsSearch[i].lat && projectsSearch[i].lng) {
                    var marker = new google.maps.Marker({
                        position: {
                            lat: projectsSearch[i].lat,
                            lng: projectsSearch[i].lng
                        },
                        map: mapInstance,
                        icon: icon,
                        id: projectsSearch[i]._id
                    });
                }
                markers[projectsSearch[i]._id] = marker;
            };
        });