Anyone had a play with MongoDB Atlas Vector Search?

help
1

See How to Model Your Documents for Vector Search | MongoDB

wondering if anyone has got this working with a Meteor project … ie to createIndex and a query with a vector work. I assuming this would not work on your local MongoDB in dev environment ie you will have to be connected to a MongoDB Atlas database?

I do see https://www.mongodb.com/docs/atlas/atlas-vector-search/create-index/

2

This is now in public preview for development and testing. I haven’t personally used it yet but I’m interested with Atlas search to replace our open search implementation.

2 Likes
3

Thanks. Will see if I can get this working.

Update - I have it running locally, will have a play later… but seems to be simple. Not sure yet of the traditionally createIndex etc works. I did i through Compass.

Another update -

Soi thanks @rjdavid I have a local Atlas and using the connection string via MONGO_URL with my local Meteor. Thanks again for Introducing a Local Experience for Atlas, Atlas Search, and Atlas Vector Search with the Atlas CLI | MongoDB Blog

AS a above, I created the index via Compass… not sure if it will with createIndex yet.

I have embeddings working (using OPenAI for this right now). I am then searching via Compass and seems to be working… depending on my query (using openAI again) it seems to working… atleast with a small set of test docs.

Now I want to see if I can run the query via an aggregate via Meteor with the standard collections/aggregation. I’ll see how that goes.

Update: seems to be working fine with -

await SomeCollection.rawCollection()
        .aggregate(pipeline)
        .toArray();

So I think safe to say initial proof of tech seems to be working ok.

Summary:

  1. Created an Vector index via Compass

    Screenshot 2024-06-13 at 9.27.16 PM
    Screenshot 2024-06-13 at 9.27.16 PM1172×271 21.7 KB

  2. When I save a doc. I call OpenAI embedding API to get an embedding for some content on a doc into the embedding field sortValEmbedding.

  3. When a user enters a query call OpenAI to get the queryVector.

  4. To retrieve docs using vector search, I use an aggregate pipeline… simple test was something like this

const pipeline = [
        {
          $vectorSearch: {
            queryVector,
            path: 'sortValEmbedding',
            numCandidates: 100,
            index: 'microAppEmbeddings',
            limit: 5,
          },
        },
      ];

      const queryResult = await MicroAppData.rawCollection()
        .aggregate(pipeline)
        .toArray();

That’s probably it for a barebones example.

1 Like
4

I’m using Atlas Search in a new project. It was a good decision saving time and money. My case is not complex, but anyway the feature is very simple to configure.

2 Likes
5

This is the most comprehensive Atlas Search demo I watched in the past. It comes with a repo for the app presented and I think it contains all one needs to know https://www.youtube.com/watch?v=sopfYTUYJxk

1 Like
6

It’s really convenient. Having to synchronise the main DB and a separate Vector DB really sucks.

2 Likes
7

BTW, any hacks to get something like this working ?

SomeCollection.rawCollection().createSearchIndex or SomeCollection.createSearchIndex working?

8

rawCollection() depends on the mongodb node driver used

9

I’ll just paste this here. It contains a very nice explanation of vectors (with Astra DB - another NoSQL DB in the market) https://www.youtube.com/watch?v=mmrTLAA4mBg