[Gist] Atlas Search API's code for anyone interested :)

help
1

In case this is useful to anyone, I was able to get code working to configure an index for Atlas Search on server startup. There were a few snags, so I wanted to share.

My project is pretty much up to date as of today, running on Meteor 2.5.5.

Because of some annoying import vs. require business between node-fetch & digest-fetch (a plugin for node-fetch), you’ll want to install using the following command in order to avoid errors about import. (Thanks to @jaschaio on GitHub for the solution, which I’ve adjusted a bit for us Meteorites).

meteor npm i node-fetch@cjs node-fetchv3@npm:node-fetch@latest digest-fetch crypto-js --save

Or at least, this was what worked for me.

I believe I’ve genericized the following code completely. Please LMK if I’ve missed something or you’ve got feedback.

It gets a list of indexes, attempts to find one for the given collection, then either creates a new one (using the suggested default config here: Create an Atlas Search Index using the Atlas Search API

It’s unclear to me based on the docs whether Atlas will update the index if the config hasn’t changed. Obviously, rebuilding the entire index when it’s not needed could be very bad, so I’ll just say that this is code to get you started and you can repurpose it however you need to.

For convenience, here are the docs: Atlas Search API Documentation

import DigestFetch from 'digest-fetch'

function isNonEmptyString(x) {
	return (typeof x === 'string' || x instanceof String) && !!x.length
}

Meteor.startup(async () => {
	const ATLAS_SEARCH_CONFIG = JSON.parse(process.env?.ATLAS_SEARCH_CONFIG) // Or whatever
	if (!ATLAS_SEARCH_CONFIG) {
		console.log(`Skipping Atlas Search configuration - no settings found...`)
		return
	}

	const { PUBLIC_KEY, PRIVATE_KEY, GROUP_ID, CLUSTER_NAME, 
		DATABASE_NAME, COLLECTION_NAME } = ATLAS_SEARCH_CONFIG

	// NOTE: You'll want to modify this to match your collection
	const ATLAS_SEARCH_INDEX_DATA = {
		"collectionName": COLLECTION_NAME,
		"database": DATABASE_NAME,
		"mappings": {
			"dynamic": false,
			"fields": {
				"title": {
					"type": "string",
					"analyzer": "lucene.standard",
					"multi": {
						"keywordAnalyzer": {
							"type": "string",
							"analyzer": "lucene.keyword"
						}
					}
				},
				"genres": {
					"type": "string",
					"analyzer": "lucene.standard"
				},
				"plot": {
					"type": "string",
					"analyzer": "lucene.standard"
				}
			}
		},
		"name": "default"
	}

	try {
		const client = new DigestFetch(
			PUBLIC_KEY,
			PRIVATE_KEY,
			{},
		)

		const BASE_URL = `https://cloud.mongodb.com/api/atlas/v1.0/`

		const indexes = await (await client.fetch(
				`${BASE_URL}groups/${GROUP_ID}/clusters/${CLUSTER_NAME}/fts/indexes/${DATABASE_NAME}/${COLLECTION_NAME}`,
				{
					factory: () => ({
						method: 'get',
					})
				}
			)).json()

			console.log(`Atlas Search index list: ${JSON.stringify(indexes)}`)

			console.log(`Attempting to find an index for collection "${COLLECTION_NAME}"...`)
			const index = indexes.find(i => i?.collectionName == COLLECTION_NAME)
			if (undefined === index) {
				console.log(`Could not find an index for collection "${COLLECTION_NAME}". Attempting to create it now...`)
				const json = await (await client.fetch(
					`${BASE_URL}groups/${GROUP_ID}/clusters/${CLUSTER_NAME}/fts/indexes`, {
						factory: () => ({
							method: 'post',
							headers: { 'Content-Type': 'application/json' },
							body: JSON.stringify(ATLAS_SEARCH_INDEX_DATA),
						})
					})).json()
				console.log(`Atlas Search index creation response: ${JSON.stringify(json)}`)
			} else {
				console.log(`Found an index for collection "${COLLECTION_NAME}". Attempting to update it now...`)

				const indexId = index?.indexID
				if (!isNonEmptyString(indexId)) {
					console.error(`Found index for collection "${COLLECTION_NAME}" but indexID property was invalid: indexID=${indexId}`)
				} else {
					// NOTE: You can md5 the stringified index data and only patch if it's been updated; that's what I did in my version anyway...
					const patchResult = await (await client.fetch(
						`${BASE_URL}groups/${GROUP_ID}/clusters/${CLUSTER_NAME}/fts/indexes/${indexId}`, {
							factory: () => ({
								method: 'patch',
								headers: { 'Content-Type': 'application/json' },
								body: JSON.stringify(ATLAS_SEARCH_INDEX_DATA),
							})
						})).json()

					console.log(`Attempted to patch existing index "${indexId}" for collection "${COLLECTION_NAME}"; index processing status is "${patchResult.status}"; full result=${Utils.j2s(patchResult)}`)
				}
			}
	} catch (e) {
		console.error(`Atlas Search index configuration failed somewhere`, `Atlas Search Failure`, e.stack)
	}
})
5 Likes