Meteor with MongoDB Atlas - Index Creation fails


#1

We want to migrate our Database over to MongoDB atlas this month. We have some thousands of documents per collections, in some even millions - so indexes are a must.

Until now, we created these by collection._ensureIndex(...). However, Atlas doesn’t let me do this:

MongoError: user is not allowed to do action [createIndex] on [mydatabase.system.indexes]

but I didn’t tell the code to crate an index on system.indexes, but on e.g. Groups

The user has the rights

dbAdminAnyDatabase@admin
readWriteAnyDatabase@admin

the connection is done by

export MONGO_URL="mongodb://<myuser>:<mypass>@cluster0-shard-00-00-<accountid>.mongodb.net:27017,cluster0-shard-00-01-<accountid>.mongodb.net:27017,cluster0-shard-00-02-<accountid>.mongodb.net:27017/mydatabase?ssl=true&replicaSet=Cluster0-shard-0&authSource=admin&retryWrites=true"

then I tried to switch over to the Internal Mongo Driver:

{client} = MongoInternals.defaultRemoteCollectionDriver().mongo
client.db(databaseName).collection(collectionName).createIndex(...)

Same Error. Anyone any idea?


#2

I don’t have an exact answer to your question as I use a local mongo server. To me your error looks more like a permission problem at the mongo end.

I have a helper function that will check for and create an index on a collection if it does not exist. It does not use _ensureIndex so it may work around your problem. The function is as follows:

// Attempt to create a mongo Index if it is not already attached to the collection.
function createMongoIndex(collection, collectionName: string, indexName: string, indexKeys: object, indexOptions?: object) {
	if (Meteor.isServer) {
		Meteor.wrapAsync(collection.rawCollection().indexInformation((err, indexInformation) => {
			if (err) {
				if (err.message==="no collection") {
					consolewarn("[Mongo Index]", collectionName, "- Does not exist (yet)");
					err=undefined;
					return;
				} else {
					consoleerror("[Mongo Index]", collectionName, "- FAIL to get index information for", collectionName, err);
					throw new Meteor.Error(err);
				}
			}
			//consoledebug(indexInformation);
			if (!indexInformation[indexName]) {
				consolelog("[Mongo Index]", collectionName, "- Create a mongo index:", indexName);
				indexOptions = indexOptions || {};
				indexOptions['name'] = indexName;
				Meteor.wrapAsync(collection.rawCollection().createIndex(indexKeys, indexOptions, (err2, indexName2) => {
					if (err2) {
						consoleerror("[Mongo Index]", collectionName, "- FAIL", err2);
						throw new Meteor.Error(err);
					}
					consolelog("[Mongo Index]", collectionName, "- SUCCESS", indexName);
				}), collection.rawCollection());
			} else {
				consoledebug("[Mongo Index]", collectionName, "- Using existing index:", indexName);
			}
		}), collection.rawCollection());
	}
}

When I set up a collection in meteor, I add a line to ensure that the index exists. Whenever the meteor server starts, it ensures that it has the correct indexes on the collection.

// Ensure that additional indexes are present on the Mongo Collection
createMongoIndex( ColSearchIndex, collectionName, 'bySearchTerm', 		{ searchTerm: 1 }, {});
createMongoIndex( ColSearchIndex, collectionName, 'bySearchId', 		{ searchId: 1 }, {});
createMongoIndex( ColSearchIndex, collectionName, 'bySearchCollection', { searchCollection: 1 }, {});

This save me from having to remember to manually add the secondary indexes to a mongo collection. They always appear when the server starts.


#3

Agreed. @kellertobi: it’s probably worth reviewing the Atlas roles and permissions model.

That’s a lot of boilerplate for something which the MongoDB database engine does for you!

If you call db.collection.createIndex() multiple times concurrently with the same index specification, only the first operation will succeed. All other operations will have no effect.


#4

I know that you can call createIndex multiple times without any real side effects. I’ve programmed for a while and have learnt to only call functions when they are necessary otherwise it’ll probably bite you on the bum in the future. I also like my logs to reflect the actual situation. I like to know if it needs to create the index or not and only attempt it if required. I probably have a bit of OCD when it comes to log messages.

I only ever need to type the one function call in my collection file and also get detailed logging so I get the best of both worlds in my opinion.


#5

I just tried the index-generation with the same user in the mongoshell.

It is not a permissions problem

in the shell it works, in meteor it doesn’t