Is there a way to simultaneously create and connect to multiple Mongo databases?
For example, we want to store each user’s data at different databases:
var database = new MongoInternals.RemoteCollectionDriver("<mongo url>");
MyCollection = new Mongo.Collection("collection_name", { _driver: database });
However, it doesn’t change database dynamically for a separate user and connecting to other database requires Meteor restarting. Is there another way to create and connect to the user-related database at the same time without restarting Meteor?
Are you trying to have a separate database per user / account? Then you are in a though spot
Ive tried that with Meteor, but doing it dynamically and in just one process is difficult. An alternative would be to go for 1 docker container per customer to run meteor in and that container would have its own dedicated connection.
Meteor does this, because it tracks changes in Mongo and pushes them from the database connection up untill the client in realtime. Simply switching connection would stop that tracking for a previously connected user.
Another solution would be to create a collection per user, but this requires you to create your own low level publications and some nasty logics to initialize collections when users log in and out. I would not recommend this approach.
A third option is to avoid meteor’s serverside minimongo and go for your own mongo connection. This allows you to simply connect, query, push and disconnect from the database on demand. (You can still use pub sub fpr this though and it allows you to use the clientside connections)
Then again, is having a separate db per user really needed? Can’t you simply use query filters to fetch user specific data? This allows you to easily create some microservices and separate your databases based on domain instead of customer
1 Like
We used the third option, so Meteor connects to the separate database when user logged in:
var userId = this.userId;
var database = new MongoInternals.RemoteCollectionDriver('mongodb://localhost:27017/' + userId);
var collection = database.open('settings');
It works perfectly at the development stage, however, it doesn’t connect to the separate database at the production:
MongoError: failed to connect to server [localhost:27017] on first connect [MongoError: connect ECONNREFUSED 127.0.0.1:27017]
Can you send the whole publication? For now I can’t see anything weird. Is Mongo actually able to start on production? Is it running on the same server as the Meteor process?
1 Like
Yes, Mongo is running on the same server as Meteor process but they are located in separate docker containers.
P.S: We use MUP (Meteor-Up) for deploying the project.
It is likely not a Meteor thing then. I think that the database connection cannot be made. The mongodb://localhost link is referring to the internal IP of the docker container. You will have to figure out trough which IP and port your MongoDB Docker is accessible. Then use that IP address to connect to it.
1 Like
UPD: It works when we use the single database in our project. The problem appears when we try to use multiple databases. Meteor can connect to the default database but it can’t connect to the different database by using the code shown above.
1 Like
did you try the below? Its slightly different and how I did it in the past:
const database = new MongoInternals.RemoteCollectionDriver(`mongodb://localhost:27017/${userId}`);
const collection = new Mongo.Collection("settings", { _driver: database });
1 Like
Yes, we tried it before, however, it didn’t work even at the development stage. It was unable to change connection between databases dynamically. For example, there are three databases at MongoDB server:
db:meteor (default database) {
'collection': countries,
'collection': timezones,
'collection': users,
...
},
db:user1 (database for user1) {
'collection': settings,
'collection': contacts,
...
},
db:user2 (database for user2) {
'collection': settings,
'collection': contacts,
...
}
When user reconnects from database “user1” to the database “user2” (simply by logging out from “user1” and logging in as “user2”) Meteor throws an error like “There is already a collection named ‘settings’”.
That makes sense. The Mongo collections are defined globally with their names. In this case ‘settings’ can only exists once no matter how much connections.
There is a similar issue like this on this thread: Remote Database has the same collection name
I think you will need to find a way to make collection names globally unique by (for example adding the userId to its name. If you want to have databases without those unique identifiers in each collection, it requires you to hack into the Mongo driver.
Can you get away with collections that have a prepended userId? Like below:
cloudspider17m
did you try the below? Its slightly different and how I did it in the past:
const database = new MongoInternals.RemoteCollectionDriver(`mongodb://localhost:27017/${userId}`);
const collection = new Mongo.Collection(`${this.userId}_settings`, { _driver: database });
1 Like
We tried to use different names in the past. But there was another issue: Meteor couldn’t reconnect from one database to another. For example, we started Meteor and logged in as “user1”, Meteor successfully connected to the database “user1”. But when we sign out from “user1” and sign in as “user2” Meteor was unable to connect to the database “user2”. But after restarting Meteor it was able to connect to the database “user2”.
To be clear. Did you try:
const user1 = new MongoInternals.RemoteCollectionDriver(`mongodb://localhost:27017/user1`);
const user2 = new MongoInternals.RemoteCollectionDriver(`mongodb://localhost:27017/user2`);
const databases = {
jiorewjafasflk: {
settings: new Mongo.Collection(`jiorewjafasflk_settings`, { _driver: user1DB }),
timezones: new Mongo.Collection(`jiorewjafasflk_timezones`, { _driver: user1DB }),
contacts: new Mongo.Collection(`jiorewjafasflk_contacts`, { _driver: user1DB }),
},
iesrkwrgerwgwr: {
settings: new Mongo.Collection(`iesrkwrgerwgwr_settings`, { _driver: user2DB }),
timezones: new Mongo.Collection(`iesrkwrgerwgwr_timezones`, { _driver: user2DB }),
contacts: new Mongo.Collection(`iesrkwrgerwgwr_contacts`, { _driver: user2DB }),
}
};
And then in your publication it should do something like below:
Meteor.publish('settings', function() {
const db = databases[this.userId];
return db.settings.find({});
});
1 Like
It finally worked after we decided to install MongoDB server outside of docker. We didn’t change any code in the project.
Thank you very much for quick answers!
1 Like
No worries. Happy to help. Glad it worked out. Was stuck in a bus anyways… 
1 Like