Can I create server sided mongodb collections from the browser?

help
1

I would like to know if I can allow end users of my app to define a schema and then create corresponding collection on the server.

I would then like to run code that finds all collections and creates an interface based on each collection’s schema.

Example:
My interface has a form to create a new interface.
User creates collection named “cow” with fields: race, weight, age.
Then I have something generic that finds that new collections among the list of collections (chickens and cows), and generates an interface accordingly for each one. The interface will be a create read update delete (CRUD) thing, generated for each collection.

I created an interface for a collection i already made, I want something that lets users create collections.

This is what I would like to do. Is it possible? Am I thinking of this in the wrong way?

2

Bad idea in general: https://docs.mongodb.com/manual/reference/limits/ see number of namespaces. You will likely run out of the limits quite quickly as every collection and index is a namespace.

A better model would be to have a collection “tables” for example with a user id. The a collection with “rows” or “documents” for the data. Every row contains the id of the table.

3

Can totally do it. I have a product totally based on user-defined Collections by submitting forms of schema, a cloud-based WYSIWYG table creator, and also supporting Autoforms

4

How do you deal with that limit? Do you run multiple mongo databases?

5

Our backend assumes user’s login name as the namespace prefix. All their created tables will have the prefix. Then at least their table names won’t clash each other.

And we are dealing with that limit by naturally limiting the number of tables they could create, according to the users’ plans they subscribed our SAAS.

6

yes, can you be a good solution. What I would be worried about is that indexes also count, so let’s say:

24.000 namespaces available
Let’s say 1 index per collection: So 12.000 collections possible.
Let’s say 10 collections per user: 1.200 users maximum.

It would worry me to follow that strategy because it’s about the max. In reality I would suppose it to be much lower when you also need indexes for performance.

But you could increase the namespaces by setting a bigger size to the namespaces file: http://stackoverflow.com/a/15136030/1177905

For WiredTiger it does not seem to be relevant anymore which might open the door for this kind of solutions:

The WiredTiger storage engine is not subject to this limitation.

The most common issue seen is too much open files:
https://groups.google.com/forum/#!topic/mongodb-user/vHKZODyh9cE

That is because it opens multiple files, at least 2 it seems, per collection.

Sharding

https://docs.mongodb.com/manual/sharding/#sharded-cluster

Mongo sharding works on collection level. That means it can shard one collection over multiple servers by getting a shard key, for example userId. Which is a field in every document.

So no, even though the limit is removed now I don’t see a very good case for dynamically creating so many collections.

And since Meteor publications can query for the right data for this user I think it’s wiser to stay with one or a few collections. An alternative would be a database per user for the big ones.

7

A better model would be to have a collection “tables” for example with a user id. The a collection with “rows” or “documents” for the data. Every row contains the id of the table.

^ can you exemplify this please?

8

any chance to look at your product?

9

hei? any chance of getting an answer please?

10

What exact question(s) do you have?

A better model would be to have a collection “tables” for example with a user id. The a collection with “rows” or “documents” for the data. Every row contains the id of the table.

How you could setup come collections, just a very rough idea:

Tables = new Mongo.Collection('tables');
Rows = new Mongo.Collection('rows');

Tables.insert({
  userId: Meteor.userId(),
  name: 'cow', 
  fields: {
    race: 'String', 
    weight: 'Number'
    age: 'Number'
});

Then in your publication:

Meteor.publish('tablesForThisUser', function() {
  Tables.find({ userId: Meteor.userId() });
});

That will give all tables of only this user. Then when the user inserts data:

Rows.insert({
  userId: Meteor.userId(), 
  tableId: "id from table here",
  race: 'green',
  weight: 123,
  age: 5
});

Then in your publication:

Meteor.publish('tableRows', function() {
  return Tables.find({ userId: Meteor.userId(), tableId: "id from table here" });
});

That should give you a basic idea. For performance, indexing etc. some changes are required but for an MVP this will work.

11

I got it before then, what you mean is just putting a document inside a document. That probably has consequences for indexing and so.

Look at my other idea and tell which you think is better:

Since fields are not mandatory, I was thinking about just going for something like creating a collection for type_of_things and another collection for all_things

The type_of_things will describe what kind of things you can insert in all_things, and what type the fields are. From that I generate an interface for updating/creating.
And all_things has everything in it, but flat. And that generates the view/delete interface.

type_of_things.insert
type_of_thing:cow
fields { name: string, weight: int, age: int }

all.things.insert
type_of_thing:cow
name:'mimosa’
weight:123
age:5

What do you think of this approach? Im thinking it will be easier to generate views from this.

12

I think it is exactly the same as I wrote?

table = type_of_things
row = all_things

Only different is that I put the _id of the table in the row where you put a name (cow) in the all_things? If that’s your point I wouldn’t care, it will work both. I suppose that it’s easier to use an id but it is for no reason a requirement.

About the indexing: Rows._ensureIndex({ “name”: 1}); would do the trick. But you need to be a bit smart at which fields to index. It depends on which views you use, that’s quite easy to test.