Foreign key relationships

hello -

i watched with great interest a training video series meteortips-screencasts and became fascinated with the concept of meteor, but quickly realized that mongoDB is not a drop-in replacement for mysql.

later i was introduced to “Meatier” which uses RethinkDB, and rethinkdb much more closely replicates the foreign key “parent-child” relationships we require. However, Meatier seems far too experimental to even consider.

so, if one was very interested in meteor, but absolutely requires the foreign-key relationships, are there any options? i did see where mongoDB is going to introduce foreign-keys, but rethinkdb seems like its a better option for us.

note too that with the fork and reunification of node.js and io.js, i hope something very similar happens with meteor and meatier. or am just being overly optimistic to consider that meteor and rethinkdb might work together as well as meteor and mongoDB do?

thank you very much.

You should check out @slava 's Meteor RethinkDB integration. The development is stalled at the moment because he’s waiting for Rethink DB to make it possible to correlate writes with feed notifications. Until they have that, RethinkDB unfortunately doesn’t allow optimistic UI (aka latency compensation). The good news is that they’ve promised it for 2.4!

In the meantime, you can look at the atmosphere package publish-composite, which essentially lets you do joins in publications. I used to be skeptical about using MongoDB having come from MySQL myself, but I found that Meteor actually provided all the tools that I needed to make things work, which together with the general awesomeness of Meteor’s other features convinced me to start using it for my projects.

1 Like

thank you helfer. i just now saw where mongo-3.2 is going to have joins $lookup – and even there i have to wonder how long it will take before meteor works with mongo-3.2

having spent a little time on rethinkDB, it does seem pretty powerful, especially using reql. and rethinkDB has the eqJoin, outerJoin, and innerJoin already.

so it sounds like you are saying go take a nice long vacation and wait patiently for rethinkdb-2.4? sounds great except i would wonder how much longer after that we can hope to see meteor–>rethinkdb, especially considering how long it took for meteor to use the latest node.

the “Meatier” founder has already discouraged me from using it because of my lack of experience.

whats a newbie who likes RethinkDB to do?

I’m curious if you have tried just using Mongo?

1 Like

hi sashko - no i have not used mongoDB. i have many child tables with two or more parent tables. with RethinkDB i can use the eqJoin and replicate the mysql foreign-key very closely.

There’s nothing that prevents you from using the equivalent of foreign keys in MongoDB, that’s exactly what I did. The only thing you can’t do is joins, but publish-composite helps with that.

Also, since MongoDB allows you to store arbitrary documents, denormalizing the data (i.e. storing the whole foreign object instead of just the key) is often also a reasonable choice. I’m not saying this would be right in your case, but it’s an option.

If you tell us a bit more about what you’re trying to build, I’m sure the smart people on this forum can give you advice for how to do it.

This is exactly the point. MongoDB not only allows you to store arbitrary documents, it is build to store arbitrary documents. When you’re coming from an relational database world (sql-databases), denormalizing data will feel wrong at first. It just doesn’t make sense, because you were always normalizing data as much as possible. Right?

Well, you can’t simply say that you must play with it a few days and you will start liking it. You must really start to think differently about data. Once your able to do that; you will love the flexibility that Mongo grants you.

So yes; go play with it. Try it out. But also; read and study as much as you can on the subject of document databases and denormalizing data.

smeijer and helfer both raise interesting points to consider. From my very limited perspective, RethinkDB seems to offer the best of both worlds (sql & mongo) without having to resort to any add-ons such as publish-composite.

I have situations where a table has three parents which may be better suited for RethinkDB.

It sounds as if RethinkDB-2.4 will eventually become part of meteor, but in the meantime, i did find this very interesting combination of RethinkDB, deepstream, and Vue.

who was it that once famously proclaimed “you can never be too rich, too thin, or overnormalize your data” ?

MongoDB 3.2 is already out, and while Meteor ships with Mongo 2.6 out of the box, there is nothing stopping you from using version 3.2. It works just fine. You just need to start your app with additional env variable:

MONGO_URL="mongodb://" meteor

Just remember to set the correct MongoDB port (27017 is the default one).

1 Like

M4v3R - thank you very much for your suggestion. are all the newest mongodb features immediately available to meteor?

While I’m not 100% sure ALL of them are available (because I didn’t check), the ones that I needed (some aggregation operators new to 3.2) worked beautifully. You just have to remember that all this goodness will only work on the server, where you have access to the full MongoDB server. Minimongo cache on the client doesn’t support many advanced Mongo features. In most cases that’s not a problem though, because you can always use a Meteor method to get your data that requires aggregation or other advanced stuff.

M4v3R - again, thank you very much. the issue (as always) is for a total newbie to see a meteor example of a child with two parents already working, rather than trying to do it themselves.

its always frustrating to spent excessive time getting a new technology to run with very new features only to eventually determine it cannot meet ones needs.

For plain parent-child joins I (and many other as you can see above) use publish-composite package. While it’s a 3rd party package, it’s quite simple really and does it’s job very well. Check out the examples author gives in the README to see if that fulfills your needs.

OK admittedly , i wish i would had seen before i launched into RethinkDB.

In mongo 3.2, there is a $lookup operator in aggregation

yes i saw that - the question is if the new $lookup option is available in meteor without using publish-composite, although publish-composite looks reasonably straightforward to install.

i am wondering now if i used the latest mongoDB with $lookup, then i would not need to use publish-composer ?

$lookup operator has two downsides:

  • it only works as a part of an aggregation pipeline, not in normal Mongo queries. Refer to MongoDB docs to learn more about aggregation pipeline.
  • it only work on the server, which means that to get the data you have to explicitly fetch it via While this is something you might want sometimes, the main advantage of Meteor (in my opinion) lies in its powerful data sync stack, where you write publications on the server, subscribe to them on the client, and you get real-time updates without explicitly requesting them.

So like I said, while in some cases it might be helpful to use Mongo aggregation methods (in areas of your app that do record counting, summing or similar, for example on pages that display Metrics/statistics or in cron jobs), most of the time you’ll want to use publications and subscriptions, and that’s where publish-composite enters, enabling you to publish related collections together.