Best package for denormalization?

Denormalization is not very fun. With MongoDB it’s often necessary though.
I found this package, which is pretty nice! I thought it’d be more popular though, and was surprised to see it hasn’t been updated in two years. I even found an unfixed bug preventing the use of nested fields.

So is there some other package that I’ve missed, which everyone is using instead?
Or do you have some good patterns for denormalization to share maybe? :slightly_smiling_face:

1 Like

I’ve started working on my own denormalization package, which supports some more advanced features, like many-to-many relationships, inversed relationships and nested reference fields. The primary purpose is to be integrated into grapher. I’ll probably duplicate the “cache count” and “cache field” features that jeanfredrik:denormalize has too.

2 Likes

PeerDB, does this. Is very automated.

Hmm, maybe I wasted two whole days of work then…
The question is if PeerDB will work well with Grapher, which is what I’m using for everything right now. Both Grapher and PeerDB kind of seem to want to be their own framework. Also, I don’t know any coffeescript, but to use PeerDB, I’d have to define the Documents in coffeescript files, right?

It is basically a schema, but you can still use plain js in cs, and cs translates to js.

Anyway doesn’t they both do the same thing? PeerDB is suppose to support edits on relations outside peerdb.

Yeah I know, but it’s hard to know the exact equivalents between CS and JS (without experience anyway). A CS class is not exactly the same as a JS class for example, and I have no idea what @Meta means.

I belief they are, CS compiles directly to JS, so it can’t implement class deriviation. There are tools that let you see what means what, and repo has issue about translating this stuff by hand for read me with some success.

@mitar may want to mention a word denormalization somewhere in read me so that atmosphere has a chance at picking it up, as it is absent besides being a tag to repo itself.

I out of the gates ctrl+f for denormolization when found it, so it’s not there.

There are some examples here: https://github.com/peerlibrary/meteor-peerdb/issues/30

But I agree, at the time of writing, I was not really testing the API out to see how easy it is to use it from vanilla JavaScript side. I would have done things differently today, but this is where we are. :slight_smile:

While not a package as such, there’s some good information in the Meteor guide on handling denormalisation: https://guide.meteor.com/collections.html#denormalization

Just thought I’d mention I’ve made a lot of updates to my package, and I would say it’s stable and ready for use now :slight_smile:

New features:

  • cacheCount - cache the number of related documents, also supports a selector
  • cacheField - cache a merged/transformed version of a document’s own fields
  • migration system - both semi-manual and automatic options
  • “recursive caching” - by caching the cacheField of another collection, you can create “chains” of three or more collections
  • a more extensive readme
1 Like

Very nice! Bookmarked!

1 Like

Is there a way to use the denormalize package only in a specific context, for a tabular table for example?

I need denormalization in order for my tabular tables to search joined fields correctly, but I don’t need it globally for the collection throughout my app, which it seems the denormalize package does by default.

I was also looking into the dated Meteor Denormalized Views package for the same thing. It seems it’s designed exactly for creating a singular denormalized collection for tabular but I’m having trouble figuring out how to make the filter dynamic. It seems like it must be hard-coded (e.g. the “Donald” bit) at the time of defining the denormalized view.

Where I would need the filter to be dynamic as to not run the denormalized view on the entire collection. But only those documents that belong to the user and the context. Same issue.

I guess a method could be used to dynamically generate the denormalized view on the server, but then how is destruction handled?

@thebarty

Although I don’t have much experience with Mongo, it and GraphQL have caught my attention. Wouldn’t the entire single endpoint structure of GraphQL sort of address the de-normalizing issues this article discusses?