DDP Router goes Open Source

TL;DR;

DDP Router is now open source! Check it out at github.com/meteor/ddp-router.

Background

Nearly a year ago, I posted the Introduction of DDP Router. Back then, we started to look for projects willing to try it out in the wild, hopefully confirming that it makes the app perform significantly better.

It took us a few months to polish out the bugs (big shout out to @landland for very thorough testing!), but due to limited resources, the project was suspended. We’ve discussed open sourcing parts of it, but it was unclear which parts make sense to do so, and who should maintain it afterwards.

The recent discussions (What if Meteor Was Created in 2025? and Rethinking Meteor – managing subscription state) suggested that maybe instead of keeping the project on the hook, it could be open sourced and inspire the community. Or maybe even be continued solely by the interested people.

What is DDP Router?

It’s a performance-oriented project, sponsored by Meteor Software, and delivered by yours truly :heart: Here’s a brief description of what it is exactly:

Current state

The tests we made so far showed that:

  • Some applications just work (e.g., Atmosphere)
  • Some applications hit some of the edge cases that need to be implemented and/or fixed (i.e., they utilize some less-popular query operators and cursor options, e.g., skip: 0).
  • Some applications work almost perfectly except for one or two features, and we still don’t know why. It was hard to pinpoint those while testing closed source tool on closed source applications. Hopefully, we can solve these now!
  • There’s a list of limitations and known issues.

As for the performance improvement – it’s definitely there. However, it’s important to explain what was measured exactly:

  • The baseline is a single Meteor instance running the standard oplog tailing.
  • The compared scenario runs both the DDP Router and a single Meteor instance.
    • The aforementioned CPU and RAM gains are totals, not Meteor-specific. In other words, your Meteor instance does almost nothing, and the DDP Router does the heavy lifting.

The performance improvement was higher in publication-heavy applications; there was only a very slight difference in applications using only methods.

What’s included

Of course, to make the DDP Router happen, I had to reimplement a lot of Meteor’s core logic. All of it, implemented in Rust, is now available. A few things I think may be interesting for people wanting to dive deeper into Meteor’s “magic”:

  • DDPMessage struct for everyone who’d like to build their own DDP-related tools.
  • BSON → EJSON transformation.
  • MiniMongo’s Matcher, Projector, and Sorter. Yes, that’s basically the entire MiniMongo’s logic (including almost all of its tests verbatim).
  • Two-level Mergebox (and its Session). Note that it’s more than what Meteor does, as the DDP Router is capable of both publications and multiplexing publications originating from Meteor (i.e., it fallbacks to Meteor-based publications if the cursor is not supported).

What’s next

We’d love to hear what you think about it! Maybe you’d like to give it a try with your application, build something new on top of it, or at least get inspired – hopefully it’ll benefit the community :heart:

Also, I’ll give a DDP-oriented talk on the upcoming Meteor Impact 2025. Of course, there will be something about the DDP Router, too. See you there!

9 Likes

Is it weird that I’m more glad that you’re going to present a talk at Meteor Impact than the release of DDP router? :sweat_smile:

1 Like