DDP Router goes Open Source

TL;DR;

DDP Router is now open source! Check it out at github.com/meteor/ddp-router.

Background

Nearly a year ago, I posted the Introduction of DDP Router. Back then, we started to look for projects willing to try it out in the wild, hopefully confirming that it makes the app perform significantly better.

It took us a few months to polish out the bugs (big shout out to @landland for very thorough testing!), but due to limited resources, the project was suspended. We’ve discussed open sourcing parts of it, but it was unclear which parts make sense to do so, and who should maintain it afterwards.

The recent discussions (What if Meteor Was Created in 2025? and Rethinking Meteor – managing subscription state) suggested that maybe instead of keeping the project on the hook, it could be open sourced and inspire the community. Or maybe even be continued solely by the interested people.

What is DDP Router?

It’s a performance-oriented project, sponsored by Meteor Software, and delivered by yours truly :heart: Here’s a brief description of what it is exactly:

Current state

The tests we made so far showed that:

  • Some applications just work (e.g., Atmosphere)
  • Some applications hit some of the edge cases that need to be implemented and/or fixed (i.e., they utilize some less-popular query operators and cursor options, e.g., skip: 0).
  • Some applications work almost perfectly except for one or two features, and we still don’t know why. It was hard to pinpoint those while testing closed source tool on closed source applications. Hopefully, we can solve these now!
  • There’s a list of limitations and known issues.

As for the performance improvement – it’s definitely there. However, it’s important to explain what was measured exactly:

  • The baseline is a single Meteor instance running the standard oplog tailing.
  • The compared scenario runs both the DDP Router and a single Meteor instance.
    • The aforementioned CPU and RAM gains are totals, not Meteor-specific. In other words, your Meteor instance does almost nothing, and the DDP Router does the heavy lifting.

The performance improvement was higher in publication-heavy applications; there was only a very slight difference in applications using only methods.

What’s included

Of course, to make the DDP Router happen, I had to reimplement a lot of Meteor’s core logic. All of it, implemented in Rust, is now available. A few things I think may be interesting for people wanting to dive deeper into Meteor’s “magic”:

  • DDPMessage struct for everyone who’d like to build their own DDP-related tools.
  • BSON → EJSON transformation.
  • MiniMongo’s Matcher, Projector, and Sorter. Yes, that’s basically the entire MiniMongo’s logic (including almost all of its tests verbatim).
  • Two-level Mergebox (and its Session). Note that it’s more than what Meteor does, as the DDP Router is capable of both publications and multiplexing publications originating from Meteor (i.e., it fallbacks to Meteor-based publications if the cursor is not supported).

What’s next

We’d love to hear what you think about it! Maybe you’d like to give it a try with your application, build something new on top of it, or at least get inspired – hopefully it’ll benefit the community :heart:

Also, I’ll give a DDP-oriented talk on the upcoming Meteor Impact 2025. Of course, there will be something about the DDP Router, too. See you there!

12 Likes

Is it weird that I’m more glad that you’re going to present a talk at Meteor Impact than the release of DDP router? :sweat_smile:

2 Likes

Hi Harry97,

Not at all! It’s exciting to see new developments in the community, and a talk at Meteor Impact is a great opportunity to share insights and connect with others. The DDP Router sounds like a fantastic project, but hearing about it in a presentation could help clarify its impact and use cases. Looking forward to both the release and the talk!

1 Like

I love to know that a project like this exists!
For sure, bringing the meteor core to a compiled and low-level language will make MeteorJS stand out among all the real-time implementations available in the market.
I will dedicate some time to understand it better and contribute on it when possible.

Very interesting congrats and thank you! I will definitely check this some time in the future!

One question, I see we don’t get socks.js so it means there are no fallbacks if we use ddp router? Or are there provided by rust library?

Meteor doesn’t use SockJS for anything else than the compatibility layer, so it was easier to require a standard WebSocket communication than handling both that and the SockJS-wrapped messages.

We’re not using DDP Router (yet?) but we do use DISABLE_SOCKJS=true for a slight performance improvement. I also described it in here.

1 Like

I see. There can be some strict firewall rules in corporate networks where websockets are forbidden but I guess nowadays it’s super rare as websockets are so widespread so maybe I will do the same.

So you’re using Meteor with SockJS’s XHR polling only?

I mean that’s how fallback works, if for some reason (usually firewall blocks) websockets doesn’t work then socksjs use polling in that case. I have never tested it though and I really think this should be super rare nowadays so getting rid of socksjs ir probably safe.

Maybe one more question - did you try to stress test and compare number of websockets connections? I assume there is should be massive improvement there as it rust websockets is in the same league as uws from my quick check.
And for performance tests maybe it would be interesting to test how many clients/subscriptions/ddp messages per second both setups can handle at 100% cpu. I think there should be a huge difference (way more than 20%).
Anyway maybe it’s opportunity for me to learn rust :smiley:

Unfortunately I didn’t. If anyone would like to do it, I’ll gladly help, but I’m out of capacity to do it anytime soon.

1 Like

I have one more question - does it support subscription strategies? E.g. NO_MERGE_NO_HISTORY?

It does not, but it’s rather easy to do – all of the logic revolves around the Mergebox and implementing the necessary no-op alternatives (e.g., a dummy version of MergeboxDocument).

OK I see, then for now I won’t be able to test it but maybe later I will try to learn some basics of rust and check it out in code.