Meteor 3.5-beta: Performance & Change Streams
Hello everyone! We are excited to announce the first beta of Meteor 3.5. This release is heavily focused on introducing MongoDB Change Streams to the ecosystem.
For a deep dive into the “why” behind Change Streams, check out our detailed discussion here.
Experimenting with 3.5-beta
To start testing the new features, use the following commands:
Create a New App
# Create a new Meteor app using Meteor 3.5-beta
meteor create my-app --release 3.5-beta.3
Note: Improvements in this beta are not enabled by default for new apps. See the Setup section below for instructions.
Update Your App
# Update your existing Meteor app to version 3.5-beta
meteor update --release 3.5-beta.3
Change Streams Setup
To enable MongoDB optimizations, follow these steps:
- Database Requirement: Ensure you are using a Replica Set or a Sharded Cluster. See the MongoDB documentation for more details.
- Configuration: Add the following to your
settings.json:
{
"packages": {
"mongo": {
"reactivity": ["changeStreams", "oplog", "pooling"]
*// Meteor will try Change Streams first, then oplog,*
*// and if that doesn't work, fall back to pooling.*
}
}
}
Refer to the docs for further technical details.
Highlights
40% More Scalability with Change Streams
The traditional Oplog driver works by tailing all database changes and performing a “diff” process against client-side documents. As connections scale, this architecture hits a critical bottleneck: processing capacity.
In our tests, we discovered that while Oplog manages memory well under normal loads, it struggles to deliver data to clients fast enough during high traffic. This creates a backlog of pending queries and connections that eventually leads to Out of Memory (OOM) crashes.
Change Streams solve this by handling data on-demand (streaming). Instead of accumulating data locally, it processes it as it flows, allowing the system to maintain stability. Even under extreme stress, the system experiences simple timeouts rather than a fatal process crash.
Preliminary Benchmarks
The following data was captured using an artillery script in a dedicated machine (2 vcpu with 8 GB of RAM) running a standard Meteor app (otel) hosted in galaxy (using premium instance - 1 container of 8 vcpus with 8GB ) with high-frequency add/delete operations:
Oplog: Maxed out at 1200 VUs (10 VUs/s for 120s). Beyond this, the process suffered from persistent timeouts followed by an OOM crash.
Change Streams: Handled up to 1680 VUs (14 VUs/s for 120s). Beyond this, we observed only timeouts, but the process remained stable with no OOM.
Each VU ran 10 connections, one connection each 0.5s doing a insert
VU = Artillery Virtual User
Conclusion: Change Streams offer a 40% increase in connection capacity and significantly better resilience, effectively eliminating OOM errors caused by high data transfer volume.
(Image from montiAPM)
Reproducing the benchmark
> git clone git@github.com:meteor/performance.git
> cd otel
> git checkout otel
> docker-compose up -d mongo-replica
# edit your settings.json enabling changeStreams or Oplog
# then run the meteor app
> MONGO_OPLOG_URL="mongodb://localhost:27017,localhost:27018,localhost:27019/local?replicaSet=rs0" MONGO_URL="mongodb://localhost:27017,localhost:27018,localhost:27019/?replicaSet=rs0" METEOR_NO_DEPRECATION=true meteor run --settings ./settings.json --port 8080 --inspect
# in another shell, run the artillery
> npx artillery run tests/artillery/add-task.yml
Faster CI/CD
We have removed Travis CI from our GitHub pipelines.
- Before: ~40+ minutes waiting for PR checks.
- Now: ~10 minutes via GitHub Actions.
Big Thanks to Our Contributors
Community contributions are the backbone of Meteor 3.5.
- Special Mention: Huge thanks to @radekmie for the surgical review on the Change Streams implementation.
- Core Contributors: @italojs, @radekmie.
Your Feedback
Your feedback is crucial in adjusting this beta before the official launch. Community testing is key to ensuring Meteor 3.5 and the Change streams integration is stable and flexible.
You can use this thread to ask questions.
You can also check the existing Change Streams integration forum post for more details and to share your feedback. The post will also serve as a place to add insights on the integration and explain further options and ways it can be used.
