It’s time to revisit our Roadmap (https://docs.meteor.com/roadmap.html) and shape the future direction of the framework. Your valuable insights and feedback have always been instrumental in shaping Meteor’s development, and we’re eager to hear your thoughts once again.
As you are aware, our team is working hard to release Meteor 3.0, removing Fibers from our core, and running Meteor.js on Node.js 18. It has been an enormous effort, bigger than anyone has imagined. But we are getting there, if you want to follow it closely, please check this topic and our Fibers Public Board.
To kickstart the discussion, please, take a look at our existing roadmap. I just made a quick update, adding items to Finished and separating the Meteor 3.0 release.
We would love your input on what should be added, removed, or modified. For this, consider that all the Meteor 3.0 items will be done.
We had an internal meeting, and I asked the team to choose the items they think we should do next, and these are the chosen ones:
New Meteor Guide;
Keep improving Typescript support;
Linux ARM Support;
Improve Meteor build time;
Tree-shaking;
Bring Redis-oplog to the core;
Better file upload package;
We appreciate your ongoing support and look forward to hearing your valuable feedback. Together, let’s make Meteor even more powerful, intuitive, and exciting for the years to come!
I will be working on Email high output queues and Push Notifications for Meteor 3 once released.
As for Redis-oplog I think … not only it should not make it into the core but it should be phased out altogether.
“Change streams allow applications to access real-time data changes without the complexity and risk of tailing the oplog.”
It doesn’t seem like a lot of a new Meteor at the surface, or the DX, therefore I think there is no need for a new Meteor Guide. There is possibly something I don’t know but I personally don’t think a new one is required unless the API and features change a lot. Relevant updates are indeed necessary.
PWA support. Meteor already works really well for PWAs. But perhaps it would be beneficial if the guide would have a section talking about what it takes to make a Meteor app work as a PWA. Would be good to have a general service worker example.
Vite. Many frameworks have moved to Vite and pretty much made it into a standard. Meteor’s Vue 3 integration uses Vite as well. Perhaps a more general move to Vite could be considered? Not really sure if that would be viable at all, but speed increases would likely be massive.
There is always a bad perception for pub/sub. New comers are using it without seeing the dark side of this beautiful feature. When the developed product is presented to the user, serious performance problems are experienced. In this case, it causes developers to move away from Meteor.js.
We both know the solution is easy. Use methods or use redis-oplog. Both of them are very easy to implement but if you are using Meteor.js for the first time. It’s always a magic / an obscurity. New developers may seem simpler to blame Meteor.js for this and switch to different framework.
Bringing Redis-oplog and Redis to the core as MongoDB could be a great.
Just to add: there is an item in the linked page about CapacitorJS, which is good (I believe a good alternative to Cordova). But the lowest hanging fruit is giving full support to PWA. We currently have a meteor app that is a web app, a PWA (desktop and mobile), an Android App (installable from Play Store), Windows App (installable from Windows Store), and an iOS app (installable from App Store) - all from the same PWA codebase.
Tree shaking - either finish the PR or move to Vite where I think you’d get tree shaking for free.
Support for using Minimongo synchronously in 3.0+ - I think some solution here is important. Without it, my understanding is that the UI will need to be rewritten because it requires using the async API on the client in 3.0+. It feels unnecessary to use the async API to fetch data from Minimongo when the data is already there and theoretically can be fetched syncronously. See Support `*Local` methods in Collections API · meteor/meteor · Discussion #12644 · GitHub and zodern’s comment here.
Improved Methods - I like having Minimongo as the source of truth for data on the client. Otherwise I have to devise another global store which is basically duplicating what Minimongo offers. Right now, if I fetch data via Methods, they aren’t merged into Minimongo. The idea here would be to do something akin to pub-sub-lite’s Enhanced Methods – cache Method calls (and their result data), automatically merge data into Minimongo, and emit mutation update messages from server to the caller of the Method (from). If pub / sub can scale via change streams then maybe this is not needed or at least becomes less important.
Minimongo indexes and support more mongo operators in Minimongo - This also ties into using Minimongo as the source of truth for data on the client. In the past, I’ve integrated with a 3rd party API and then dumped the results into Minimongo Local Collection. It works very well but I recall having to do some hacks because you can’t create indexes with Minimongo. One solution here could be to use NeDB nedb and minimongo · meteor/meteor · Discussion #11598 · GitHub
Official Jobs package - I think making wildhart’s Meteor Jobs package an officially supported Jobs package could be the way to go.
Official Push Notifications package - I think making paulischa’s package an officially supported Push Notifications package could be the way to go.
Bring the Meteor’s Collections API on par with the native MongoDB driver - There are some mongo methods that look helpful but currently require using rawCollection(), meaning they won’t be isomorphic.
As I’ve been working on Meteor pretty extensively recently, let me share what I’d like to see in the future:
Investigate MongoDB Change Streams. As @jam already said, I’ve done some experiments already, but it’s unclear whether it’s always better. I expect it to be more of an opt-in thing, as it has a completely different performance characteristic.
That said, I still think that cultofcoders:redis-oplog should become a core package, along with oplogtoredis for seamless integration with rawCollection and external writes.
Feature parity in Minimongo. That’s mostly about the methods that would truly benefit the frontend code both in terms of performance and usability – distinct and findAndModify are the best examples here.
Other engines in Minimongo. Right now it’s a completely in-memory one, based on plain JavaScript objects. I’m thinking that IndexedDB or similar solutions would be truly interesting for a multitude of reasons: simplification, performance, memory usage, offline capabilities.
Secondary indexes in Minimongo. That’s optional, as it’s rather unlikely to have that much data locally. Also, different engines (see above) could alleviate this problem.
Better support for TypeScript. We’ve managed to bring the @types/meteor to core, but there’s still a lot to do. I think going all-in for TypeScript would be a really smart move.
It’d also be nice to see something like zodern:relay or grubba-rpc in core. I’m not sure how the final API should look like, though.
Since I’m currently maintaining it, I can get started on that, together with @radekmie and oplogtoredis. I think that as @paulishca suggested the focus should be on change streams and updating redis-oplog to become a scaling library in general as per @guncebektas suggestion.
Maybe there should be some official generator for this.
Beside the other things that have been said together with offline focus I would also like to see meteor-desktop integrated to kind of finish the set. Would be really cool also to be able to create just desktop and offline apps with optional sync to serves be it meteor-desktop or PWA installation.
I’ll think more on this and try to produce a list for the roadmap later.
From the existing roadmap I think that all the way to next release the items are good as they are. The real discussion starts from candidate items and beyond. I have put my notes and suggestions (in no particular order) here and will update as new things come to mind.
Existing items
Update and fix Meteor Client Bundler or Improve DDP Client;
Both please. First updating the bundler and fixing any issues that come up. I think a discussion needs to be had how to proceed with Vite and similar tools in this area.
As for improving DDP client, @harry97started experimenting with Primus, so I think there are some interesting areas that can be discussed. But in the first step it is about doing little improvements where we can.
Improve Passwordless package;
No contest here.
Support building mobile apps using CapacitorJS;
No contest here.
Bring Redis-oplog to core;
Yes and merge with oplogtoredis. Probably the first step towards using change streams so that we can have even better scaling story.
MongoDB Change Streams support;
As per above.
Better file upload support via DDP;
I think we need to in general have a clear story here how file uploads should be best handled in Meteor. From basic FTP servers to AWS and specific services.
Beyond
Node.js v20 in Meteor v3.2 or as soon as possible and do our best to keep at the latest version.
Update dependencies in core. These are rarely touched, but I think they should be periodically revisited and updated to take advantage for the latest features and fixes.
Accounts
Accounts support for Webauthn aka Hardware keys 2FA Discussion
I think this should be priority for the accounts package.
OAuth 2.1 support
Need to do this sooner rather than later. Discussion
More relevant functionality as described on Enterprise Ready to be awesome out of the box.
Expand accounts functionality
Accounts packages are probably the most popular feature of Meteor. I think we should build on that and allow additional features to help building amazing user applications.
For example connected accounts and account switching, Apple OAuth into core, LDAP Discussion
MongoDB related
MongoDB feature parity
We have fallen behind MongoDB functionality. This needs to be rectified as soon as possible, so that people don’t need to use rawCollection and other workarounds.
Minimongo improvements
Minimongo is one of the great things of Meteor that many don’t even notice, but it needs a major upgrade when it comes to features and additional capabilties as @radekmie pointed out. I think there is a potential for collaboration with MongoDB here if we can make minimongo standalone and allow for example for data syncs between minimongo and MongoDB on server after offline period and so on.
Performance
This is kind of nebulous, but I would like to see if there is anything we can do to improve Google Lighthouse performance scores from the tree-shaking to bundler and other options. Not even close to my area of expertise so I leave this to @zodern and other experts.
Collaboration sync server over DDP
For real-time collaboration like Google Docs there needs to be extra architecture to do things like conflict resolution and other things. Yjs is one example and I think having something like that together with all the other real-time features that Meteor has would just complete the offering. Related to the next item.
Desktop, offline and PWA
I would really like to see Electron or similar integration with Meteor. Together with that also improved options on PWA and offline capabilities and caching. Maybe something along the lines of SyncedStore.
Testing
I think app and package testing need a serious look. We need better official support for things like Cypress and CI integration, not to mention stubs and guides.
Especially for local test runs it would be good to have a test DB and some nice function that would populate it with test data to be used for the tests and then it would erase it when the tests are done. Easy integration with existing test suites and tools.
Storybooks
Storybooks came out of Meteor. Right now to integrate them with Meteor is painful. At the very least we should have an extension for Storybooks to provide stubs or something to make it run. I think there is a good potential here for an extension that would allow you to set different values that could be switched around in the interface like different users and so on.
Localhost SSL
SSL is pretty much given today. So I think it is time to have it by default on localhost as well.
Browser policy packages
The browser policy packages haven’t had an update in a while to a point that other solutions started to get recommended. Personally I would like to take a look on them and give them a refresh and see if there is anything specific that Meteor can specifically offer over having a competing NPM package. If we can’t offer anything extra then we should deprecate them.
Other packages
logging - let’s make more use out of this package
callback-hook - should become the standards for hook across Meteor
If it fits in fairly easily with the rest of Meteor, Vite (does this include Improving Meteor Build Time/Tree-shaking?)
Is it known yet how many Meteor apps are using SQL? If so it might be helpful to have Accounts support for SQL in addition to Mongo, especially now that mySQL/postGresQL have JSON field support.
I think Mongodb change stream works with oplog by itself. It’s builtin feature. Meteor also works with oplog, it’s like doing the same job.
The most advantage when using change stream api is when you scale your Meteor Apps horizontally by adding multiple Meteor instances. The oplog scanning job handled by Mongodb only, not on every Meteor instance.
I don’t want to get into details here (we can do that in the GitHub discussion), but it’s not that obvious.
In short, Oplog generates one event for each change, whereas Change Streams one for each change for every stream. That means, Meteor would still have to go through all messages anyway (and hit the same issues as with Oplog) or have specialized streams per-publication.
The problem with the latter is that if one change impacts multiple publications (i.e., multiple users subscribe to different documents but there are some that many of them subscribe to; e.g., my friends’ profiles), then instead of processing this change once, we process it multiple times. Sure, it can be mitigated by reusing subscriptions as much as possible (I mentioned it here: 🚀 Meteor Scaling/Performance Best Practices - #3 by radekmie), but for some cases it can lead to a significantly lower performance.
I also vote for improving the Oplog implementation!
We are running into some performance issues with the current Oplog implementation at the moment (click here) and are looking for solutions.
We tried redis-oplog in the past but ran into some weird issues with other libraries also using redis (in our case: “bull-queue”) and had to remove it again because of that, because the bull-queue is a must have for us.
Also, since we have a “split setup” for our biggest database changes (2 separate “non meteor” plain node apps called “workers” compared to 4 meteor “main app” docker containers on a separate server) redis-oplog was also a “no go” for us because there was no really nice way to also get oplog updates “from the outside” (our workers) into redis.
We have no experience with oplogtoredis but: the more libraries and “intermediaries” you add to a “chain of things”, which all have to work in harmony and should have a good performance, the worse usually performance will become. so yeah …
At the moment we are really not sure of what is our way forward … also thinking of removing Oplog completely again …
