I like the Meteor packaging system and it has some unique selling points, not sure why they need to be mutually exclusive.
Voting for:
- Passkeys
- Improve Meteor build time;
- Tree-shaking;
- If it fits in fairly easily with the rest of Meteor, Vite (does this include Improving Meteor Build Time/Tree-shaking?)
Is it known yet how many Meteor apps are using SQL? If so it might be helpful to have Accounts support for SQL in addition to Mongo, especially now that mySQL/postGresQL have JSON field support.
1 Like
Just for background information to anyone interested, this is the current vite:bundler package by Akryuym that combines Meteor with Vite: meteor-vite/packages/vite-bundler at main · Akryum/meteor-vite · GitHub
As other topics have links, thought I’d link it here as well.
1 Like
I think Mongodb change stream works with oplog by itself. It’s builtin feature. Meteor also works with oplog, it’s like doing the same job.
The most advantage when using change stream api is when you scale your Meteor Apps horizontally by adding multiple Meteor instances. The oplog scanning job handled by Mongodb only, not on every Meteor instance.
1 Like
I don’t want to get into details here (we can do that in the GitHub discussion), but it’s not that obvious.
In short, Oplog generates one event for each change, whereas Change Streams one for each change for every stream. That means, Meteor would still have to go through all messages anyway (and hit the same issues as with Oplog) or have specialized streams per-publication.
The problem with the latter is that if one change impacts multiple publications (i.e., multiple users subscribe to different documents but there are some that many of them subscribe to; e.g., my friends’ profiles), then instead of processing this change once, we process it multiple times. Sure, it can be mitigated by reusing subscriptions as much as possible (I mentioned it here: 🚀 Meteor Scaling/Performance Best Practices - #3 by radekmie), but for some cases it can lead to a significantly lower performance.
2 Likes
I also vote for improving the Oplog implementation!
We are running into some performance issues with the current Oplog implementation at the moment (click here) and are looking for solutions.
We tried redis-oplog in the past but ran into some weird issues with other libraries also using redis (in our case: “bull-queue”) and had to remove it again because of that, because the bull-queue is a must have for us.
Also, since we have a “split setup” for our biggest database changes (2 separate “non meteor” plain node apps called “workers” compared to 4 meteor “main app” docker containers on a separate server) redis-oplog was also a “no go” for us because there was no really nice way to also get oplog updates “from the outside” (our workers) into redis.
We have no experience with oplogtoredis but: the more libraries and “intermediaries” you add to a “chain of things”, which all have to work in harmony and should have a good performance, the worse usually performance will become. so yeah …
At the moment we are really not sure of what is our way forward … also thinking of removing Oplog completely again …
We are using both. In our case, redis-oplog has its redis DB that is different from the bull-queue redis DB. Separate your redis DBs to prevent any collisions. Or if you are forced to use only one redis DB, use namespaces; i.e. Bull use a prefix config to minimize collisions with other apps using the same redis DB
1 Like
We’re using oplogtoredis as well as the Redis for caching ourselves and it “just works”. Actually, we use the same Redis instance for multiple Meteor applications (almost 10), and all of them have oplogtoredis set up. As @rjdavid said – it’s all about using databases (you can set it up in the connection string).
As for the additional setup – it’s a Docker container we launched some time ago (two, three years back?) and it just works.
2 Likes
These are some items I would be interested in that haven’t already been listed:
Compatibility tables / support docs
It would be nice to have someplace we can easily check:
- which versions of the MongoDB server a release of Meteor supports
- browser support
- which ECMAScript version is fully supported in all arch’s. Currently, it’s not always clear if a feature we can use in modern browsers would also be supported in legacy browsers.
- compatibility for build plugins and core packages commonly used by build plugins. Build plugins are are unique in that they are built when published and bundle a copy of their deps, so which Meteor version you publish a build plugin with affects which older Meteor versions it can run in.
- which Node and npm version are used by each Meteor release
More thorough docs
It’s improved the last couple years, but there are still many things that are undocumented. It would be nice if someone would go through the change log at least as far back as Meteor 1.3, and the issues from the old docs repo including those that were closed for being old, and add anything missing. Another thing that would be nice is going through the docs, migration guides, and tutorials and improving the language. There are many sections with poor grammar or that explain things in a more complicated and unclear way than necessary.
Security Updates
Currently, Meteor only consistently provides security updates for the latest release. This makes it difficult for companies that are a version or two behind since they have to rush to update to the latest release before they were planning to, and sometimes run into undocumented breaking changes or bugs that make it difficult or impossible to update. I know companies that migrated away from Meteor because it was too difficult for them to stay on a Meteor release that had a node version with the latest security patches.
Also, sometimes that are delays of up to several weeks before there is a new Meteor release after there is a new node version with a fix for a security issue.
It would be nice if Meteor in some form committed to providing security updates in a way that addresses these issues.
npm installer improvements
When I created the npm installer, I tried to make it work with as wide range of node version as I reasonably could to make it easier to install Meteor. However, now it only works with a limited range of node versions, none of which are still supported by Node which makes installing Meteor more difficult. There are also a few issues open documenting long standing issues with the installer.
Windows for a while now comes with a native tar, so it would be good to use that instead of 7zip when available. This would make the installer smaller, and Windows Defender has some exceptions for the native tar which should make Meteor faster to install.
Remove ECMAScript.compileForShell
This could maybe be done by moving the function to the babel-compiler package, or moving the code to the one place it is used. This function causes every build plugin and app that uses the ecmascript package to have a full copy of the babel-compiler package, in addition to every Meteor release. A few years ago, I found that half the disk usage (I think half was ~5gb) in ~/.meteor was just copies of babel-compiler, and it was also a large reason installing Meteor was so slow on Windows.
Isobuild
Linker rewrite
This is probably one of the next things I will work on. The linker in Meteor is far behind the state of the art in terms of performance. Linkers in js bundlers have always been slow, and Meteor is missing many of the improvements other bundlers have implemented the past few years. It might even be possible to remove the disk cache, which causes its own performance issues.
Fully enable in-place builds on Windows
On Windows it is still disabled for client only builds, which for large apps can cause client only rebuilds to be significantly slower than full rebuilds. A couple years ago I finally figured out how we can safely fix this, so I or someone else just needs to spend the time to implement and test it.
Recursive file watchers on all platforms
We switched to using recursive file watchers on Windows and macOS, which is more efficient and detects file changes much quicker than how Meteor previously worked. Linux doesn’t support recursive file watchers, and the implementation used in most libraries would have created too many file watchers so it still uses the old file watching implementation. Recently @parcel/watcher added some features that would allow us to switch to it and use it on all platforms
We should also look into reducing the number of files Meteor watches. There are some folders Meteor considers as immutable, so Meteor only needs a single watcher for each of those.
Replace LRU cache
Many of the performance issues in Meteor the last few years have been from LRU caches not being configured large enough. However, constantly increasing the size of the caches will allow Meteor to use more memory than necessary for smaller apps. We should look into building a custom in-memory cache implantation that tracks which entries are needed and which can be freed so we don’t have to set an arbitrary size for each cache.
Rewrite the js-analyze code to use SWC
This part of isobuild handles scanning for imports and for globals, and over the last couple years it has become a performance issue. This would be a good place to experiment with using rust and/or wasm to speed up parts of the build process.
Switch back to using native tar
I don’t really understand the reason for Meteor switching away from it, but it seems like at some point there was a Meteor release that broke permissions in tar files created on Windows, so Meteor switched to using a js tar implementation that allows Meteor to fix the permissions when extracting the tar file on linux/macOS. Recently I found that the js tar implementation is significantly slower, and makes downloading a Meteor release or Meteor packages up to 2 - 3 times slower. It would be worth reviewing this and finding a way to switch back to using native tar.
noticed that the last few years, there hasn’t been as much public information on the reason for changes compared to before. The detailed PR’s and commit messages that Meteor had for many years is what allowed me to understand how and why Meteor worked the way it did and to start contributing to it.
Regarding switching to vite, I’m not sure that is the best path forward. Each bundler makes very different trade offs, and vite and isobuild are very different. A recent example is top level await. Having async code in the root of modules was very common with fibers, so we spent a lot of effort into creating a spec-compliant implementation for Meteor. In comparison, vite uses a different implementation in development and production, and the production implementation behaves differently and is not spec compliant.
It seems that the two main things missing from Meteor, besides features that we can add, are:
- performance
- vite has a larger ecosystem, at least in part to having better support for build plugins
For performance, I have a long list of things we can improve before we even need to think about a new architecture. Its been a few years since we’ve had a release focused on build performance.
For build plugins, it is surprisingly difficult to create a build plugin for Meteor - there are many bugs and it is difficult to create a well working plugin, and there is very little documentation. Besides that, Meteor’s current design for build plugins is from Meteor 1.2, and I’m not confident that being designed around batching has aged well. It probably is time to design a new, simplified build plugin api.
9 Likes
- For our users, tree-shaking would be the most important – by far.
- Reduced build times would be greatly appreciated for more sensible workflows
- TypeScript support is important to us for reducing bugs
4 Likes
There are currently some pain points when using meteor:
- Developer experience
- Reduction of the build time
- Faster hot refresh times
- Better TypeScript support
- User experience
- Tree-shaking
- Passkey support for securing current sessions
2 Likes
I see a big potential for PWA, especially for these workflows:
- create a new project with PWA compatibility out of the box
- build the project as PWA, ready for publish to the playStore/AppStore/WIndows Store
I’d like to give my two cents here as well (not ordered by priority):
Client Bundle Reduction
This involved the need for tree shaking but also reviewing core code. For example I did a simple lighthouse test with a new meteor project (meteor create testproject) and it showed me Registers an unload listener which is not recommended and I think part of some older packages.
Test coverage
Core and packages, especially with Tinytest, should be able to determine coverage.
Dependabot support for Atmosphere packages
I know this sounds too magical but I think we should consider that there are many NPM dependencies hidden in Meteor packages that won’t find their way into package-lock.json which is used by dependency management tools to check for updates and vulnerabilities.
Meteor Badges for shields.io
Sounds meta but think about having automatically badges rendered by .meteor/release or package.js.
3 Likes
Could you elaborate on the typescript support? We use refapp:meteor-typescript and other than typescript crashing because of oom problems, it works great.
Are you referring to types distributed with packages maybe? We tried using zodern:types but the quality control of embedded types wasn’t there yet
I agree with what most people have said, but I would add on that as meteor grows it should become more extensible. Meteor has a lot of really cool tools, but any specific tool will start to break down when the app gets complex enough. Right now the biggest pain points for us are with the build and bundling processes since those are the hardest to customize from within Meteor. For instance, one thing that recently came up for me was Meteor doesn’t support babel.config.js files and doing what I wanted in .babelrc was extremely cumbersome.
The other thing I think Meteor can help with is just better built in tools for testing, it was relatively hard to figure out how to get stuff setup correctly on circle ci the first time and there are a bunch of specialized tools that end up being pretty hard to use with Meteor.
2 Likes
I continue to love Meteor; now that I’ve embraced React I feel more secure in my apps futures. If there’s anything that can push me off Meteor now, it would be:
- Server side rendering, and this is becoming a growing need. Prerender is cool but feels clunky, I’ve had issues with it, etc. My latest app, Ninjalerts, will need SSR on millions of Ethereum transactions and I don’t think Prerender will cut it.
- scaling data later, excited to see redis-oplog coming into core
- modernizing MongoDB packages, and ChangeStreams support
- Meteor-Desktop is a hot mess and my app is in trouble because of it; I felt safe with it as it was part of community-packages, etc, but now my app is on a 2 year old Electron version with no reasonable upgrade path, bugs, etc
2 Likes
Ninjalerts is so cool!
Would React SSR components be compatible with Meteor and possibly work for your SSR needs?
1 Like
I haven’t explored it yet - would it?? 
I think there’s probably some things that need to be adjusted for subsriptions/etc - maybe its an easy lift though?
I’ve seen react-router-ssr but it’s not clear what’s going on with it / would feel safer with a vendor-supported package
I think React Server Components are a fairly new approach to SSR. I haven’t looked into them much yet at all. There are some recent YouTube videos with Dan Abramov talking about them.
What is your need for SSR?
-
So the user will have initial content to consume while loading the JS bundle? RSC cannot help because it requires JS in the client to operate. So before RSC can work, it still needs the JS bundle
-
So googlebot and other search engines can easily crawl your page (faster and more efficiently)? RSC will not be able to help because, again, it requires JS in the client to operate
-
Hey, googlebot can already crawl JS apps? Yes. But it does not need RSC to do that.
Does RSC work with SSR? Yes. But the output is the same as the normal SSR since everything must be created/built in the server before sending everything (one page) to the client. Therefore, from the perspective of SSR, there are no benefits from RSC. It will only be an added complexity.
Then why use RSC with SSR? If you are planning to use RSC on your main app and you also need SSR, the good news is that RSC can work with SSR without duplicating work. But there is no reason to use RSC because of SSR.
2 Likes