Seeking Community Input: Meteor.js Roadmap

announce
21

We are using both. In our case, redis-oplog has its redis DB that is different from the bull-queue redis DB. Separate your redis DBs to prevent any collisions. Or if you are forced to use only one redis DB, use namespaces; i.e. Bull use a prefix config to minimize collisions with other apps using the same redis DB

1 Like
22

We’re using oplogtoredis as well as the Redis for caching ourselves and it “just works”. Actually, we use the same Redis instance for multiple Meteor applications (almost 10), and all of them have oplogtoredis set up. As @rjdavid said – it’s all about using databases (you can set it up in the connection string).

As for the additional setup – it’s a Docker container we launched some time ago (two, three years back?) and it just works.

2 Likes
23

These are some items I would be interested in that haven’t already been listed:

Compatibility tables / support docs
It would be nice to have someplace we can easily check:

  • which versions of the MongoDB server a release of Meteor supports
  • browser support
  • which ECMAScript version is fully supported in all arch’s. Currently, it’s not always clear if a feature we can use in modern browsers would also be supported in legacy browsers.
  • compatibility for build plugins and core packages commonly used by build plugins. Build plugins are are unique in that they are built when published and bundle a copy of their deps, so which Meteor version you publish a build plugin with affects which older Meteor versions it can run in.
  • which Node and npm version are used by each Meteor release

More thorough docs
It’s improved the last couple years, but there are still many things that are undocumented. It would be nice if someone would go through the change log at least as far back as Meteor 1.3, and the issues from the old docs repo including those that were closed for being old, and add anything missing. Another thing that would be nice is going through the docs, migration guides, and tutorials and improving the language. There are many sections with poor grammar or that explain things in a more complicated and unclear way than necessary.

Security Updates
Currently, Meteor only consistently provides security updates for the latest release. This makes it difficult for companies that are a version or two behind since they have to rush to update to the latest release before they were planning to, and sometimes run into undocumented breaking changes or bugs that make it difficult or impossible to update. I know companies that migrated away from Meteor because it was too difficult for them to stay on a Meteor release that had a node version with the latest security patches.

Also, sometimes that are delays of up to several weeks before there is a new Meteor release after there is a new node version with a fix for a security issue.

It would be nice if Meteor in some form committed to providing security updates in a way that addresses these issues.

npm installer improvements
When I created the npm installer, I tried to make it work with as wide range of node version as I reasonably could to make it easier to install Meteor. However, now it only works with a limited range of node versions, none of which are still supported by Node which makes installing Meteor more difficult. There are also a few issues open documenting long standing issues with the installer.

Windows for a while now comes with a native tar, so it would be good to use that instead of 7zip when available. This would make the installer smaller, and Windows Defender has some exceptions for the native tar which should make Meteor faster to install.

Remove ECMAScript.compileForShell
This could maybe be done by moving the function to the babel-compiler package, or moving the code to the one place it is used. This function causes every build plugin and app that uses the ecmascript package to have a full copy of the babel-compiler package, in addition to every Meteor release. A few years ago, I found that half the disk usage (I think half was ~5gb) in ~/.meteor was just copies of babel-compiler, and it was also a large reason installing Meteor was so slow on Windows.

Isobuild

Linker rewrite
This is probably one of the next things I will work on. The linker in Meteor is far behind the state of the art in terms of performance. Linkers in js bundlers have always been slow, and Meteor is missing many of the improvements other bundlers have implemented the past few years. It might even be possible to remove the disk cache, which causes its own performance issues.

Fully enable in-place builds on Windows
On Windows it is still disabled for client only builds, which for large apps can cause client only rebuilds to be significantly slower than full rebuilds. A couple years ago I finally figured out how we can safely fix this, so I or someone else just needs to spend the time to implement and test it.

Recursive file watchers on all platforms
We switched to using recursive file watchers on Windows and macOS, which is more efficient and detects file changes much quicker than how Meteor previously worked. Linux doesn’t support recursive file watchers, and the implementation used in most libraries would have created too many file watchers so it still uses the old file watching implementation. Recently @parcel/watcher added some features that would allow us to switch to it and use it on all platforms

We should also look into reducing the number of files Meteor watches. There are some folders Meteor considers as immutable, so Meteor only needs a single watcher for each of those.

Replace LRU cache
Many of the performance issues in Meteor the last few years have been from LRU caches not being configured large enough. However, constantly increasing the size of the caches will allow Meteor to use more memory than necessary for smaller apps. We should look into building a custom in-memory cache implantation that tracks which entries are needed and which can be freed so we don’t have to set an arbitrary size for each cache.

Rewrite the js-analyze code to use SWC
This part of isobuild handles scanning for imports and for globals, and over the last couple years it has become a performance issue. This would be a good place to experiment with using rust and/or wasm to speed up parts of the build process.

Switch back to using native tar
I don’t really understand the reason for Meteor switching away from it, but it seems like at some point there was a Meteor release that broke permissions in tar files created on Windows, so Meteor switched to using a js tar implementation that allows Meteor to fix the permissions when extracting the tar file on linux/macOS. Recently I found that the js tar implementation is significantly slower, and makes downloading a Meteor release or Meteor packages up to 2 - 3 times slower. It would be worth reviewing this and finding a way to switch back to using native tar.

noticed that the last few years, there hasn’t been as much public information on the reason for changes compared to before. The detailed PR’s and commit messages that Meteor had for many years is what allowed me to understand how and why Meteor worked the way it did and to start contributing to it.

Regarding switching to vite, I’m not sure that is the best path forward. Each bundler makes very different trade offs, and vite and isobuild are very different. A recent example is top level await. Having async code in the root of modules was very common with fibers, so we spent a lot of effort into creating a spec-compliant implementation for Meteor. In comparison, vite uses a different implementation in development and production, and the production implementation behaves differently and is not spec compliant.

It seems that the two main things missing from Meteor, besides features that we can add, are:

  • performance
  • vite has a larger ecosystem, at least in part to having better support for build plugins

For performance, I have a long list of things we can improve before we even need to think about a new architecture. Its been a few years since we’ve had a release focused on build performance.

For build plugins, it is surprisingly difficult to create a build plugin for Meteor - there are many bugs and it is difficult to create a well working plugin, and there is very little documentation. Besides that, Meteor’s current design for build plugins is from Meteor 1.2, and I’m not confident that being designed around batching has aged well. It probably is time to design a new, simplified build plugin api.

9 Likes
24
  1. For our users, tree-shaking would be the most important – by far.
  2. Reduced build times would be greatly appreciated for more sensible workflows
  3. TypeScript support is important to us for reducing bugs
4 Likes
25

There are currently some pain points when using meteor:

  1. Developer experience
  • Reduction of the build time
  • Faster hot refresh times
  • Better TypeScript support
  1. User experience
  • Tree-shaking
  • Passkey support for securing current sessions
2 Likes
26

I see a big potential for PWA, especially for these workflows:

  • create a new project with PWA compatibility out of the box
  • build the project as PWA, ready for publish to the playStore/AppStore/WIndows Store
27

I’d like to give my two cents here as well (not ordered by priority):

Client Bundle Reduction

This involved the need for tree shaking but also reviewing core code. For example I did a simple lighthouse test with a new meteor project (meteor create testproject) and it showed me Registers an unload listener which is not recommended and I think part of some older packages.

Test coverage

Core and packages, especially with Tinytest, should be able to determine coverage.

Dependabot support for Atmosphere packages

I know this sounds too magical but I think we should consider that there are many NPM dependencies hidden in Meteor packages that won’t find their way into package-lock.json which is used by dependency management tools to check for updates and vulnerabilities.

Meteor Badges for shields.io

Sounds meta but think about having automatically badges rendered by .meteor/release or package.js.

3 Likes
28

Could you elaborate on the typescript support? We use refapp:meteor-typescript and other than typescript crashing because of oom problems, it works great.

Are you referring to types distributed with packages maybe? We tried using zodern:types but the quality control of embedded types wasn’t there yet

29

I agree with what most people have said, but I would add on that as meteor grows it should become more extensible. Meteor has a lot of really cool tools, but any specific tool will start to break down when the app gets complex enough. Right now the biggest pain points for us are with the build and bundling processes since those are the hardest to customize from within Meteor. For instance, one thing that recently came up for me was Meteor doesn’t support babel.config.js files and doing what I wanted in .babelrc was extremely cumbersome.

The other thing I think Meteor can help with is just better built in tools for testing, it was relatively hard to figure out how to get stuff setup correctly on circle ci the first time and there are a bunch of specialized tools that end up being pretty hard to use with Meteor.

2 Likes
30

I continue to love Meteor; now that I’ve embraced React I feel more secure in my apps futures. If there’s anything that can push me off Meteor now, it would be:

  • Server side rendering, and this is becoming a growing need. Prerender is cool but feels clunky, I’ve had issues with it, etc. My latest app, Ninjalerts, will need SSR on millions of Ethereum transactions and I don’t think Prerender will cut it.
  • scaling data later, excited to see redis-oplog coming into core
  • modernizing MongoDB packages, and ChangeStreams support
  • Meteor-Desktop is a hot mess and my app is in trouble because of it; I felt safe with it as it was part of community-packages, etc, but now my app is on a 2 year old Electron version with no reasonable upgrade path, bugs, etc
2 Likes
31

Ninjalerts is so cool!

Would React SSR components be compatible with Meteor and possibly work for your SSR needs?

1 Like
32

I haven’t explored it yet - would it?? :thinking:

I think there’s probably some things that need to be adjusted for subsriptions/etc - maybe its an easy lift though?

I’ve seen react-router-ssr but it’s not clear what’s going on with it / would feel safer with a vendor-supported package

33

I think React Server Components are a fairly new approach to SSR. I haven’t looked into them much yet at all. There are some recent YouTube videos with Dan Abramov talking about them.

34

What is your need for SSR?

  • So the user will have initial content to consume while loading the JS bundle? RSC cannot help because it requires JS in the client to operate. So before RSC can work, it still needs the JS bundle

  • So googlebot and other search engines can easily crawl your page (faster and more efficiently)? RSC will not be able to help because, again, it requires JS in the client to operate

  • Hey, googlebot can already crawl JS apps? Yes. But it does not need RSC to do that.

Does RSC work with SSR? Yes. But the output is the same as the normal SSR since everything must be created/built in the server before sending everything (one page) to the client. Therefore, from the perspective of SSR, there are no benefits from RSC. It will only be an added complexity.

Then why use RSC with SSR? If you are planning to use RSC on your main app and you also need SSR, the good news is that RSC can work with SSR without duplicating work. But there is no reason to use RSC because of SSR.

2 Likes
35

I was going to ask the same thing, it is not clear what the SSR use case is either.

I have used SSR to render custom URLs, speed up landing pages and social sharing but the rest where done via SPA. I understand why it might useful for websites to render content with data on request but for webapps, what is the exact need?

36

It’s pretty much for SEO and rendering link previews on social media. Imagine building a forum on Meteor without proper SSR.

I don’t care much for hydration/etc - think its a bit overrated as most SPAs will load really fast and you can use dynamic modules if they become clunky.

I have read for a while that Google can crawl JS apps, but I have not seen this work with any of my Meteor apps. I’ve always had to use prerender.

1 Like
37

Another addition from a real-world use-case.

Again I traveled through high-tech Germany by state-of-the-art railroad /irony off and faced the issue of having no mobile internet connectivity at all. Meteor development is pretty good off line, however there is one exception. When creating a new project with an older Meteor release it tries to get the latest release until timeout. I’d like to skip this and just use my older version and the skeleton, supporting this version.

Summarized: complete offline development capability.

3 Likes
38

That you can do today I believe with Meteor SSR.

39

I’m really eager for Linux ARM support for these 2 reasons:

  • Enable the ability to build vscode devcontainers running linux on Mac M1 hardware
  • Enable the use of Oracle Cloud ARM (free-forever) instances for CI (github action runner or drone agent)

The PR seems to be ready for a while now, and I’m currently running meteor from a checkout and manually merging the PR to get my pipelines to work.

1 Like
40

I’ve run into a use case that I think can be interesting as a Meteor feature. The line of thinking here is, the key for Meteor is to support web standards and abstract away the boring stuff, but not necessarily provide opinionated features. Those are easy to build once the foundational pieces are there.

One thing I’d like to see is good support for push notifications

  • currently, getting it to work with web/cordova is tricky
    • in contrast, Capacitor has its own push notifications plug-in
  • most end up using Firebase Cloud Messaging but
    • its a wierd integration
    • adds a heavy and complicated SDK to the app
    • adds a layer of latency between your service and customer
  • third party packages for this look abandoned

I think the advantage to bringing this into Meteor core is:

  • you can associated every user loginToken with every pushRegistrationId
    • when a user logs out, the pushRegistrationId is deleted
    • users can manage push notifications across various logins/devices
  • every Meteor loginToken can store some extra data on what the device name is, where it was signed in, etc
    • you can then build an interface where people can see where they are signed in, if they have push on, to sign out a particular device, to pause push on a particular device, etc
  • maybe difficult; do not send push notification if the device has an active websocket session
    • I wonder if redis-oplog can open the door to some kind of simple presence detection for Meteor users
2 Likes