Fibers Public Roadmap and Meteor 3.0

announce
96

If we limit the scope to only app code on the server, it might not be too much work to add an experimental mode for native ESM. This doesn’t have to be part of Meteor 3. If we make it optional, we can add it in 3.x and iterate on it. There would be many breaking changes with native ESM.

Some of the things involved would be:

  • updating import scanner to handle ESM
  • not linking or having a simplified linking process
  • changing the boot process
  • making sure source maps on the server still work correctly
  • the current implementation of Assets requires there to be an app scope, which there wouldn’t be with ESM
  • allow app code to import packages in a way that live bindings still work
  • figure out how to handle packages and apps importing the same npm package to avoid having multiple instances of it where Node wouldn’t
  • change how core-runtime detects when all code is loaded
  • add a resolve hook so Node can find the Meteor packages and the app’s npm dependencies

The more interesting part is the client. For small apps, using native ESM might work, but the loading time scales with the number of modules, so at some point it becomes too slow. There’s also a number of features missing in ESM that Meteor would be expected or required to have. I’ve been thinking about this for a while. When I find the time, I will write a post describing the details, different options, trade-offs, what other bundlers have done, etc.

Three of the goals we would definitely have are:

  • reasonably fast loading time even for apps with 10,000’s of modules
  • Consistent behavior with the server’s module system
  • Support for npm packages that use commonjs

I don’t think Meteor’s current module system is that outdated. If we designed a new module system today, it might look very different, or, depending on the goals, features, and the trade offs you were willing to make, you might end up with a very similar design. Some new projects, such as Turbopack or bun, have many similarities with Meteor’s module system. Having said that, I am very interested in alternative designs that have the same benefits of the current system and minimize the issues with it (for example: bundle size, build time complexity, runtime complexity, some core packages unable to use modules). Since we would need the client and server module systems to behave consistently, while experimenting with native ESM would be a good time to explore this.

Before investing in native ESM, it might be good to fix the two main issues with Meteor’s current module system:

  • there is one code path left in reify that we never optimized, and in some cases can add a few seconds to the page load time
  • update Meteor’s resolver to support the package.json exports field. This probably will be needed even if we switch to native ESM
10 Likes
97

I think breaking changes can be avoided. All Meteor APIs are already exposed globally via Package.*. By implementing an ESM customization hook, the Package.* references can simply be exported to the import statements of any native modules.

This will be very similar to what I’ve done with native client-side ESM: my import map simply maps the specifiers to a file that exports stuff from Package.*. (Note, import maps are on the Node.js roadmap as an alternative standards-aligned way to achieve this later).

Customization hooks also allow transpiling content (f.e. they can read a .css file (from the specifier of an import statement), and make the default export be a string containing that CSS, and we can do similar with TypeScript, etc).

File caching would be needed to avoid compilation every single restart if files haven’t changed. A customization hook could also communicate with a dev server that provides cached results.

Here’s the Node’s example on how to begin to add URL imports like Deno: Modules: node:module API | Node.js v21.5.0 Documentation

Here’s a sample of what implementing meteor/foo modules might look like:

// meteor-hooks.mjs
export function load(url, context, nextLoad) {
  // Handle 'meteor/*' specifiers
  if (url.startsWith('meteor/')) {
    const lib = url.split('/')[1] // f.e. "tracker" in "meteor/tracker"
    return { format: 'module', source: `export default Package['${lib}']` }
  }

  // Let Node.js handle all other URLs.
  return nextLoad(url);
}

User code for that implementation pattern would need to get the default instead of a named export, and type defs would need to be adjusted, but it would work:

import tracker from 'meteor/tracker'
const {Tracker} = tracker

Alternatively, if we ensure that meteor packages all export the Uppercase name of the package, we can make it a named export:

// meteor-hooks.mjs
export function load(url, context, nextLoad) {
  // Handle 'meteor/*' specifiers
  if (url.startsWith('meteor/')) {
    const lib = url.split('/')[1] // f.e. "tracker" in "meteor/tracker"
    const pascalCased = dashToPascalCase(lib) // f.e. tracker to Tracker or foo-bar to FooBar
    return { format: 'module', source: `export const ${pascalCased} = Package['${lib}']['${pascalCased}']` }
  }

  // Let Node.js handle all other URLs.
  return nextLoad(url);
}

And then user code would be more like today’s:

import {Tracker} from 'meteor/tracker'

Doing it on native ESM with customization hooks (and/or with import maps later) you won’t have to worry about this. The native module resolution algo will handle that.

No longer resolve hooks (CommonJS), but customization hooks.

That’s a good point. I’ve yet to experience a problem big enough where I needed to bundle my app (with native CommonJS or with native ESM), but it imagine it could certainly be needed.

Bundling would be an optimization for that case, but I think it should be built anew, not based on the ecmascript package, but perhaps on something like esbuild or Vite if not something custom. It will need to support standards like import.meta.url which, when bundled, should not change meaning (console.log(import.meta.url) should log the same both with no bundling or with bundling).

I think the first step will be to get the native ESM mode working, without any such optimizations up front.

Trying to do it all at once will be far too much work, and having a plain ESM mode will already be ideal to begin with: people can start to use their own build tools to target native ESM mode, f.e. compile their own TS to JS, while existing users with existing apps can, for now, continue to use ecmascript and typescript packages.

An initial native ESM mode will usable for new Meteor apps, while existing apps may have migration work to do, but at least it would be a good start until optimzation features come later.

Even starting with no customization hooks, and just global Package for Meteor APIs, and user code as plain ES modules, would be great. Any step that gets us onto on the native ESM path will be great.

It won’t be needed with native ESM, the native loader read exports fields.

98

Wooo hoo! It runs! Excited to give this a test drive!

Screenshot 2024-01-18 at 6.20.17 PM
Screenshot 2024-01-18 at 6.20.17 PM1882×1134 146 KB

2 Likes
99

ESM, please! It’s 2024!

1 Like
100

Hello everyone!

Here is another update on what we’ve been working on in the past few weeks.

We fixed an issue where Symbol.asyncIterator on Cursor was not working. We updated the Reify package inside the core where it wasn’t updated yet. Also worked on issues like: Insert not working for collection created of the fly.

Right now, we are working on:

We also moved forward with the docs, and it’s looking really good. Here is a sneak peek:

Screenshot 2024-01-25 at 16.19.45
Screenshot 2024-01-25 at 16.19.451268×985 105 KB

We are planning on releasing a Beta 1 pretty soon, so stay tuned!

You can follow the updates on our overview doc , and on this Github PR

15 Likes
101

Hello everyone!

Here is one more update.

We release one more Beta, this one being beta.4.

It includes:

We’re already working on the tasks for the next one, so stay tuned!

You can follow the updates on our overview doc , and on this Github PR.

8 Likes
102

Hello everyone! If you are still not aware, a few days ago, we released our Beta 6!

You can check more about it here.

Your test and feedback is really important. See you soon!

6 Likes
103

Hi y’all! Just a quick update.

We’ve been working hard to finish some tasks in the past couple of weeks, and soon (between this week and the next one), we’ll release our last Beta!

After that, we’ll start working on the first RC right away.

Right now, I’m working on this PR, which is really important as it’ll update our skeletons to work with Meteor 3.

We also have this one that we want to fix before releasing this Beta. Then we’ll be ready.

That’s it for now, see you soon!

8 Likes
104

I just organized the content of the discussion and added estimated release dates for the last Beta and first RC versions.

5 Likes
105

For those deploying to AWS Node machines (EBS) or those using AWS SDK (such as @aws-sdk/client-s3) after 1 of May, Node 14 is no longer supported. Announcing the end of support for Node.js 14.x in the AWS SDK for JavaScript (v3) | AWS Developer Tools Blog

1 Like
106

I did not find any related announcement about deployment to AWS being affected by the drop of support of Node 14 from AWS SDK for JS V3.

P.S. This just reminded me that I need to move a number of projects from AWS SDK V2 to V3

107

I get the notification in Meteor 2.x projects where the SDK is installed.

Screenshot 2024-03-25 at 7.55.33 PM
Screenshot 2024-03-25 at 7.55.33 PM981×330 59.2 KB

I tried to deploy a Node 14 on a Node 20 machine and it didn’t work. Node 14 cannot be selected any more. If you have a new project it is possible that it cannot be deployed (although the bundle packs its own Node 14). If I remember right, I had problems related to node-gyp. I would assume the Node 14 AMI still exists after 1st of May and new machines can be started in the same autoscaling group.

In the worst case scenario, I believe deployments can be done on custom AMIs.

108

Node 14 is available until September 30, 2024, according to this page:

109

We have Meteor (and node 14) running on EB instances that come with Node 18. I don’t think the underlying installed version matters - as Meteor then runs it’s own bundled one.

110

How are you deploying to production? Did you mean that you are running the meteor tool in production (with the bundled node version)?

We do not run “meteor” nor the “bundled node version” in production. But our custom AMI uses the meteor team’s forked node version. Some folks might be ok with the provided AMI by AWS

111

We use mup and the EB plugin.

mup will use nvm to install the version of node required by meteor - so the version bundled with the AMI isn’t important.

1 Like
112

AFAIR the problem with deploying Meteor 2.15 on a Node 20 machine in EBS was related to Python and the compilation of/by pre-gyp/node-gyp which I think is used/necessary by bcrypt.

1 Like
113

Updates about the first Meteor 3.0 RC Release:

Due to some roadblocks, our ETA for the RC Release has been postponed to 19th April.

These are the tasks for the first RC:

Estimated Release Dates

  • Last Meteor 3.0 Beta: March 26, 2024 - meteor update --release 3.0-beta.7
  • First Meteor 3.0 RC: April 19, 2024
  • Meteor 2.16: We plan to start working on a release after the first RC

You can follow the updates on What’s left until an official Meteor 3.0? Discussion, and on the Release 3.0 PR.

8 Likes
114

It is time for a Weekly update :tada: :comet:

We are proud to say that we have the first Release Candidate for Meteor 3.0 :star_struck::partying_face:
You can check more details about it here.

The priority of the next RC will be to review and fix issues with Cordova’s experience.

Next Releases:

  • Meteor 2.16 [PR]
    • We plan to release it by the end of this week.
  • Meteor 3.0 RC (next): May 3, 2024.
    • The plan is to have a new release candidate every 2 weeks.

You can follow the updates on What’s left until an official Meteor 3.0? Discussion, and on the Release 3.0 PR.

5 Likes
115

It is time for a Weekly update :tada: :comet:

We are proud to say that we have the Release for 2.16 beta :star_struck::partying_face:
You can check more details about it here .

Tasks and fixes for the next Meteor 3.0 RC:

Next Releases:

  • Meteor 2.16 RC [PR]
    • We plan to release it early next week, probably Monday (May 6th).
  • Meteor 3.0 RC (next): Friday, May 3rd, 2024.
    • The plan is to have a new release candidate every 2 weeks.

You can follow the updates on What’s left until an official Meteor 3.0? Discussion, and on the Release 3.0 PR.

3 Likes