Atmosphere Packages + SWC/Rspack

Upgrading Atmosphere build pipeline for SWC/Rspack and modern ES6 libraries

Problem Statement

Modern ES6 libraries like helia.js (IPFS) and cornerstone3d.js (medical imaging) compile successfully when added to the main Meteor application but break when added to Atmosphere packages. This document outlines strategies for upgrading the Atmosphere package compile
pipeline to use SWC and support modern ES6/ESM libraries.

Claude Prompt

Claude, I’ve attached you to a local copy of https://github.com/meteor/meteor, which contains the meteor.js build tool. One simple request: please ultrathink on strategies for upgrading the Atmosphere package compile pipeline to also use Speedy Web Compiler, and to support modern ES6 libraries, such as helia.js and cornerstone3d.js (which will compile when those npm libraries are added to the main application, but break when added to an atmosphere package). You may wish to refer to https://forums.meteor.com/, https://github.com/meteor/meteor/issues, and https://github.com/meteor/meteor/discussions for more information on the current state of the refactor to SWC and Rspack in the main app and in the Atmosphere build pipeline.

Current State Analysis

SWC Integration Status

  • SWC is already integrated in Meteor 3.3+ for main app transpilation
  • Location: /packages/babel-compiler/babel-compiler.js (v1.12.14 of @meteorjs/swc-core)
  • Automatic fallback to Babel for incompatible code (nested imports, React compiler)
  • Minification also uses SWC (/packages/standard-minifier-js/plugin/minify-js.js)

Why Modern Libraries Break in Atmosphere Packages

Critical Files in Build Pipeline

/tools/isobuild/resolver.ts - Module resolution (mainFields, exports)
/tools/isobuild/compiler.js - Main compilation orchestration
/tools/isobuild/compiler-plugin.js - Plugin system, ImportScanner
/tools/isobuild/linker.js - Module linking (namespace differences)
/tools/isobuild/bundler.js - NodeModulesDirectory isolation
/tools/isobuild/import-scanner.ts - Module dependency scanning
/packages/babel-compiler/ - SWC/Babel transpilation
/packages/modules-runtime/ - Runtime module resolution

Meteor Roadmap Context

  • Phase 5 (Meteor 3.4): External bundler integration (Rspack/ESBuild/Rollup)
  • Next Releases: “Support package.json exports fields” listed as priority
  • Ongoing: PR #13520 for exports field support

Recommended Implementation Strategies

Strategy 1: “SWC Parity for Packages” (Immediate - Low Risk)

Goal: Enable the same SWC transpilation in packages that the main app already uses.

Modifications:

  1. /packages/babel-compiler/babel-compiler.js
  • Remove default excludePackages behavior
  • Add package-aware SWC configuration inheritance
  • Handle package-specific Reify post-processing
  1. /tools/isobuild/resolver.ts (lines 102-113)
 // CURRENT (server only uses main):
 this.mainFields = ["main"];

 // PROPOSED (server can use module field):
 this.mainFields = ["module", "main"];
  1. /packages/modules-runtime/server.js
  • Enhance Module.prototype.useNode for ESM imports via dynamic import

Complexity: 2-3 weeks | Risk: Low | Library Support: Partial


Strategy 2: “Modern Module Resolution” (Primary - Medium Risk)

Goal: Complete package.json “exports” field support and unify resolution across app/packages.

Modifications:

  1. /tools/isobuild/resolver.ts - Major enhancements
  • Complete exports field implementation (building on commit bb90b23c7e)
  • Add condition resolution: import, require, node, browser, default
  • Support conditional exports with nested conditions
  • Handle self-referencing packages
 // Add conditions based on architecture
 private conditions: string[];

 // Server:
 this.conditions = ['node', 'import', 'require', 'module-sync', 'default'];

 // Web:
 this.conditions = ['browser', 'import', 'module', 'default'];
  1. /tools/isobuild/import-scanner.ts (lines 159-198)
  • Respect type: “module” in package.json
  • Handle .mjs files natively without Reify transformation
  • Support import assertions/attributes
  1. /tools/isobuild/compiler-plugin.js (lines 1243-1252)
  • Add exports field awareness to PackageSourceBatch.getResolver()
  • Pass appropriate conditions based on target architecture
  1. /tools/isobuild/linker.js (lines 528-587)
  • Support ESM native output format
  • Handle type: “module” packages without closure transformation

Complexity: 4-6 weeks | Risk: Medium | Library Support: Good


Strategy 3: “External Bundler Delegation” (Future - Aligned with 3.4)

Goal: Allow packages to opt-in to Rspack/ESBuild processing for their dependencies.

Modifications:

  1. New File: /tools/isobuild/external-bundler.js
  • Abstraction layer for external bundlers
  • Package author configuration interface
  1. /tools/isobuild/bundler.js (NodeModulesDirectory)
  • Support pre-bundled node_modules
  • Handle meteor.bundler package.json configuration
  1. Package Configuration Schema:
 {
   "meteor": {
     "bundler": {
       "type": "rspack",
       "entry": "./src/index.js",
       "externals": ["meteor/*"]
     }
   }
 }

Complexity: 8-12 weeks | Risk: Medium | Library Support: Excellent


Strategy 4: “ESM-First Package Format” (Long-term)

Goal: New package format with native ESM support.

New Format Example (meteor-package.json):

 {
   "name": "my-package",
   "version": "1.0.0",
   "type": "module",
   "exports": {
     ".": "./src/index.js",
     "./client": "./src/client.js"
   },
   "meteor": {
     "depends": ["meteor", "mongo"],
     "provides": ["my-api"]
   }
 }

Complexity: 10-14 weeks | Risk: Medium | Library Support: Excellent


Recommended Implementation Order

 Phase 1 (Immediate - Meteor 3.4.x):
 ├── Strategy 1: SWC Parity for Packages
 └── Strategy 2: Modern Module Resolution (Priority)

 Phase 2 (Meteor 3.5):
 └── Strategy 3: External Bundler Delegation

 Phase 3 (Meteor 4.0):
 └── Strategy 4: ESM-First Package Format

Verification Plan

  1. Create test packages with helia.js and cornerstone3d.js as dependencies
  2. Build and run on all architectures (os, web.browser, web.browser.legacy)
  3. Verify imports resolve correctly from both package code and app code
  4. Check bundle sizes and build times before/after
  5. Test HMR functionality with package changes

Key External Resources

2 Likes

While this AI plan can have good insights on what can be done, the main purpose of integrating Rspack or SWC more closely into the Meteor bundler (the one that transpiles Atmosphere packages) should be treated as major work, with unclear benefits and a lot of risk.

There are many moving parts in the Meteor bundler, and the reason we adopted an independent external bundler like Rspack is because we aimed for an independent integration. In my opinion, we should not risk time and opportunities to improve other areas by trying to update old Meteor bundler pieces. That said, this can be a shared effort with the community and anyone is welcome to open PRs to cover parts of this plan. Still, I can foresee many changes triggering more and more considerations, as usually happens when modifying a legacy structure. The good part of this plan is that it proposes incremental efforts towards the long-term goal, so feel free to experiment with it.

That said, and back to the original goal: I strongly recommend projects move as much logic as possible out of Atmosphere packages and into the app level, or into an NPM package within a monorepo if you prefer. This gives you full support for ESM packages like helia.js and cornerstone3d.js for your project code. Moving to NPM packages is a more future-proof approach because you rely on standards and your app benefits from it, not only because Rspack can handle modern build dependencies, but also because it improves portability for other kinds of projects, migrations, and future features that a bundler like Rspack will likely cover on its own. I’d recommend doing this incrementally for the pieces of code that need it, instead of rely on a the Meteor bundler to support it.

2 Likes

Is this a stance of the Meteor core or more of a personal preference of yours? I’m not pointing fingers just genuinely asking

Because if it’s more of an official stance then we can grow to accept less support for Meteor packages as a whole and should consider outright moving and building NPM packages instead of Atmosphere ones.

1 Like

Definitely its a personal recommendation, to overcome a long limitation in Meteor projects, and because I prefer them. But the decision is on you. If, as time passes, the new era of bundlers, tools, and standards can’t be ported to the old, hard-to-maintain Atmosphere packaging pipeline, then you need a path around it, and the only one I feel can really overcome that is to think in NPM standards over Atmosphere. An official statement would make sense, but I don’t think we’ve acted accordingly.

Officially, we’ve delivered the Rspack integration. Now you even have the opportunity to consider fully ESM apps (before, you basically couldn’t). But we haven’t decided to abandon Atmosphere packages. What we’ve said is that we’ll offload hard tasks to external tooling over time, to reduce maintenance and unlock the benefits people are getting with Rspack today: better dev experience, plugins, and more advanced bundle customization. We’ve also said we’ll keep backwards compatibility whenever its possible, especially when its a “quick” win and doesn’t turn into a long struggle.

Atmosphere packages can still evolve if Meteor users put resources into them. On the core side, we’ve kept putting effort into them. Recently we improved them by adding ways to optimize with devOnly and devDepends (long request coming from GitHub issues), and more minor things may come next. The core roadmap doesn’t include any major direction for Atmosphere packages. We’ll update the roadmap soon to reorganize it, but the content itself hasn’t changed much. It doesn’t consider major atmosphere packages feature coverage or deprecations. We still live with them and provide ways to overcome their limitations.

One thing is saying everyone should migrate strictly, and thats not what we’ve said from core by today. Though I have my own opinions, I still see opportunities to work on Atmosphere packages. I still dive into the core to make changes and attend issues, implement specific atmosphere packages myself, and a lot of existing are available and working without requiring full rewrites. So why change something that works and spend a lot of effort on migrations and deprecations again?

Anyway, this can open a interesting discussion on what everybody think about them, and all that may influence future decisions surely.

2 Likes

Thanks for weighing in on this, @nacholozano .

This plan that Claude generated seems ambitious and complicated, which is why I posted it so people could kick the tires a bit. It roughly makes sense. but yes… lots of edge cases and quicksand to beware of.

Your comments about moving things to NPM are well received, and I’ve been thinking through what the steps would be to do that. We’ve wound up using Atmosphere in large part, because its syntax supports modular workflows. In particular, the —extra-packages has been very useful, and at some level, I’ve done consulting for the past decade relying on it.

Thinking about how we might move forward, I could imagine the following tool-chains providing the same functionalities but moving solidly toward NPM:

# need a way to compile the npm package
$ meteor build-package packages/my-npm-pkg

# and then to (optionally) publish it
$ meteor npm-publish packages/my-npm-pkg

# just-in-time addition of packages has been one of the best things about Atmosphere
$ meteor run —extra-npm-package my-npm-pkg

# just-in-time addition on deploy,ent has been a long-requested feature
$ meteor deploy —extra-npm-package my-npm-pkg

Would the above be more more practical and aligned with the direction that Meteor Dev Team wants to encourage the community to head towards? Fewer legacy dependencies, can build it from scratch with modern tooling, code coverage, etc.

I’m tempted to run the above through Claude, and get an estimate of complexity.

Would you say it is the right time to start considering a naming convention for NPM packages?

For instance, if I wanted to move an Accounts package to an NPM and it is not an official Meteor package, that would be @owner/accounts-bang. What the convention should be to filter the public NPM library for “our” Meteor packages?

Using @owner seems fairly straightforward enough. NPM supports tags, so as long as one includes meteor as a tag, others should be able to filter and find migrated packages.

1 Like

I asked Claude what it would take to re-implement the --extra-packages functionality using NPM + SWC + Rspack, and it seemed to think it could be done with only 3000 lines of code.

I agree with this approach. Another option is prefixing packages with meteor- when they depend on Meteor exclusively and are meant to be used in that context. But I like tags better, since you can keep broader compatibility beyond Meteor while still signaling Meteor support (for example, by checking for Meteor at runtime).

Could Claude directly open a PR for that? I suspect it would go red on many CI tests and Meteor checkout testing. AI isn’t infallible and it still needs supervision, iteration, and re-prompting with engineering context. I wouldn’t trust broad plans or rough estimates on complex tasks, but I’d use the info to build a more tailored plan that fits what we know about the system and iterate from there, definitely great insights AI provide.

Again, work to improve Atmosphere packages in this direction isn’t planned in the roadmap right now. Core priorities are elsewhere. Still, if someone (even with AI) can drive efforts towards meaningful improvements here, we’ll help with reviews and make sure nothing breaks, up to the point where it can be included in a final release.

I don’t understand well how the --extra-packages option helps you. Could you exemplify a use case you really see it useful in contrast of just have NPM packages directly referenced on package.json file or via Monorepo/Workspaces handling?

1 Like

The problem is not only with us searching for a tag. I see the problem more for the rest installing a package by the name and description just to find out that it has Meteor as a … “dependency” :))

I am thinking of @owner/meteorjs-package-name.
This way it is also clear in your package.json which of the NPMs are actual Meteor packages, which you can’t see with tags.

Hi Nacho,
Yes, I think we can open an entire PR for the reimplementation of —extra-packages as —extra-npm-packages using Claude to scaffold it out. If done surgically, we might be able to avoid the quicksand of trying to update legacy functionality, and do clean implementation that gets substantially the same functionality. If it’s something you’re willing to review, it’s something that I’d be willing to stage and work on.

As for the use case, it’s basically boils down to accounting, but that spills over to a bunch of squishy items ranging from API surface area, to certification, to legal liability, to trust networks, and division of labor. cloning and adjusting the package.json breaks certification, and requires that teams re-certify the software each time they change the package.json. Let’s step through it:

To get something federally certified, we need to pass a bunch of integration tests (~700+) and publish a software manual, or otherwise publish and pin down the package.json file. The idea is that the federal certification process wants a specific version of the software, which is consistent, presumably bug free, and deterministic. If we pass certification tests that say that a REST API behaves a particular way, it shouldn’t randomly change if we install Cornerstone or IPFS, and they pull in an extra copy of express that has middleware, right?

Now then, people can already clone the NodeOnFHIR repo, and add libraries to the base package.json. But that’s going to break certification, right? We saw this a lot in Meteor 2.x days with regard to UI libraries that relied on React prior to v16 and the Hooks API. Install the wrong library into the main package.json, and you have two copies of React at different versions, and the entire front end blows up.

So, what the —extra-packages does, is it lets us create a plugin architecture to the base tech stack, and it provides a way for us to lock down the main stack and guarantee that there is core functionality that is invariant. The published package.json allows us to pin down a specific combination of dependencies that we can get through certification testing (REST API via express, OAuth2 libraries, FHIR schemas, etc all working together). And —extra-packages allows us to just-in-time add extra workflow modules without changing that core package.json file.

Remember: NodeOnFHIR was developed like a giant Venn diagram of overlapping functionality; with core APIs refactored from dozens of prototypes. The idea is that we are trying to create a distributed app (App) that shares the same REST APIs, so my app can communicate with the next app, and the next app. It’s okay if people add extra workflows, user interfaces, processing libraries, whatever. Have at it. But those core APIs that have passed certification shouldn’t be fussed with. Swapping out the security encryption algos or moving to GraphQL because an engineer thinks it’s more performant or a LLM suggests it just doesn’t cut it, as far as social contracts and certification goes.

So, the idea is that we’re locking down a known API surface area in the main package.json, and we can —extra-packages custom workflows as needed, without breaking the certification. In the case of bad actors, the JIT extra packages quickly isolates ill behaving software, outside the certified scope. In the case of good actors, it accelerates time-to-market and prototyping, because they don’t have to re-certify software

Basically, it’s just an accounting method to lock down the APIs, as part of a social contract (which in the case of healthcare software involved in Medicaid/Medicare, involves rigorous contracts that establish allows participation in trust networks). Having extra-packages and downstream compile points are fine. It’s not so much an issue of technology, so much; it’s more about accounting and reproducibility.

Does that make sense?