3.4-rc.1 Release Candidate, Faster Builds, Smaller Bundles and Modern Setups with the Rspack integration ⚡

I get the reasoning for unwinding dependency on Atmosphere, and was around for the original angst and community arguments when NPM pulled the rug out from under Atmosphere’s value proposition. Since then, I’ve found Atmosphere packages to retain some value for workflow modularization. But I’m not wedded to Atmosphere, could be convinced to migrate to NPM. I think the bigger question at this time is the NPM build pipeline and package publishing.

A ‘meteor rollup’ command would go a long way to paving the way to migrating off of Atmosphere.

1 Like

This does happen (Meteor.isAppTest === true), but only when running meteor test --full-app, which is the only context where it’s expected according to docs. Are you running in that mode?

Ah you are right about the spec, the problem was more about not loading anything that was outside of client/server folders.

I agree that eager testing based on the client and server folders is missing. I’ll try to implement it.


Edit: Ok, I am checking your repository deeply, and I see the scenarios where it doesn’t, not sure why. But, is a confirmation that those tests you added there is what Meteor 3 bundler would do, right?

Update: Good progress so far. I’ll treat this as the final effort to ensure backward compatibility for eager tests in Meteor 3.4. We’re close to the official release, and I prefer not to add more changes now. Any additional updates (non-critical) will be handled in the Meteor 3.4.x series.

Ok yeah I updated my repo to something that was absolutely passing tests in Meteor 3 bundler.

Looks like you got the isAppTest thing working, but my understanding is there are 2 maybe 3 contexts, and isAppTest was only being set in one (the client) and needed to also be set in the server.

The unit test split I can work around if there is a new standard, but definitely need some way to separate client from server unit tests, and to me the simplest is to just mimic meteor 3 and base it on the folder they are under.

My understanding is there should only be one set of APP tests running, its unit tests that have both.

I think I have the fix for both scenarios: unit tests and full-app functional tests, with all the missing behaviors discussed. I need to add your tests to the modern suite so we can track coverage in Meteor core. I’ll also verify nothing else broke while adding the eager-testing specifics.

Thank you for the reproductions. They make it easier to move fast, and you always share repos with clear paths to the issues. :pray:

Only issue I found was with mySpec.spec.ts at the project root. It tries to import something that doesn’t exist at that level. I assume this was a mistake, so I ignored the file. Please confirm it wasn’t meant to do something else.

Hey, sharing some news here.

In a simple project, I’m able to use rspack locally and also build it with Docker. But I’m still facing issues with the public assets - they seem to be missing in the final bundle. You can see the logo not loading here https://binge-staging-2-codeftw.svc-us5.zcloud.ws/ (with rspack) but ok here https://binge.quave.dev (no rspack).

I didn’t have time to investigate the root cause to be honest, but @nachocodoner has access to the private repo.

Here is the Dockerfile I’m using currently:

FROM zcloudws/meteor-build:3.4-beta.12 as builder

WORKDIR /build/source
USER root

# Ensure Node.js is available in /usr/bin for rspack builds
# Some npm scripts use #!/usr/bin/env node and need node in PATH
RUN if ! command -v node >/dev/null 2>&1 || [ ! -f /usr/bin/node ]; then \
      NODE_PATH=$(command -v node 2>/dev/null || find /usr/local /home/zcloud/.meteor -name node -type f -executable 2>/dev/null | head -1) && \
      if [ -n "$NODE_PATH" ] && [ -f "$NODE_PATH" ]; then \
        mkdir -p /usr/bin && \
        ln -sf "$NODE_PATH" /usr/bin/node; \
      fi; \
    fi && \
    if ! command -v npm >/dev/null 2>&1 || [ ! -f /usr/bin/npm ]; then \
      NPM_PATH=$(command -v npm 2>/dev/null || find /usr/local /home/zcloud/.meteor -name npm -type f -executable 2>/dev/null | head -1) && \
      if [ -n "$NPM_PATH" ] && [ -f "$NPM_PATH" ]; then \
        mkdir -p /usr/bin && \
        ln -sf "$NPM_PATH" /usr/bin/npm; \
      fi; \
    fi

RUN chown zcloud:zcloud -R /build

USER zcloud
COPY --chown=zcloud:zcloud . /build/source

ENV METEOR_DISABLE_OPTIMISTIC_CACHING=1
# Ensure PATH includes /usr/bin where we symlinked node
ENV PATH="/usr/bin:/usr/local/bin:${PATH}"

# Install dependencies with optional dependencies (needed for rspack native bindings)
# Clean install approach to avoid npm optional dependency bugs
# The issue is that npm sometimes fails to install optional dependencies for native bindings
# We'll detect the platform and explicitly install the correct binding
# Normalize architecture: x86_64 -> x64, aarch64 -> arm64
RUN rm -rf node_modules package-lock.json && \
    meteor npm --no-audit --no-fund install && \
    ARCH=$(uname -m | sed 's/x86_64/x64/; s/aarch64/arm64/') && \
    PLATFORM="linux-${ARCH}-gnu" && \
    (meteor npm --no-audit --no-fund install "@rspack/binding-${PLATFORM}" 2>/dev/null || \
     meteor npm --no-audit --no-fund install "@rspack/binding-linux-x64-gnu" 2>/dev/null || true) && \
    meteor build --platforms web.browser --directory ../app-build

FROM zcloudws/meteor-node-mongodb-runtime:3.4-beta.12-with-tools
COPY --from=builder /build/app-build/bundle /home/zcloud/app

RUN cd /home/zcloud/app/programs/server && chmod +rw npm-shrinkwrap.json && npm --no-audit install

WORKDIR /home/zcloud/app

ENTRYPOINT ["/scripts/startup.sh"]

There are probably a lot of unnecessary commands there, but I want to make it work first and then optimize it.

Also, we’ve updated our base image, so an issue with node not being recognized outside Meteor should be fixed in the base image and not need these PATH fixes anymore in this Dockerfile.

Great progress overall, Nacho and team. It seems we’re getting closer and closer to a final fully working version :tada:

Our customers at Quave Cloud and Quave Services are excited with these updates.

6 Likes

Hey, good news, our assets are working fine after adding this to rspack.config.js

plugins: [
    // Copy all static assets from public folder to ensure they're included in the build
    // This is necessary for PWA files (sw.js, manifest.json), images, and other static assets
    new rspack.CopyRspackPlugin({
        patterns: [
            {
                from: 'public',
                to: '.',
                noErrorOnMissing: true,
            },
        ],
    }),
],

I think this should work by default as this is Meteor default behavior, let us see what @nachocodoner has to tell.

Our Dockerfile is also simple again, before it was missing Node.js on the PATH but now it’s fixed on our base image.

FROM zcloudws/meteor-build:3.4-beta.12 as builder

WORKDIR /build/source
USER root

RUN chown zcloud:zcloud -R /build

USER zcloud
COPY --chown=zcloud:zcloud . /build/source

RUN meteor npm i --no-audit  && meteor build --directory ../app-build

FROM zcloudws/meteor-node-mongodb-runtime:3.4-beta.12-with-tools

COPY --from=builder /build/app-build/bundle /home/zcloud/app

RUN cd /home/zcloud/app/programs/server && npm i --no-audit

WORKDIR /home/zcloud/app

ENTRYPOINT ["/scripts/startup.sh"]

Staging: https://staging-binge.quave.dev/
Prod: https://binge.quave.dev/

5 Likes

Have you tested adding assets in a simple non-monorepo app to see what happens? Any meteor build I tried copied assets by default in all envs without overrides, and tests cover this. Checking a non-monorepo case within your image could help rule that out and narrow the issue.

If I understand correctly, the app is a monorepo using npm namespaces, and the image only copies the web app folder, losing the root-level context with all node modules. In a monorepo, the root defines dev-only dependencies and other simplifications, likely affecting the build procedure. After verifying the non-monorepo case, I’d recommend copying the entire monorepo into the Docker context before building. If needed, remove root dependencies after the build.

I hope this helps shed more light on the scenario you hit.

That rspack change fixed my public assets also, thanks!

Live prod site at https://www.localmealprep.com/

I’d already replaced the logo/favicon image assets with CDN refs as a temporary fix, but the static robots.txt at the root had still been missing.

Here’s my full config for reference:

import { defineConfig } from '@meteorjs/rspack'
import rspack from '@rspack/core'
import path from 'path'
import { fileURLToPath } from 'url'

const __dirname = path.dirname(fileURLToPath(import.meta.url))

export default defineConfig(() => ({
  performance: {
    maxAssetSize: 512000,
    maxEntrypointSize: 512000,
    hints: process.env.NODE_ENV === 'production' ? 'warning' : false,
  },
  plugins: [
    new rspack.CopyRspackPlugin({
      patterns: [
        {
          from: 'public',
          to: '.',
          noErrorOnMissing: true,
        },
      ],
    }),
  ],
  resolve: {
    alias: {
      '@api': path.resolve(__dirname, 'imports/api'),
      '@lib': path.resolve(__dirname, 'imports/lib'),
      '@schemas': path.resolve(__dirname, 'imports/schemas'),
      '@ui': path.resolve(__dirname, 'imports/ui'),
    },
  },
}))

No @nachocodoner, it’s just a folder. I don’t use any monorepo tools. The fact that it’s a monorepo is completely ignored during the Docker process - we don’t send the whole code to our builder machine, so for the build process it’s not a monorepo at all.

I was even testing it by always firing the deploy from inside the web folder.

And again, this is a very simple project, without any fancy configurations, pretty standard project started from our Quave Meteor template.

The issue is that, for reasons we don’t know yet, the environment in your image doesn’t copy the public folder, so you need extra configuration in rspack.config.js.

From manual tests and test coverage, using meteor build --directory XXX works as expected, without any extra rspack.config.js config. I just added test coverage for that. You can see in the tests how adding assets in each app makes them appear in the final bundle built with meteor build.

Commit: 6b8be3b2e681162f7b17491566019b275dd21345

You reported privately that running meteor build on your machine worked as well, but not in the container. The container using your image behaves differently in the Meteor/Rspack build procedure. This doesn’t happen in other environments we tested so far, including dev and production envs; and other containers (including mine), where assets are bundled properly as part of meteor build. We still need to find why your image behaves differently to apply the proper fix there.

Using CopyRspackPlugin is a good workaround for your image. You could automate adding it on the fly for apps built with your image. We should not add it to core because it’s overconfiguration for most cases and can waste time copying assets, hurting the dev and build experience in large apps.

The proper fix should go into the image once we understand why assets don’t land there. At most, add a flag in the core to opt in to that CopyRspackPlugin config so you can enable it in your image, and not affect most of the scenarios where it works well. It’s still a workaround though. I’m concerned that if it behaves differently in one scenario, it may do differently in other cases, requiring additional workarounds in the future. But for now, it will help.

The fact that it’s a monorepo is completely ignored during the Docker process - we don’t send the whole code to our builder machine, so for the build process it’s not a monorepo at all.

The monorepo note was just an idea to test and hunt down the root cause. Yes, it’s an independent folder, but when copied it might carry node_modules status from a monorepo with symlinks that could affect things. Maybe not the problem, but the image does behave differently from every other environment we tested, so ideally we should debug further in different directions.


Btw, as a reference for everyone on another issue you faced: running meteor build on a Mac arm64 machine can trigger parcel watcher errors. It’s reported on GitHub. The workaround is to temporarily disable the modern watcher. This is another case where we should find the root cause, not force a workaround for everyone.

"meteor": {
  "modern": {
    "watcher": false
  }
}

At most, add a flag in the core to opt in to that CopyRspackPlugin config so you can enable it in your image,

As noted above, Meteor 3.4-beta.13 will add a new override file: rspack.config.override.<extension>, where <extension> matches your project’s rspack.config.<extension>.

If this file exists, its overrides and extensions apply to all configs, similar to the project root. This makes it easy for environments to provide their own aside the app’s config. For example, you can add the override below to your container image to apply custom rules to all apps using it.

const { defineConfig } = require("@meteorjs/rspack");
const rspack = require("@rspack/core");

module.exports = defineConfig((Meteor) => {
  return {
    plugins: [
      new rspack.CopyRspackPlugin({
        patterns: [
          {
            from: "public",
            to: ".",
            noErrorOnMissing: true,
          },
        ],
      }),
    ],
  };
});

This still seems a workaround for your issue, but hopefully at some point in the future we can understand the root issue, maybe others will hit it soon in other circumstances we can hunt down over time. At least with this solution we don’t need to add CopyRspackPlugin for everyone or pay the performance cost of extra config.

I’ll update here when it’s ready to test for everyone. The implementation and tests are already in place.


Btw, if the issue with assets only happens as part of build, you don’t need to include CopyRspackPlugin in development mode. You can add it only when Meteor.isBuild is true, and save the overconfig. So you could do something like this:

plugins: [
  Meteor.isBuild && new rspack.CopyRspackPlugin({
    patterns: [
      {
        from: "public",
        to: ".",
        noErrorOnMissing: true,
      },
    ],
  }),
].filter(Boolean),

For 3.4-beta.13 I have one issue affecting the local package case reported by @storyteller. If a quick fix isn’t possible (for the moment no luck on fixing it), I’ll postpone it or provide a workaround, since it’s an edge case.

3 Likes

I’m using this version now and it is working fine.

Last extensibility & contributions release available: 3.4-beta.14. This release marks the final beta, wrapping up long-awaited production bundle optimizations, the last round of feedback for the Rspack integration, and including the remaining community contributions.

We’ve also included the final round of community contributions:

With this final beta, we’ve reached the key milestone pursued throughout the Meteor–Rspack integration. The next step is the official 3.4 release.

In about a week, we’ll publish the release candidate, the last chance to apply quick or critical fixes. After a short testing period, we’ll proceed with the 3.4 final release. During this time, we’ll continue reviewing feedback, updating docs and tutorials for Rspack, and preparing for the official launch.


Remember to check out the docs for details of the new modern build stack.

:page_with_curl: Modern Build Stack docs

:comet: Meteor Bundler optimizations docs

:zap: Rspack Bundler integration docs

5 Likes

After many hours and 3 or 4 different attempts, I am glad to say that I moved one of my projects to 3.4.
What I learned:

  • did a lot of mistakes in how I built some of the parts of my project in React.
  • in VS Code I was using a dead end launch configuration. It worked for exactly what I was doing, but it did not with any other things.
  • Rspack is fast.
  • besides Eslint, Rspack keeps a lot of things in check as far as the project structure goes.
  • defining global variables is really elegant. I used them mostly for things such as url roots for CDNs or other services.

The main issue I encountered in VS Code was that everything worked ok if I started the script from package.json and not much worked when starting with the Run and Debug of VS Code. This, combined with cumbersome innovation sneezed out of my brain years back in the shape of local NPM packages.

From here, looking forward to more improvements based on the documentation available.

Thanks a lot guys and I apoligize @nachocodoner if you ever felt your work was not appreciated. It was, been around long enough to understand what it takes to make something like this happen. Thank you.

4 Likes

Don’t need to apologize for anything. I appreciate you shared feedback constructively based on the issues you faced. I understand it can be hard to move to new configurations and tools. One good thing in Meteor 3.4 is that you can opt in and switch when you feel it’s the right moment. While you do, it’s great to get feedback from a developer’s perspective, above all in a beta.

I’m glad that thanks to the feedback, the release now has better docs, helpers and options for the common setups, and most importantly, the channel stays open to keep improving. Above all, I wanted everyone to feel the initial goals of this long-needed effort, which I started years ago in private and later with some public experiments, and see not only faster build times, but also smaller bundles and modern setups available for everyone who needs them via an integration to the Meteor core.

Let’s keep improving it through feedback and get it in an even better state in time!

8 Likes

This is probably not related, but I’m kinda stumped.

Has there been any big difference in how Meteor handles npm in the last couple updates? Our circleci is failing to install oxlint and tsgo correctly and its blocking me from testing this on our ci.

if (!nativeBinding) throw loadErrors.length > 0 ? Error("Cannot find native binding. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.", { cause: loadErrors.reduce((err, cur) => (cur.cause = err, cur)) }) : Error("Failed to load native binding");

I also got rspack to fail ot install


Error: Cannot find native binding. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.

Cannot find module './rspack.linux-x64-gnu.node'
Require stack:
- /home/circleci/app/node_modules/@rspack/binding/binding.js
Cannot find module '@rspack/binding-linux-x64-gnu'

I’ve had essentially the same CI config for multiple years and have never had anything similar to these problems, but updating to beta.14 and I can’t seem to get around it.

I had something similar; adding the following to my Dockerfile “fixed” it:

RUN rm -rf package-lock.json

It goes right before the meteor npm

RUN rm -rf package-lock.json
RUN meteor npm --no-audit --no-fund install  && meteor build --platforms web.browser --directory ../app-build

Probably not ideal to remove the package-lock.json, but that’s what’s working for me for now.

Meteor Client Bundles 3.3 vs 3.4 in my main project:

Meteor 3.3:
js: compressed 706kb, total size: 2.472kb

Meteor 3.4:
js: compressed 529kb, total size: 1.658kb

2 Likes

@nachocodoner I just updated from beta.12 to beta.14 and my build is now failing with this error:

p5s4j 11/18 4:05:28.176 PM e[35mThe following entries will be added to .gitignore:e[0m
p5s4j 11/18 4:05:28.176 PM e[35m  • _builde[0m
p5s4j 11/18 4:05:28.176 PM e[35m  • */build-assetse[0m
p5s4j 11/18 4:05:28.176 PM e[35m  • */build-chunkse[0m
p5s4j 11/18 4:05:28.176 PM e[35m  • .rsdoctore[0m
p5s4j 11/18 4:05:28.176 PM e[32m✅ Gitignore entries for Meteor Modern-Tools build context directories addede[0m
p5s4j 11/18 4:05:28.182 PM e[91me[31mRspack plugin error: Could not find rspack.config.js, rspack.config.ts, rspack.config.mjs, or rspack.config.cjs. Make sure @meteorjs/rspack is installed correctly.e[0m
p5s4j 11/18 4:05:28.195 PM e[0me[91mpackages/core-runtime.js:189
p5s4j 11/18 4:05:28.195 PM             throw error;
p5s4j 11/18 4:05:28.195 PM             ^
p5s4j 11/18 4:05:28.195 PM 
p5s4j 11/18 4:05:28.195 PM Error: Could not find rspack.config.js, rspack.config.ts, rspack.config.mjs, or rspack.config.cjs. Make sure @meteorjs/rspack is installed correctly.
p5s4j 11/18 4:05:28.196 PM     at getConfigFilePath (packages/rspack/lib/processes.js:167:9)
p5s4j 11/18 4:05:28.196 PM     at runRspackBuild (packages/rspack/lib/processes.js:434:22)
p5s4j 11/18 4:05:28.196 PM     at module.wrapAsync.self (packages/rspack/rspack_plugin.js:305:9)
p5s4j 11/18 4:05:28.196 PM     at processTicksAndRejections (node:internal/process/task_queues:105:5)
p5s4j 11/18 4:05:28.196 PM 
p5s4j 11/18 4:05:28.196 PM Node.js v22.21.1
p5s4j 11/18 4:05:29.210 PM The command 'bash -l -c meteor npm i --no-audit  && meteor build --directory ../app-build' returned a non-zero code: 1
p5s4j 11/18 4:05:29.220 PM e[0m"error":"Error on build job.",

I don’t have any other changes besides Meteor bump.