New 3.4-beta.12 Release, Faster Builds, Smaller Bundles and Modern Setups with the Rspack integration ⚡

I get the reasoning for unwinding dependency on Atmosphere, and was around for the original angst and community arguments when NPM pulled the rug out from under Atmosphere’s value proposition. Since then, I’ve found Atmosphere packages to retain some value for workflow modularization. But I’m not wedded to Atmosphere, could be convinced to migrate to NPM. I think the bigger question at this time is the NPM build pipeline and package publishing.

A ‘meteor rollup’ command would go a long way to paving the way to migrating off of Atmosphere.

1 Like

This does happen (Meteor.isAppTest === true), but only when running meteor test --full-app, which is the only context where it’s expected according to docs. Are you running in that mode?

Ah you are right about the spec, the problem was more about not loading anything that was outside of client/server folders.

I agree that eager testing based on the client and server folders is missing. I’ll try to implement it.


Edit: Ok, I am checking your repository deeply, and I see the scenarios where it doesn’t, not sure why. But, is a confirmation that those tests you added there is what Meteor 3 bundler would do, right?

Update: Good progress so far. I’ll treat this as the final effort to ensure backward compatibility for eager tests in Meteor 3.4. We’re close to the official release, and I prefer not to add more changes now. Any additional updates (non-critical) will be handled in the Meteor 3.4.x series.

Ok yeah I updated my repo to something that was absolutely passing tests in Meteor 3 bundler.

Looks like you got the isAppTest thing working, but my understanding is there are 2 maybe 3 contexts, and isAppTest was only being set in one (the client) and needed to also be set in the server.

The unit test split I can work around if there is a new standard, but definitely need some way to separate client from server unit tests, and to me the simplest is to just mimic meteor 3 and base it on the folder they are under.

My understanding is there should only be one set of APP tests running, its unit tests that have both.

I think I have the fix for both scenarios: unit tests and full-app functional tests, with all the missing behaviors discussed. I need to add your tests to the modern suite so we can track coverage in Meteor core. I’ll also verify nothing else broke while adding the eager-testing specifics.

Thank you for the reproductions. They make it easier to move fast, and you always share repos with clear paths to the issues. :pray:

Only issue I found was with mySpec.spec.ts at the project root. It tries to import something that doesn’t exist at that level. I assume this was a mistake, so I ignored the file. Please confirm it wasn’t meant to do something else.

Hey, sharing some news here.

In a simple project, I’m able to use rspack locally and also build it with Docker. But I’m still facing issues with the public assets - they seem to be missing in the final bundle. You can see the logo not loading here https://binge-staging-2-codeftw.svc-us5.zcloud.ws/ (with rspack) but ok here https://binge.quave.dev (no rspack).

I didn’t have time to investigate the root cause to be honest, but @nachocodoner has access to the private repo.

Here is the Dockerfile I’m using currently:

FROM zcloudws/meteor-build:3.4-beta.12 as builder

WORKDIR /build/source
USER root

# Ensure Node.js is available in /usr/bin for rspack builds
# Some npm scripts use #!/usr/bin/env node and need node in PATH
RUN if ! command -v node >/dev/null 2>&1 || [ ! -f /usr/bin/node ]; then \
      NODE_PATH=$(command -v node 2>/dev/null || find /usr/local /home/zcloud/.meteor -name node -type f -executable 2>/dev/null | head -1) && \
      if [ -n "$NODE_PATH" ] && [ -f "$NODE_PATH" ]; then \
        mkdir -p /usr/bin && \
        ln -sf "$NODE_PATH" /usr/bin/node; \
      fi; \
    fi && \
    if ! command -v npm >/dev/null 2>&1 || [ ! -f /usr/bin/npm ]; then \
      NPM_PATH=$(command -v npm 2>/dev/null || find /usr/local /home/zcloud/.meteor -name npm -type f -executable 2>/dev/null | head -1) && \
      if [ -n "$NPM_PATH" ] && [ -f "$NPM_PATH" ]; then \
        mkdir -p /usr/bin && \
        ln -sf "$NPM_PATH" /usr/bin/npm; \
      fi; \
    fi

RUN chown zcloud:zcloud -R /build

USER zcloud
COPY --chown=zcloud:zcloud . /build/source

ENV METEOR_DISABLE_OPTIMISTIC_CACHING=1
# Ensure PATH includes /usr/bin where we symlinked node
ENV PATH="/usr/bin:/usr/local/bin:${PATH}"

# Install dependencies with optional dependencies (needed for rspack native bindings)
# Clean install approach to avoid npm optional dependency bugs
# The issue is that npm sometimes fails to install optional dependencies for native bindings
# We'll detect the platform and explicitly install the correct binding
# Normalize architecture: x86_64 -> x64, aarch64 -> arm64
RUN rm -rf node_modules package-lock.json && \
    meteor npm --no-audit --no-fund install && \
    ARCH=$(uname -m | sed 's/x86_64/x64/; s/aarch64/arm64/') && \
    PLATFORM="linux-${ARCH}-gnu" && \
    (meteor npm --no-audit --no-fund install "@rspack/binding-${PLATFORM}" 2>/dev/null || \
     meteor npm --no-audit --no-fund install "@rspack/binding-linux-x64-gnu" 2>/dev/null || true) && \
    meteor build --platforms web.browser --directory ../app-build

FROM zcloudws/meteor-node-mongodb-runtime:3.4-beta.12-with-tools
COPY --from=builder /build/app-build/bundle /home/zcloud/app

RUN cd /home/zcloud/app/programs/server && chmod +rw npm-shrinkwrap.json && npm --no-audit install

WORKDIR /home/zcloud/app

ENTRYPOINT ["/scripts/startup.sh"]

There are probably a lot of unnecessary commands there, but I want to make it work first and then optimize it.

Also, we’ve updated our base image, so an issue with node not being recognized outside Meteor should be fixed in the base image and not need these PATH fixes anymore in this Dockerfile.

Great progress overall, Nacho and team. It seems we’re getting closer and closer to a final fully working version :tada:

Our customers at Quave Cloud and Quave Services are excited with these updates.

6 Likes

Hey, good news, our assets are working fine after adding this to rspack.config.js

plugins: [
    // Copy all static assets from public folder to ensure they're included in the build
    // This is necessary for PWA files (sw.js, manifest.json), images, and other static assets
    new rspack.CopyRspackPlugin({
        patterns: [
            {
                from: 'public',
                to: '.',
                noErrorOnMissing: true,
            },
        ],
    }),
],

I think this should work by default as this is Meteor default behavior, let us see what @nachocodoner has to tell.

Our Dockerfile is also simple again, before it was missing Node.js on the PATH but now it’s fixed on our base image.

FROM zcloudws/meteor-build:3.4-beta.12 as builder

WORKDIR /build/source
USER root

RUN chown zcloud:zcloud -R /build

USER zcloud
COPY --chown=zcloud:zcloud . /build/source

RUN meteor npm i --no-audit  && meteor build --directory ../app-build

FROM zcloudws/meteor-node-mongodb-runtime:3.4-beta.12-with-tools

COPY --from=builder /build/app-build/bundle /home/zcloud/app

RUN cd /home/zcloud/app/programs/server && npm i --no-audit

WORKDIR /home/zcloud/app

ENTRYPOINT ["/scripts/startup.sh"]

Staging: https://staging-binge.quave.dev/
Prod: https://binge.quave.dev/

4 Likes

Have you tested adding assets in a simple non-monorepo app to see what happens? Any meteor build I tried copied assets by default in all envs without overrides, and tests cover this. Checking a non-monorepo case within your image could help rule that out and narrow the issue.

If I understand correctly, the app is a monorepo using npm namespaces, and the image only copies the web app folder, losing the root-level context with all node modules. In a monorepo, the root defines dev-only dependencies and other simplifications, likely affecting the build procedure. After verifying the non-monorepo case, I’d recommend copying the entire monorepo into the Docker context before building. If needed, remove root dependencies after the build.

I hope this helps shed more light on the scenario you hit.

That rspack change fixed my public assets also, thanks!

Live prod site at https://www.localmealprep.com/

I’d already replaced the logo/favicon image assets with CDN refs as a temporary fix, but the static robots.txt at the root had still been missing.

Here’s my full config for reference:

import { defineConfig } from '@meteorjs/rspack'
import rspack from '@rspack/core'
import path from 'path'
import { fileURLToPath } from 'url'

const __dirname = path.dirname(fileURLToPath(import.meta.url))

export default defineConfig(() => ({
  performance: {
    maxAssetSize: 512000,
    maxEntrypointSize: 512000,
    hints: process.env.NODE_ENV === 'production' ? 'warning' : false,
  },
  plugins: [
    new rspack.CopyRspackPlugin({
      patterns: [
        {
          from: 'public',
          to: '.',
          noErrorOnMissing: true,
        },
      ],
    }),
  ],
  resolve: {
    alias: {
      '@api': path.resolve(__dirname, 'imports/api'),
      '@lib': path.resolve(__dirname, 'imports/lib'),
      '@schemas': path.resolve(__dirname, 'imports/schemas'),
      '@ui': path.resolve(__dirname, 'imports/ui'),
    },
  },
}))

No @nachocodoner, it’s just a folder. I don’t use any monorepo tools. The fact that it’s a monorepo is completely ignored during the Docker process - we don’t send the whole code to our builder machine, so for the build process it’s not a monorepo at all.

I was even testing it by always firing the deploy from inside the web folder.

And again, this is a very simple project, without any fancy configurations, pretty standard project started from our Quave Meteor template.

The issue is that, for reasons we don’t know yet, the environment in your image doesn’t copy the public folder, so you need extra configuration in rspack.config.js.

From manual tests and test coverage, using meteor build --directory XXX works as expected, without any extra rspack.config.js config. I just added test coverage for that. You can see in the tests how adding assets in each app makes them appear in the final bundle built with meteor build.

Commit: 6b8be3b2e681162f7b17491566019b275dd21345

You reported privately that running meteor build on your machine worked as well, but not in the container. The container using your image behaves differently in the Meteor/Rspack build procedure. This doesn’t happen in other environments we tested so far, including dev and production envs; and other containers (including mine), where assets are bundled properly as part of meteor build. We still need to find why your image behaves differently to apply the proper fix there.

Using CopyRspackPlugin is a good workaround for your image. You could automate adding it on the fly for apps built with your image. We should not add it to core because it’s overconfiguration for most cases and can waste time copying assets, hurting the dev and build experience in large apps.

The proper fix should go into the image once we understand why assets don’t land there. At most, add a flag in the core to opt in to that CopyRspackPlugin config so you can enable it in your image, and not affect most of the scenarios where it works well. It’s still a workaround though. I’m concerned that if it behaves differently in one scenario, it may do differently in other cases, requiring additional workarounds in the future. But for now, it will help.

The fact that it’s a monorepo is completely ignored during the Docker process - we don’t send the whole code to our builder machine, so for the build process it’s not a monorepo at all.

The monorepo note was just an idea to test and hunt down the root cause. Yes, it’s an independent folder, but when copied it might carry node_modules status from a monorepo with symlinks that could affect things. Maybe not the problem, but the image does behave differently from every other environment we tested, so ideally we should debug further in different directions.


Btw, as a reference for everyone on another issue you faced: running meteor build on a Mac arm64 machine can trigger parcel watcher errors. It’s reported on GitHub. The workaround is to temporarily disable the modern watcher. This is another case where we should find the root cause, not force a workaround for everyone.

"meteor": {
  "modern": {
    "watcher": false
  }
}

At most, add a flag in the core to opt in to that CopyRspackPlugin config so you can enable it in your image,

As noted above, Meteor 3.4-beta.13 will add a new override file: rspack.config.override.<extension>, where <extension> matches your project’s rspack.config.<extension>.

If this file exists, its overrides and extensions apply to all configs, similar to the project root. This makes it easy for environments to provide their own aside the app’s config. For example, you can add the override below to your container image to apply custom rules to all apps using it.

const { defineConfig } = require("@meteorjs/rspack");
const rspack = require("@rspack/core");

module.exports = defineConfig((Meteor) => {
  return {
    plugins: [
      new rspack.CopyRspackPlugin({
        patterns: [
          {
            from: "public",
            to: ".",
            noErrorOnMissing: true,
          },
        ],
      }),
    ],
  };
});

This still seems a workaround for your issue, but hopefully at some point in the future we can understand the root issue, maybe others will hit it soon in other circumstances we can hunt down over time. At least with this solution we don’t need to add CopyRspackPlugin for everyone or pay the performance cost of extra config.

I’ll update here when it’s ready to test for everyone. The implementation and tests are already in place.


Btw, if the issue with assets only happens as part of build, you don’t need to include CopyRspackPlugin in development mode. You can add it only when Meteor.isBuild is true, and save the overconfig. So you could do something like this:

plugins: [
  Meteor.isBuild && new rspack.CopyRspackPlugin({
    patterns: [
      {
        from: "public",
        to: ".",
        noErrorOnMissing: true,
      },
    ],
  }),
].filter(Boolean),

For 3.4-beta.13 I have one issue affecting the local package case reported by @storyteller. If a quick fix isn’t possible (for the moment no luck on fixing it), I’ll postpone it or provide a workaround, since it’s an edge case.

2 Likes

I’m using this version now and it is working fine.