Meteor Client Bundles 3.3 vs 3.4 in my main project:
Meteor 3.3:
js: compressed 706kb, total size: 2.472kb
Meteor 3.4:
js: compressed 529kb, total size: 1.658kb
2 Likes
@nachocodoner I just updated from beta.12 to beta.14 and my build is now failing with this error:
p5s4j 11/18 4:05:28.176 PM e[35mThe following entries will be added to .gitignore:e[0m
p5s4j 11/18 4:05:28.176 PM e[35m • _builde[0m
p5s4j 11/18 4:05:28.176 PM e[35m • */build-assetse[0m
p5s4j 11/18 4:05:28.176 PM e[35m • */build-chunkse[0m
p5s4j 11/18 4:05:28.176 PM e[35m • .rsdoctore[0m
p5s4j 11/18 4:05:28.176 PM e[32m✅ Gitignore entries for Meteor Modern-Tools build context directories addede[0m
p5s4j 11/18 4:05:28.182 PM e[91me[31mRspack plugin error: Could not find rspack.config.js, rspack.config.ts, rspack.config.mjs, or rspack.config.cjs. Make sure @meteorjs/rspack is installed correctly.e[0m
p5s4j 11/18 4:05:28.195 PM e[0me[91mpackages/core-runtime.js:189
p5s4j 11/18 4:05:28.195 PM throw error;
p5s4j 11/18 4:05:28.195 PM ^
p5s4j 11/18 4:05:28.195 PM
p5s4j 11/18 4:05:28.195 PM Error: Could not find rspack.config.js, rspack.config.ts, rspack.config.mjs, or rspack.config.cjs. Make sure @meteorjs/rspack is installed correctly.
p5s4j 11/18 4:05:28.196 PM at getConfigFilePath (packages/rspack/lib/processes.js:167:9)
p5s4j 11/18 4:05:28.196 PM at runRspackBuild (packages/rspack/lib/processes.js:434:22)
p5s4j 11/18 4:05:28.196 PM at module.wrapAsync.self (packages/rspack/rspack_plugin.js:305:9)
p5s4j 11/18 4:05:28.196 PM at processTicksAndRejections (node:internal/process/task_queues:105:5)
p5s4j 11/18 4:05:28.196 PM
p5s4j 11/18 4:05:28.196 PM Node.js v22.21.1
p5s4j 11/18 4:05:29.210 PM The command 'bash -l -c meteor npm i --no-audit && meteor build --directory ../app-build' returned a non-zero code: 1
p5s4j 11/18 4:05:29.220 PM e[0m"error":"Error on build job.",
I don’t have any other changes besides Meteor bump.
Have you tried running npm install again from the project/app? I’ll update that message to suggest it, since this already came up before and required doing that.
1 Like
I think this is a Node/npm ecosystem issue (see #4828), triggered more visibly after upgrading to Rspack 1.6.0 (which adds new native bindings) and Node 22.21.0.
It’s a known npm bug with optionalDependencies where a stale or platform-mismatched package-lock.json can cause npm to skip installing the correct native binding. When that happens, npm looks at the lockfile, decides “no native optional dependency for this platform”, and the install fails, the same error a lot of other projects are seeing if you look into the log message.
1 Like
I haven’t changed our Dockerfile.
So I believe we already do what you’re asking me to do, see:
FROM zcloudws/meteor-build:3.4-beta.14 as builder
WORKDIR /build/source
USER root
RUN chown zcloud:zcloud -R /build
USER zcloud
COPY --chown=zcloud:zcloud . /build/source
RUN meteor npm i --no-audit && meteor build --directory ../app-build
FROM zcloudws/meteor-node-mongodb-runtime:3.4-beta.14-with-tools
COPY --from=builder /build/app-build/bundle /home/zcloud/app
RUN cd /home/zcloud/app/programs/server && npm i --no-audit
WORKDIR /home/zcloud/app
ENTRYPOINT ["/scripts/startup.sh"]
And again, something definitely changed from .12 to .14. It was working fine and is still working fine with .12.
So great! I bet your build times have also improved.
Ok, the Client bundle shaved off 20%. Anyone noticed a doubling of the server memory?
254MB with 3.3.2 and 515 with 3.4-Beta.14
Yes, the same from 3.3.2 to 3.4-Beta-8.
I’m working on the last reports: legacy build issues and problems updating rspack and @meteorjs/rspack when upgrading from beta versions, since I now have reproductions for these.
Regarding the server RAM doubling issue, I haven’t noticed anything, and it’s true I haven’t really checked it deeply yet. It’s a good reminder to look at those metrics as well. I’ll review RAM usage in some apps migrated to Rspack. Have you seen this effect in simple Meteor apps too? I’ll test those as well, because if it doesn’t happen generally it may be related to specific apps, dependencies, or config that I’d like to understand.
Any extra info or reproductions you have are welcome, so I can focus on all reports and review them straight away.
2 Likes
I was able to research the issue. With your image I was able to see it happening consistently and reproduce it with this one-line command:
meteor create --release 3.4-beta.12 --react 34beta14 && \
cd 34beta14 && \
meteor build --directory ../test-app && \
meteor update --release 3.4-beta.14 && \
meteor build --directory ../test-app
This has appeared in previous betas, and the workaround has been to re-run npm install after the NPM dependency bump (rspack dependencies in the case of beta.14 when meteor run/build). I tried to find a way so that meteor build / meteor run would ensure the new node_modules are installed correctly, but I couldn’t get it to work. It seems that if you run npm install or bump dependencies inside the Meteor process (a Node process), the node_modules are not guaranteed to be installed properly in the running project. In this case, @meteorjs/rspack and rspack bumped in beta.14 end up missing from node_modules (and actually more dependencies are missing), and you need to run npm install from your shell instead. I tried several approaches, including doing it outside the project context, but none of them fixed this automatically.
Because of that, I updated meteor update to bump rspack and @meteorjs/rspack earlier during the Meteor release update, and added a log message suggesting to run npm install to keep dependencies and the project stable. After this PR is merged and the Meteor tool is published, once your image updates Meteor, the first meteor npm i --no-audit could address this.
In any case, I suggest adding an automatic retry after the first meteor build attempt as a likely fallback fix:
RUN meteor npm i --no-audit \
&& (meteor build --directory ../app-build \
|| (meteor npm i --no-audit && meteor build --directory ../app-build))
With this, it worked in my environment. I hope this helps.
1 Like
I’ve been able to test the Galaxy app (a real-world production app) migrated to Meteor 3.4-beta.14 with Rspack and check the RAM consumption. On my side, I haven’t seen any increase in RAM usage, actually, I’m seeing lower RAM usage on the live data graphs.
I’m not completely sure what’s happening in your cases, but it’s likely that your issue is coming from somewhere else. If Rspack is the culprit, it might be related to a specific dependency or configuration, and I’d really like to have a reproducible setup to investigate.
@paulishca, @harry73: could I ask for your help to create a minimal app that demonstrates the issue? Ideally, we’d have the exact code isolated so we can compare how Meteor 3.3.x and 3.4 behave in terms of resource usage for your specific use cases.
Since I’ve seen several apps that don’t show this RAM usage problem, I’m planning to move forward with the RC soon. At this point, I don’t see this as a general issue affecting the major of apps.
If there are other users who have already migrated to Rspack, could you also check and share your server RAM usage results?
1 Like
Where I am seeing the RAM problem is my main project, very large, years of development, wouldn’t really know where to start to reproduce something relevant.
I am having other similar projects but much smaller, which I can update to 3.4 and see what is happening. They all use the same mix: React, MUI, React Router, Redux.
I will get back on this, possibly tomorrow.
Something weird is happening with the full-app tests and I don’t think this is behaving like Meteor 3.3
In my main app I am getting server errors in the browser console and that just doesn’t make any sense.
I also see that it says “RUNNING APP SERVER TESTS” but the server part doesn’t make any sense in this context so I think there is some confusion.
My understanding of 3.3 full app testing infrastructure is that when you run meteor test --full-app --driver-package meteortesting:mocha
You end up with 3 different contexts.
- A normal client bundle starting from the mainModule.client file with Meteor.isAppTest set to true
- A normal server bundler starting from the mainModule.server file with Meteor.isAppTest set to true
- A testing context that eager loads all .app-test. files and runs the mocha tests. My app depends on being able to go to localhost:3000 and navigating around via puppeteer.
My repro has a test that fails because of that now
The current 3.4 is doing something very weird and does not enable my tests to work.
Also there is something else funky as my circle ci is unable to find any tests when its trying to run them, but locally its fine. I still need to investigate this issue, but something there is broken.
Edit on circleci this is the entirety of my _build/test/server-rspack.js
/*!
* /**
* * @file server-rspack.js
* * @description Bundled output generated by Rspack for tests
* * --------------------------------------------------------------------------
* * ⚡ Rspack Test Server App (Development)
* * --------------------------------------------------------------------------
* * • [ server-entry.js ] ──▶ [■ server-rspack.js ] ──▶ [ server-meteor.js ]
* *
* * This file is the bundle that Rspack outputs for tests. It contains all of
* * your test code in one optimized file. Next step is loading this bundle via
* * `server-meteor.js`.
* *
* * ⚠️ Note: This file is autogenerated. It is not meant to be modified manually.
* * These files also act as a cache: they can be safely removed and will be
* * regenerated on the next build. They should be ignored in IDE suggestions
* * and version control.
* * /
*
* /* Code generated * /
*
*/
(() => {
var __webpack_modules__ = ({
"./. eager recursive \\.(?:test%7Cspec)s?\\.[^.]+$ exclude: (?:^%7C\\/).*\\/node_modules\\/.*$%7C(?:^%7C\\/).*\\/\\.meteor\\/.*$%7C(?:^%7C\\/).*\\/public\\/.*$%7C(?:^%7C\\/).*\\/private\\/.*$%7C(?:^%7C\\/).*\\/_build\\/.*$%7C(?:^%7C\\/).*\\/fakeServer\\/.*$%7C(?:^%7C\\/).*\\/app\\/android$%7C(?:^%7C\\/).*\\/app\\/ios$%7C(?:^%7C\\/).*\\/cloudslicer\\/.*$%7C(?:^%7C\\/).*\\/tools\\/.*$%7C(?:^%7C\\/).*\\/dataAnalyzer\\/.*$%7C(?:^%7C\\/).*\\/app\\/app\\/.*$%7C(?:^%7C\\/).*\\/app\\/e2e\\/.*$%7C(?:^%7C\\/).*\\/app\\/.*tests.*\\/.*$%7C(?:^%7C\\/).*\\/app\\/Assets\\/.*$%7C(?:^%7C\\/).*\\/app\\/node_modules$%7C(?:^%7C\\/).*\\/[^/]*\\.native\\.js$%7C(?:^%7C\\/).*\\/gen\\/.*$%7C(?:^%7C\\/).*\\/services\\/target$%7C(?:^%7C\\/).*\\/app\\/.*$%7C(?:^%7C\\/).*\\/client\\/.*$":
/*!****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************!*\
!*** ./.|eager|/\.(?:test%7Cspec)s?\.[^.]+$/|exclude: /(?:^|\/).*\/node_modules\/.*$|(?:^|\/).*\/\.meteor\/.*$|(?:^|\/).*\/public\/.*$|(?:^|\/).*\/private\/.*$|(?:^|\/).*\/_build\/.*$|(?:^|\/).*\/fakeServer\/.*$|(?:^|\/).*\/app\/android$|(?:^|\/).*\/app\/ios$|(?:^|\/).*\/cloudslicer\/.*$|(?:^|\/).*\/tools\/.*$|(?:^|\/).*\/dataAnalyzer\/.*$|(?:^|\/).*\/app\/app\/.*$|(?:^|\/).*\/app\/e2e\/.*$|(?:^|\/).*\/app\/.*tests.*\/.*$|(?:^|\/).*\/app\/Assets\/.*$|(?:^|\/).*\/app\/node_modules$|(?:^|\/).*\/[^/]*\.native\.js$|(?:^|\/).*\/gen\/.*$|(?:^|\/).*\/services\/target$|(?:^|\/).*\/app\/.*$|(?:^|\/).*\/client\/.*$/ ***!
\****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************/
(function (module) {
function webpackEmptyAsyncContext(req) {
// Here Promise.resolve().then() is used instead of new Promise() to prevent
// uncaught exception popping up in devtools
return Promise.resolve().then(function() {
var e = new Error("Cannot find module '" + req + "'");
e.code = 'MODULE_NOT_FOUND';
throw e;
});
}
webpackEmptyAsyncContext.keys = () => ([]);
webpackEmptyAsyncContext.resolve = webpackEmptyAsyncContext;
webpackEmptyAsyncContext.id = "./. eager recursive \\.(?:test%7Cspec)s?\\.[^.]+$ exclude: (?:^%7C\\/).*\\/node_modules\\/.*$%7C(?:^%7C\\/).*\\/\\.meteor\\/.*$%7C(?:^%7C\\/).*\\/public\\/.*$%7C(?:^%7C\\/).*\\/private\\/.*$%7C(?:^%7C\\/).*\\/_build\\/.*$%7C(?:^%7C\\/).*\\/fakeServer\\/.*$%7C(?:^%7C\\/).*\\/app\\/android$%7C(?:^%7C\\/).*\\/app\\/ios$%7C(?:^%7C\\/).*\\/cloudslicer\\/.*$%7C(?:^%7C\\/).*\\/tools\\/.*$%7C(?:^%7C\\/).*\\/dataAnalyzer\\/.*$%7C(?:^%7C\\/).*\\/app\\/app\\/.*$%7C(?:^%7C\\/).*\\/app\\/e2e\\/.*$%7C(?:^%7C\\/).*\\/app\\/.*tests.*\\/.*$%7C(?:^%7C\\/).*\\/app\\/Assets\\/.*$%7C(?:^%7C\\/).*\\/app\\/node_modules$%7C(?:^%7C\\/).*\\/[^/]*\\.native\\.js$%7C(?:^%7C\\/).*\\/gen\\/.*$%7C(?:^%7C\\/).*\\/services\\/target$%7C(?:^%7C\\/).*\\/app\\/.*$%7C(?:^%7C\\/).*\\/client\\/.*$";
module.exports = webpackEmptyAsyncContext;
}),
});
// The module cache
var __webpack_module_cache__ = {};
// The require function
function __webpack_require__(moduleId) {
// Check if module is in cache
var cachedModule = __webpack_module_cache__[moduleId];
if (cachedModule !== undefined) {
return cachedModule.exports;
}
// Create a new module (and put it into the cache)
var module = (__webpack_module_cache__[moduleId] = {
exports: {}
});
// Execute the module function
__webpack_modules__[moduleId](module, module.exports, __webpack_require__);
// Return the exports of the module
return module.exports;
}
// webpack/runtime/has_own_property
(() => {
__webpack_require__.o = (obj, prop) => (Object.prototype.hasOwnProperty.call(obj, prop))
})();
// webpack/runtime/make_namespace_object
(() => {
// define __esModule on exports
__webpack_require__.r = (exports) => {
if(typeof Symbol !== 'undefined' && Symbol.toStringTag) {
Object.defineProperty(exports, Symbol.toStringTag, { value: 'Module' });
}
Object.defineProperty(exports, '__esModule', { value: true });
};
})();
var __webpack_exports__ = {};
// This entry needs to be wrapped in an IIFE because it needs to be in strict mode.
(() => {
"use strict";
/*!***************************************************!*\
!*** ./.meteor/local/test/server-eager-tests.mjs ***!
\***************************************************/
__webpack_require__.r(__webpack_exports__);
{
const ctx = __webpack_require__(/*! ../../.. */ "./. eager recursive \\.(?:test%7Cspec)s?\\.[^.]+$ exclude: (?:^%7C\\/).*\\/node_modules\\/.*$%7C(?:^%7C\\/).*\\/\\.meteor\\/.*$%7C(?:^%7C\\/).*\\/public\\/.*$%7C(?:^%7C\\/).*\\/private\\/.*$%7C(?:^%7C\\/).*\\/_build\\/.*$%7C(?:^%7C\\/).*\\/fakeServer\\/.*$%7C(?:^%7C\\/).*\\/app\\/android$%7C(?:^%7C\\/).*\\/app\\/ios$%7C(?:^%7C\\/).*\\/cloudslicer\\/.*$%7C(?:^%7C\\/).*\\/tools\\/.*$%7C(?:^%7C\\/).*\\/dataAnalyzer\\/.*$%7C(?:^%7C\\/).*\\/app\\/app\\/.*$%7C(?:^%7C\\/).*\\/app\\/e2e\\/.*$%7C(?:^%7C\\/).*\\/app\\/.*tests.*\\/.*$%7C(?:^%7C\\/).*\\/app\\/Assets\\/.*$%7C(?:^%7C\\/).*\\/app\\/node_modules$%7C(?:^%7C\\/).*\\/[^/]*\\.native\\.js$%7C(?:^%7C\\/).*\\/gen\\/.*$%7C(?:^%7C\\/).*\\/services\\/target$%7C(?:^%7C\\/).*\\/app\\/.*$%7C(?:^%7C\\/).*\\/client\\/.*$");
ctx.keys().forEach(ctx);
}
})();
module.exports = __webpack_exports__;
})()
;
Whereas on my local machine it has all the files loaded in there correctly. I have no idea why these two would be different other than its a Mac and Linux. I have verified the test files are there and it should be loading, but its just not.
Hey, thanks for the reproduction. Would you mind creating a similar version for Meteor 3.3.x?, just for the browser case.
There are a lot of dependencies and I’m having a hard time quickly downgrading it to compare what is expected and understand it better.
I’ve never used Meteor tests this way, it feels like you’re using them for E2E. I didn’t know this was compatible, I’ve always used Playwright/Cypress for that.
Also there is something else funky as my circle ci is unable to find any tests when its trying to run them, but locally its fine. I still need to investigate this issue, but something there is broken.
Weird, could you share the entire path on your CI env? Maybe they contain something is ignored.
At ./.meteor/local/test/* you can see the files that eagerly load modules. Like:
{
const ctx = import.meta.webpackContext('/', {
recursive: true,
regExp: /\.app-(?:test|spec)s?\.[^.]+$/,
exclude: /(?:^|\/).*\/node_modules\/.*$|(?:^|\/).*\/\.meteor\/.*$|(?:^|\/).*\/public\/.*$|(?:^|\/).*\/private\/.*$|(?:^|\/).*\/_build\/.*$|(?:^|\/).*\/ignore[^/]*$|(?:^|\/).*\/client\/.*$/,
mode: 'eager',
});
ctx.keys().forEach(ctx);
}
This is automatically generated, but maybe could it contain a pattern in exclude affecting your CI?
I am posting this for the message in red ```meteor npm install -D @swc…
I think this should be installed as a dependency and not a dev dependency.
My circle ci is at:
/home/circleci/app/
I do have /app/ in my .meteorignore but that should only be affecting folders called /app/ that are below the root. I would never expect moving my app to change the behavior of how it functions.
Here is the same app (minus the alias) that works just fine on Meteor 3.3
And the testing infrastructure is described here
This is similar to test mode, with key differences:
It loads test files matching *.app-test[s].* and *.app-spec[s].*.
It does eagerly load our application code as Meteor normally would.
Sets the Meteor.isAppTest flag to be true (instead of the Meteor.isTest flag).
This means that the entirety of your application (including for instance the web server and client side router) is loaded and will run as normal. This enables you to write much more complex integration tests and also load additional files for acceptance tests.
So I think the correct thing to do is, the client bundle is the normal client bundle with isAppTest = true
And the server bundle includes the server.mainEntry as well as all the .app-test. files
Edit:
I tried to remove the /app in meteorignore to see what happened and then I get
[Transpiler] Used SWC for ecmascript/ecmascript.js (package) 🔴 Cache miss (os.linux.x86_64)
Killed
Exited with code exit status 137
With no other information, what does the exit code 137 mean?
My project is a large one with Vue, Vuetify, vue-router, vuex. I will try to reproduce but without much hope.
Ok @nachocodoner my second little chat project, shaved off 115 kb of the client and I can see a clear drop in server memory usage. This … complicated the matter for me now :))) I will have to see what inflates the memory on my other project.
2 Likes
Thank you.
Your input has been determinant for me to understand the issue and provide a quick fix.
I will include this on the next RC version as well as all other last critical/quick fixes reported.
Regarding the CI, I’m not sure what the problem is. Could it be from the CI environment itself? Maybe a missing timeout or something similar. Exit code 137 usually means the OS/CI killed the process because it was using too many resources, like RAM. Maybe you need to tweak the limits.
I know that by removing the entry in .meteorignore it will load more things until it crashes, maybe. Or maybe running the tests eagerly requires more RAM. I don’t know for sure without a way to further debug that scenario. Could you try to run just few tests first by removing most of them, and incrementally enable them to see the crash point?