Optimize package bundler errors

core
1

Hello everyone,

I sometimes run into errors like this:

packages/core-runtime.js:189
            throw error;
            ^

TypeError: Cannot read properties of undefined (reading 'public')
    at module (packages/modules.js:15208:31)
    at fileEvaluate (packages/modules-runtime.js:335:7)
    at Module.require (packages/modules-runtime.js:237:14)
    at Module.mod.require (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:30:33)
    at Module.moduleLink [as link] (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:102:22)
    at module (packages/modules.js:3054:829)
    at fileEvaluate (packages/modules-runtime.js:335:7)
    at Module.require (packages/modules-runtime.js:237:14)
    at Module.mod.require (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:30:33)
    at Module.moduleLink [as link] (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:102:22)
    at module (packages/modules.js:13004:511)
    at fileEvaluate (packages/modules-runtime.js:335:7)
    at Module.require (packages/modules-runtime.js:237:14)
    at Module.mod.require (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:30:33)
    at Module.moduleLink [as link] (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:102:22)
    at module (packages/modules.js:12875:1551)
    at fileEvaluate (packages/modules-runtime.js:335:7)
    at Module.require (packages/modules-runtime.js:237:14)
    at Module.mod.require (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:30:33)
    at Module.moduleLink [as link] (/Users/patrick/.meteor/packages/modules/.0.20.2.1oomq3o.ya6e++os+web.browser+web.browser.legacy+web.cordova/npm/node_modules/@meteorjs/reify/lib/runtime/index.js:102:22)
    at module (packages/modules.js:13257:510)
    at fileEvaluate (packages/modules-runtime.js:335:7)

Node.js v20.18.0
=> Exited with code: 1

I am aware that I am causing this error, but this is just an example.

The issue is: The debugging is incredibly hard, because I have zero information where to look - not even in which npm module, folder, file, anything …

Can this be improved somehow? Deadling with errors like this is sooo much wasted time.

Thanks, best, Patrick

2 Likes
2

Any input on this one from the Meteor Core team? :slight_smile:

3

Hey @klabauter sorry for the delay, sometimes it’s hard to keep up; it’s something that bothers me too, we will research how we can improve it and keep you posted on it. Perhaps source maps could help here, we will check if that would solve it and prepare a simple guide for it.

4

The modules.js file contains many of the modules from npm packages that the app imports. Normally you would use source maps to have a better error. However, generating source maps can be slow and considering the size of modules.js, Meteor never has generated a source map in order to have faster builds.

If you use my minifier it generates source maps for modules.js. There’s some rare edge cases where it might not be as accurate as if Meteor created them, but otherwise it should work well and has minimal impact on performance.

The part of Meteor that combines the individual files into a single file is called the linker. Snowpack, and later vite, removed the linking step for faster rebuilds. However, there are benefits with linking, and some newer bundlers chose to implement it while being as fast as vite.

Last year I started rewriting Meteor’s linker to make it faster and simpler, and address some issues such as missing source maps for modules.js. As part of this, I discovered even the faster source map libraries other bundlers created were too slow for the desired performance, so I finished writing my own library (which is also used by my minifier).

Source maps contain a list of mappings, that say this line and column in the output file was originally from this line and column from this input file. To reduce the size of the mappings, they are encoded as relative values - instead of an absolute line or column, it’s the offset from the previous mapping. Source map libraries generally decode the mappings into absolute locations, store them, and then encode the mappings again. In the context of what the linker does, most of that work is unnecessary so my library works directly with the encoded values as much as possible.

By using the fact that the mappings are encoded as relative positions, we can very efficiently generate mappings for files in modules.js. This makes having source maps for modules.js almost free.

The linker rewrite was part 1 of a 3 step plan to fully support top level await, address some of the root causes behind slow builds, and have a better architecture to add other features like tree shaking. I took a break from contributing to Meteor after they stopped sponsoring me. A little later they indicated the plan was to have Meteor switch to a different bundler, so now there doesn’t seem to be any reason to resume work on it.

The linker rewrite is here. Many additional optimizations, and the switch to my source map library hasn’t been pushed to github.

The plan was:

  1. Step 1 - rewrite the linker to remove the technical debt added from top level await, always generate code compatible with top level await, make it fast enough to remove the need for caching in the linker, and to always have source maps
  2. Step 2 - create a new caching system that can be used by the linker and import scanner, and remove the linker disk cache. This would remove many performance issues Meteor currently has. One of the goals is to replace the LRU caches with a different algorithm, or create a system that tracks which cache entries are used in each architecture and automatically cleans up.
  3. Step 3 - rearchitect the import scanner to work with module graphs and have better performance. This would reduce the effect top level await would have on the client bundle size to the same as it would be with webpack, and make it easier to implement tree shaking and other features without significantly affecting performance.
2 Likes