Modernizing Meteor's Bundler

Wouldn’t that mean that we need to still maintain Meteor bundler and build tool for packages though?

2 Likes

Is there much difference between DDP and HTTP for this particular use-case?

1 Like

Very nice plans. One thing to keep in mind here is that, as far as I can tell, choosing Vite would also make the Vue-Meteor integration a lot more straightforward and easier to maintain. For example, in the current Vue 2/Meteor integration about 25% (rough ballpark, citing from memory here) lines of code implement the tracker and around 75% deal with making things work with the bundler. For Vue 3, all of the latter could be left aside it seems as it already works well with Vite. The same could probably be said for other view libraries as well.

Take my comment with a grain of salt, though. I have not looked into this topic very deeply. So there might be surprises. But it should not be very much work to go over the most popular view libraries and see how well do they work with Vite out of the box.

2 Likes

I’m not opposed to updating to a modern bundler, if we can add everything back in (isomorphism, reactivity, data management, etc).

We don’t really use code splitting or lazy imports all that much; and to be honest, I have a webkit template that I’ll sometimes build UI interface in because it’s so much lighter.

But there is one feature/issue that would be great to get fixed:

When compiling React components in Atmosphere packages, the bundler will create some sort of root Javascript object that it attaches all the subsequent components to; and when it rebuilds the code in the final codebase, it attaches them via a React component instead of a React function; which breaks the Rule of Hooks and all downstream pure functions.

What winds up happening, is that you can’t pass a prop or children elements from a root (parent) component defined in the main app into a pure functional component defined in an Atmosphere package. There’s a sort of cleaving, where all the React functional packages defined in Atmosphere packages get attached to a React component, breaking props passing through the functions from App code into Atmosphere code.

That has been my single biggest headache with the bundler for the past 4 or 5 years.

That being said, if we adopt a new bundler, I’m sure it will introduce new headaches.

2 Likes

Indeed, Meteor bundler remains responsible for compiling meteor/* packages and integrating them into the compiled bundle through the modern bundler process.

Additionally, Meteor would focus on minimizing core and package sizes to reduce the basic bundle weight. I have seen already a lot of work around this reduction or to remove stale libraries used on core. Anyway, a migration to npm where possible would enable tree shaking reductions at the app level.

2 Likes

DDP

The DDP approach in Meteor creates dynamic modules per submodule path from the module used in the import() statement. These modules are served on-the-fly as strings in response to client requests, providing advantages like live updates and efficient handling of increased dynamic imports.

The downside is that it relies on the DDP responses, requiring a server connection for proper serving. This makes offline support for dynamic bundles impossible and lacks built-in support to include these dynamic bundles in the native compilation.

The next picture illustrates a response in the simpletasks app of a Meteor dynamic import when loading the tasks page.

HTTP

HTTP is the main communication protocol for the web. Webpack and Vite compile dynamic modules into single files or can adjust granularity via configuration, matching similarly as Meteor’s behavior likely. Direct file generation enables inclusion in native compilation, supporting offline availability, using service worker caching for faster module loading, CDN, and all tools and mechanisms implemented on the top of HTTP.

Opting for a standard approach used widely in various companies and projects globally minimize any disadvantages. The reactive aspect in Meteor may be the only drawback I can think of, but it doesn’t seem to me a life-changing implementationat all.

The next picture illustrates a response in the simpletasks-webpack app of a standard dynamic import when loading the tasks page.

2 Likes

It’s important to support any view library embraced by Meteor developers. Although I lack experience with Vue, I’m up to examine a basic app and explore the approach in both Webpack and Vite, gaining a better understanding of your concerns. A quick search showed me an existing guide on supporting Vue 3 in webpack with loaders.

Any recommended Meteor-Vue projects for evaluating modern bundler support?

Anyway, I envision the modern adoption process into Meteor as flexible enough to switch from one to another through a configuration field in the meteor.config.js file, allowing users to choose their preferred bundler even. However, Meteor should prioritize one and consider expanding or replacing it when a switch becomes beneficial or to cover more scenarios as the one you mention with Vue if not possible with the other.

1 Like

I’m not opposed to updating to a modern bundler, if we can add everything back in (isomorphism, reactivity, data management, etc).

That is surely perserved by delegating the compilation of Meteor specific characteristics to the Meteor bundler itself.

We don’t really use code splitting or lazy imports all that much; and to be honest, I have a webkit template that I’ll sometimes build UI interface in because it’s so much lighter.

There are large projects urgently require bundler reduction. Standard implementations with modern bundlers are a clear win here.

When compiling React components in Atmosphere packages, the bundler will create some sort of root Javascript object that it attaches all the subsequent components to

Atmosphere packages may encounter issues, making users to use direct file references rather than relying on packages. I have experienced this on the past actually.

That being said, if we adopt a new bundler, I’m sure it will introduce new headaches.

I know. I would say here that any thing related with our profession brings up headaches :rofl: We are problem-solvers by nature.

Switching to a modern bundler expand our knowledge base by going into a larger pool of company and developer experiences. Meteor’s current issues often require a few experienced developers to navigate through outdated, undocumented procedures, as well as reimplementing things that are largerly managed outside our smaller community.

I wouldn’t be afraid to embrace the new when the community claims for modern procedures, rather than persisting in a struggle with the old.

4 Likes

Good take.

Regarding options, does the developer have the option to opt out of using a modern bundler and stick with just the Meteor bundler?

2 Likes

Definitely, the meteor.config.js should extend configuration for both modern and meteor config. Consider adding a bundler field to choose between them to build the app code, favoring the modern by default.

2 Likes

Until now, dynamic imports cannot be piped through a CDN, making it dependent on main app server bandwidth (slow and expensive).

3 Likes

Both Vue and Svelte have standardized on Vite. For Vue 3, their tooling guide recommends using Vite over webpack, which was used for Vue 2 previously. And Svelte’s docs recommend SvelteKit, which is also based on Vite. So if Meteor would choose webpack, then the tradeoffs for this should really be determined before making the decision.

For Vue and perhaps other view libraries as well, I think it would be best to have a quick chat with @akryum who has maintained the Vue 2 integration with Meteor and is a member of the core Vue team. And maybe also with the author of this repo. I would assume they’d be able to lay out the pros and cons of both choices pretty well as far as view libraries are concerned.

But in any case moving to a widely used bundler would be a very good step towards aligning Meteor with the general JS ecosystem again. Instead of reinventing the wheel (view libraries, bundlers) it would be much better for Meteor to just use the industry standard and focus its own efforts on just what makes it special (pub/sub, methods, accounts system etc).

I looked it up. To currently use Vue 2 with Meteor, you need the the following two packages:

The first one makes the tracker work in Vue components (so gets you the special stuff of Meteor, like making subscriptions etc) and the second one integrates files with the bundler. The first is only around 250 lines of code, while the latter is around 10x more. Ideally Meteor whould be in a place where the bundler issue is already taken care of, since an industry standard bundler is used, and so integrating a new view library with Meteor would only be a 250 lines of code type of affair. Easy to create and easy to maintain. One can hope :slight_smile:

5 Likes

Meteor switching from Isobuild to Vite would be a clear win for compatibility with the rest of the ecosystem and would make Frontend frameworks integration very easy.

8 Likes

Would you be able to provide some thoughts on webpack vs Vite vs SvelteKit? I’m not familiar with any of these beyond the superficial so it would be great to have more input on it.

Also if I remember correctly Vite already has Meteor integration?

Integrating a modern bundler into the Meteor Core is what we need. However, the challenge is how we do.

This post outlines an approach that incorporates sufficient compiler features for any React project. It’s important to recognize that different view libraries have unique needs. I also realized on my tests now that Meteor HMR is no longer functional with this approach. I need to research better ways to solve this and enhance this aspect using my approach.

Thank you for the reference to the meteor-vite tool, which seems promising for our objectives and ensures backwards compatibility with Meteor specific build features. Definitely more complex than my proposal, but the experience is smooth and without sacrificing anything apparently. I’ll invest time in comprehending the proposed solution here, its adventages and limitations.

6 Likes

Generally, for development, I would say Vite is considerably faster, less overwhelming to configure and set up compared to webpack. And unless I’m mistaken, Vite is starting to look like the standard tool these days.

Internally Vite uses Rollup for bundling your app which is what’s comparable to webpack. Vite exposes a subset of Rollup’s configuration options for those that need it, which in most cases ends up being plugin authors, not end users. Whereas webpack again feels more involved and slower to use.

I really couldn’t agree more with Akryum. Transitioning to Vite would most certainly be a huge win for Meteor. Just the HMR alone would make it worthwhile in my eyes. And I think anyone who’ve tried Akryum’s Meteor-Vite plugin would agree. Not to mention the kind of improvement we could see to Meteor’s build times.

Working with older view frameworks, like Vue 2 for example is also a lot nicer compared to the old Meteor ecosystem packages. This is the main reason why I got into maintaining Akryum’s Meteor-Vite package. The boost in development productivity and flexibility has just been incredible.

9 Likes

And I have to agree with Nacho here. An integration into Meteor Core would be very beneficial, and have much less risk of affecting backwards compatibility while providing a better development experience. Definitely a step in the right direction.

But I would love to see this taken further. Right now, while we’re able to provide a great front-end development experience with the Meteor-Vite plugin. Once time comes to bundle for production, build times are still held back by the Meteor bundler bundling its own packages.

That being said - one thing at a time. I think the changes proposed by Nacho is going to provide a significant improvement to the Meteor development workflow. The production build times isn’t that important to be fair, as long as the development experience is good.

6 Likes

@jorgenvatle: It is really nice to have you in this dicussion, your experience and feedback is crucial for us to understand better the benefits of the build tools, but not only in the theorical side, your practical work on meteor-vite is the added value of your input. Similarly goes for @akryum.

Thank you for the quick overview you provided me on meteor-vite. I look forward to adapting the simpletasks example app using meteor-vite and making an analysis based on my simpler approach. This involves tree shaking, ESM support, measuring bundle size gains, testing rollup-plugins, exploring dynamic and lazy imports (Meteor) vs standard imports (Vite), and debugging the packages/plugins in the library you are working on.

I agree on your comments about Vite provinding a better experience, particularly the impressive HMR gains. In my previous attempts, Webpack provided better results in this aspect. However, meteor-vite addresses these issues by enabling the vite-dev server to smartly handle the bundle process and ensure HMR working.

Regarding production builds, while they may require more time due to additional processes, I believe it’s reasonable to expect this, especially initially. As you mentioned, taking one step at a time, prioritizing dev experience, and maintaining backward compatibility with Meteor are key aspects that I believe everyone in this discussion agrees on. With the main objective for Meteor to make a step forward on modernize its building capabilities.

7 Likes

Slighty related to the vite I would also want to express my love for rollup. I use it for all my npm packages in order to build multiple formats from my source:

  • esm, cjs, iife, cjs+es5
  • normal-size, minified
  • sourcemaps for each

It’s fun to work with, config is comprehensible, errors are usually understandable, upgrade minor versions has never broken a thing and plugins work very smooth.

6 Likes

Very interesting thread and tons of information!

I’m wondering if there is a possibility of using pnpm instead of npm speaking about the DX I like the pnpm speed and the way it handles your local disk space. Curious how you @nachocodoner thought about this?