Meteor app in a monorepo with Yarn Workspaces ✨

announce
1

tl;dr

It can be done: GitHub - arggh/meteor-yarn-workspaces

I was always under the impression that Meteor doesn’t really play nice in monorepos. There’s been many posts on Github and this forum where people have trouble & requests for Meteor to support looking up node_modules higher up in the folder hierarchy to work with eg. Yarn Workspaces, such as

…where houshuang wrote:

The new Yarn workspaces feature to make it easier to work with monorepos relies on hoisting common npm packages to the parent directory. Apparently this is part of the npm specification, that it will resolve by looking into the current node_modules, but also in node_modules in the parent folder. This works with node, and Webpack. However, Meteor’s bundler only looks for modules in the current node_modules folder, and when it cannot find it, it errors out. (Version METEOR@1.6-beta.16).
We have our Meteor app organized as a mono-repo with several plugin packages that are required by the Meteor app. We used to use symlinking (which works well), but are looking at transitioning to the yarn workspace model (much faster install, less hand-written scripts).

…to which @benjamn replied:

Meteor does not, and will not ever, consider node_modules directories outside your application root directory, because that would require you to somehow recreate that surrounding environment on the server where you deploy your app. The app should be self-contained.

…Which was a bummer at the time.

Luckily, later Yarn has introduced a nohoist configuration option, which allows us to use Yarn Workspaces and benefit from hoisting, except for the Meteor apps, which still want their modules strictly inside app directory’s own node_modules. That’s why the nohoist option is needed.

And it works, no manual symlinking or scripts required.

I created this little demo repo to test different solutions for working with Meteor in monorepos, next up is Npm@7 and Yarn@2. Sadly Meteor doesn’t (yet :crossed_fingers: ) support pnpm at all.

Anyway, just thought I’d share.

11 Likes
2

@arggh this is fantastic. I’ve been struggling with Meteor and monorepos for a while. This 100% works.

  • app1 (meteor app - nohoist)
  • app2 (electron client app)
  • packages
    • package1 (shared package app1/app2)
    • package2 (shared package app1/app2)

Still get benifits of yarn workspace hoisting for all non meteor packages. Thanks for this super helpful post.

Edit: but for some reason I have to run yarn install twice in a row to get packages installed. The first time throws an error

An unexpected error occurred: "ENOENT: no such file or directory, lstat '.../node_modules/mkdirp'".

Then running yarn install straight after seems to work perfectly.

3

You’re most welcome!

I encountered the same issues (having to run install twice) until I had tediously cleaned up each and every accidentally overlooked "pkg1": "file: ../../packages/pkg1" notation from all the package.json's, replacing them with valid version numbers → Yarn requires you to use actual version numbers. I think you can also use * but according to some GitHub issues, doing so will/might cause problems.

Not sure if that’s the actual remedy, but for me the yarn install's have worked reliably now for some time.

Using pnpm workspaces would allow the use of workspace:*, but sadly, like I mentioned, Meteor doesn’t work at all with pnpm. There’s an issue for that though, let’s hope @filipenevola & others take notice :crossed_fingers:

1 Like
4

Rather than another workaround, I think I was able to come up with a proper solution by patching meteor’s code a little bit and teach it to handle symbolic links inside node_modules properly. A WIP version is available here, but it certainly requires some polishing:

There’s also a corresponding thread here:

1 Like
5

Hey @apendua, we at the team came by this thread and wanted to know if there was something still missing in your repo/pr, if there is anything you need help I would love to assist :slight_smile:

2 Likes
6

@grubba Thank you for reaching out!

One thing that I am not quite sure about is using MONOREPO_ROOT environment to control the newly implemented behavior:

Perhaps you will be able to suggest a better approach.

Other than this, I am quite happy with the solution and I can confirm that it works well with the monorepos that I’ve tested against. Having said that, I believe it may require some more intensive testing before we can call it “done”, which is the area in where I may need some additional help.

2 Likes
7

I will surely give it a try!
In my opinion, sounds good having a monorepo_root variable. Not ideal, but it makes sense for the symlinks. It should be working out of the box
Do you have any opinion on this, @zodern ? have you studied/seen ways of doing monorepos work in meteor?

8

@grubba FYI, Resolve symlinks by apendua · Pull Request #12383 · meteor/meteor · GitHub

2 Likes