Optimize CI for Meteor

announce
1

I wrote a blog post on Optimizing CI for Meteor. It shows what should be cached, what cache keys to use, and other optimizations to speed up installing dependencies and building Meteor. For medium size projects these can save up to a few minutes each time the CI job is run.

30 Likes
2

I really like the part about remove-client-js :+1:

1 Like
3

Nice post! Perhaps it would be interesting for people to post their app build times?

Ours on Github Actions:

meteorengineer/setup-meteor@v1                22 s
installing node modules                       37 s
Build meteor                                  11m 7s
1 Like
4

i read your blog and this is really good…

5

I applied most of the advice provided in your blog post to this GitHub workflow. I managed to reduce the time of the test job from roughly 3 minutes to roughly 2 minutes. Took me a while to get it right…

1 Like
7

I just published a (composite) GitHub action at

that you can easily reuse with - uses: meteor-actions/install@v1. Note that the use of if inside a composite action has been recently made possible by https://github.com/actions/runner/issues/834. I am open to improvements. Maybe split the different layers of caching into multiples actions?

@zodern I have a question regarding caching the artifacts of meteor build ../dist inside a GitHub workflow: What’s the proper way to do it? By that I mean caching so that next builds execute faster.

PS, @zodern context: I’d like to cook up something similar to the new push-to-deploy Meteor Cloud feature via meteor-up.

2 Likes
8

I am afraid, those github actions will not be supported in the future, because they use

curl https://install.meteor.com/ | sh

Which is not maintained anymore (according to According to docs/source/install.md):

See also: Why is curl https://install.meteor.com/ | sh not maintained anymore

1 Like
9

The above comment from @scharf also breaks any older versions of Meteor.

For example v1.8.1 is now dead because it uses the same server.

I really find it strange the there hasn’t been any marketing on this from the Meteor team. No mention of deprecation for older version.

Seem to me like the team did a major legacy breaking change and didn’t talk about it enough. Happy to be proven otherwise.

10

Hello! Meteor hasn’t removed or deprecated any legacy version from our servers. You might be referring to let’s encrypt certificate change referred here Expired Certificates | Meteor API Docs which is not a change introduced by Meteor Team.

I would also like to highlight that even if we are “deprecating” the old install method, it will still be online and you can use it as you please, as we don’t have any plans to remove it. It’s just that using NPM is better for newcomers and has a lot of advantages in the question of approaching meteor to the node ecosystem.

You can still use the install.meteor script in your CI, without issues, as it is feature-complete in our point of view, and won’t be removed in the foreseeable future.

1 Like
11

@renanccastro Thanks for the reply! I’ll look into that.

12

The action looks nice. Handling everything adds a lot of complexity, so having a reusable action for that would be helpful. Once it is finished, I could mention it in the blog post.

If the previous build exists, meteor build deletes it before building. I am not aware of any additional caching that would help with meteor build.

Something new since I wrote the article that could be tried is setting the env var METEOR_TOOL_ENABLE_REIFY_RUNTIME_CACHE to true. It makes loading build plugins up to a few seconds faster. It is disabled by default since I was worried about issues when the reify cache becomes too large, but in CI that won’t be an issue. The reify cache files are stored in ~/.meteor.

1 Like
13

I published v2 of meteor-actions/install. Some notable changes:

  • All environment variables that were defined before have changed names (check the sources).
  • meteor-actions/install@v2 allows to specify an executable_version input to select the version of the meteor executable that will be installed (this can be the same as the version in .meteor/release, or it can be different). Either specify a version, or latest for the latest Meteor release, or local to use the same version as defined in .meteor/release. The default is local.
  • meteor-actions/install@v2 does not cache .meteor/local anymore: there is a new meteor-actions/cache-build@v2 that does cache .meteor/local and accepts an optional key input to avoid distinct builds to overwrite each-other’s caches.

You should now be able to exploit these actions on parallel jobs and workflows. For instance with meteortesting:mocha you can run TEST_CLIENT=0, TEST_CLIENT=0 --full-app, TEST_SERVER=0, and TEST_SERVER=0 --full-app in parallel. They will share the cache for the Meteor executable installation. They will not share the cache for the .meteor/local folder (unless you really want to) but should be able to restore the cache from each other in case of a cache-miss.

4 Likes
14

You should make this announcement in a new thread with all the fanfare it deserves. :tada:

1 Like
15

@storyteller Thanks! Before doing that I would like to figure out the proper way to order parts of the cache key of meteor-actions/cache-build. Here is the current state:

I think the current key is bad. The commit hash should be in there but I am not sure where. The custom key part should probably have precedence. Someone help? @zodern ?

16

I was doubtful about cache-key containing commit hash in Meteor Build Cache. Specially when no restore key have been mentioned in the blog. Does it mean on every run nothing will be restored from cache (as it makes sense to run a build only when there is a commit, which changes cache key).

Should there restore key of v1-meteor_build_cache- or I am missing something? @zodern

17

The restore keys would be:

v1-meteor_build_cache-${{ github.ref }}-${{ github.sha }}
v1-meteor_build_cache-${{ github.ref }}
v1-meteor_build_cache-

This way it would prefer the cache for the same commit, then from the same branch if available. It seems I forgot to include any explanation of the restore keys in the article.