Hello i want to know how the meteor compiler works
Step by step
You can always go through the code here: github.com/meteor/meteor/
Another good way of seeing what the compiler is doing is to run it in profile mode by setting the environment variable METEOR_PROFILE=0 (ie. profile all steps taking longer than 0ms). Though the output is extremely verbose.
On the project I’m currently working on it printed out 11,200 lines and a new project is 5,823 lines.
Most of it is just doing the same things while iterating over each package, so you can skim it to get a quick overview of how the compiler works.
Setting a larger number in the env var gives some nicer output, but with less detail:
2019-07-08 14:03:22 - 192-168-1-247 in ~/scratch/test
○ → METEOR_PROFILE=20 meteor
| (#1) Profiling: ProjectContext resolveConstraints
|
| ProjectContext resolveConstraints...............................816 ms (1)
| └─ _resolveConstraints..........................................812 ms (1)
| ├─ bundler.readJsImage.......................................178 ms (1)
| │ ├─ files.readFile 30 ms (56)
| │ └─ meteorNpm.rebuildIfNonPortable.........................138 ms (17)
| │ ├─ optimistic statOrNull 26 ms (200)
| │ └─ meteorNpm.isPortable................................104 ms (182)
| │ ├─ optimistic statOrNull 27 ms (182)
| │ └─ optimistic readJsonOrNull.........................56 ms (182)
| │ └─ optimistic readFile............................52 ms (182)
| │ └─ files.readFile 48 ms (182)
| └─ JsImage#load..............................................611 ms (1)
| ├─ runJavaScript packages/dynamic-import.js...............154 ms (1)
| │ └─ require("/node_modules/meteor/dynamic-import/server.js").152 ms (1)
| │ └─ require("./security.js") 150 ms (1)
| ├─ runJavaScript packages/ejson.js 30 ms (1)
| ├─ runJavaScript packages/ddp-client.js 33 ms (1)
| ├─ runJavaScript packages/npm-mongo.js....................121 ms (1)
| │ └─ Npm.require("mongodb") 121 ms (1)
| ├─ runJavaScript packages/logic-solver.js 32 ms (1)
| └─ runJavaScript packages/boilerplate-generator.js.........74 ms (1)
| └─ require("/node_modules/meteor/boilerplate-generator/generator.js").68 ms (1)
| └─ require("combined-stream2") 65 ms (1)
|
| Top leaves:
| require("./security.js")...................................150 ms (1)
|
| (#1) Total: 816 ms (ProjectContext resolveConstraints)
|
[[[[[ ~/scratch/test ]]]]]
=> Started proxy.
| (#2) Profiling: ProjectContext prepareProjectForBuild
|
| ProjectContext prepareProjectForBuild…801 ms (1)
| ├─ _downloadMissingPackages…37 ms (1)
| │ └─ Isopack.readMetadataFromDirectory 26 ms (78)
| └─ _buildLocalPackages…764 ms (1)
| ├─ _ensurePackageLoaded(allow-deny)…354 ms (1)
| │ ├─ _ensurePackageLoaded(meteor)…20 ms (1)
| │ │ └─ IsopackCache Load local isopack 20 ms (1)
| │ ├─ _ensurePackageLoaded(ecmascript)…237 ms (1)
| │ │ ├─ IsopackCache Load local isopack…104 ms (1)
| │ │ │ └─ Isopack#initFromPath…103 ms (1)
| │ │ │ └─ bundler.readJsImage…101 ms (1)
| │ │ │ └─ meteorNpm.rebuildIfNonPortable…96 ms (7)
| │ │ │ └─ meteorNpm.isPortable…72 ms (153)
| │ │ │ └─ optimistic readJsonOrNull…42 ms (153)
| │ │ │ └─ optimistic readFile…39 ms (153)
| │ │ │ └─ files.readFile 37 ms (153)
| │ │ └─ _ensurePackageLoaded(babel-compiler)…120 ms (1)
| │ │ └─ IsopackCache Load local isopack…102 ms (1)
| │ │ └─ Isopack#initFromPath…101 ms (1)
| │ │ └─ meteorNpm.rebuildIfNonPortable…99 ms (1)
| │ │ ├─ optimistic statOrNull 30 ms (145)
| │ │ └─ meteorNpm.isPortable…64 ms (143)
| │ │ └─ optimistic readJsonOrNull…40 ms (143)
| │ │ └─ optimistic readFile…37 ms (143)
| │ │ └─ files.readFile 36 ms (143)
| │ └─ _ensurePackageLoaded(ddp)…80 ms (1)
| │ └─ _ensurePackageLoaded(ddp-server)…63 ms (1)
| │ └─ _ensurePackageLoaded(webapp)…54 ms (1)
| │ └─ IsopackCache Load local isopack…45 ms (1)
| │ └─ Isopack#initFromPath…45 ms (1)
| │ └─ meteorNpm.rebuildIfNonPortable…34 ms (1)
| │ └─ meteorNpm.isPortable 24 ms (47)
| ├─ _ensurePackageLoaded(blaze-html-templates)…91 ms (1)
| │ └─ _ensurePackageLoaded(templating)…89 ms (1)
| │ └─ _ensurePackageLoaded(templating-runtime)…88 ms (1)
| │ └─ _ensurePackageLoaded(templating-compiler)…86 ms (1)
| │ └─ IsopackCache Load local isopack…86 ms (1)
| │ └─ Isopack#initFromPath…86 ms (1)
| │ └─ bundler.readJsImage…84 ms (1)
| │ └─ meteorNpm.rebuildIfNonPortable…74 ms (9)
| │ └─ meteorNpm.isPortable 50 ms (178)
| ├─ _ensurePackageLoaded(minifier-css)…75 ms (1)
| │ └─ IsopackCache Load local isopack…75 ms (1)
| │ └─ Isopack#initFromPath…75 ms (1)
| │ └─ meteorNpm.rebuildIfNonPortable…68 ms (1)
| │ └─ meteorNpm.isPortable…49 ms (131)
| │ └─ optimistic readJsonOrNull…34 ms (131)
| │ └─ optimistic readFile…32 ms (131)
| │ └─ files.readFile 31 ms (131)
| ├─ _ensurePackageLoaded(mongo) 31 ms (1)
| ├─ _ensurePackageLoaded(standard-minifier-css)…76 ms (1)
| │ └─ IsopackCache Load local isopack…76 ms (1)
| │ └─ Isopack#initFromPath…76 ms (1)
| │ └─ bundler.readJsImage…74 ms (1)
| │ └─ meteorNpm.rebuildIfNonPortable…68 ms (12)
| │ └─ meteorNpm.isPortable…49 ms (295)
| │ └─ optimistic readJsonOrNull 23 ms (295)
| └─ _ensurePackageLoaded(standard-minifier-js)…87 ms (1)
| └─ IsopackCache Load local isopack…87 ms (1)
| └─ Isopack#initFromPath…86 ms (1)
| └─ bundler.readJsImage…85 ms (1)
| └─ meteorNpm.rebuildIfNonPortable…79 ms (12)
| └─ meteorNpm.isPortable…60 ms (165)
| └─ optimistic readJsonOrNull…39 ms (165)
| └─ optimistic readFile…37 ms (165)
| └─ files.readFile 36 ms (165)
|
| Top leaves:
| files.readFile…248 ms (1694)
|
| (#2) Total: 801 ms (ProjectContext prepareProjectForBuild)
|
| (#3) Profiling: Build App
|
| files.stat 0 ms (1)
| files.unlink 0 ms (2)
| files.readFile 2 ms (1)
| files.readdir 0 ms (1)
| Build App…3,001 ms (1)
| └─ bundler.bundle…3,001 ms (1)
| └─ files.withCache…3,001 ms (1)
| ├─ compiler.compile(the app)…503 ms (1)
| │ └─ files.withCache…502 ms (3)
| │ └─ compileUnibuild (the app)…502 ms (3)
| │ ├─ Isopack#ensurePluginsInitialized…467 ms (36)
| │ │ └─ JsImage#load…465 ms (5)
| │ │ ├─ runJavaScript packages/ecmascript-runtime-server.js.74 ms (4)
| │ │ │ ├─ require("/node_modules/meteor/ecmascript-runtime-server/runtime.js").41 ms (3)
| │ │ │ │ └─ require(“core-js/modules/es7.string.pad-start”) 38 ms (3)
| │ │ │ └─ require("./node_modules/meteor/ecmascript-runtime-server/runtime.js") 31 ms (1)
| │ │ ├─ runJavaScript packages/babel-compiler.js 26 ms (4)
| │ │ ├─ runJavaScript packages/dynamic-import.js.39 ms (2)
| │ │ │ └─ require("/node_modules/meteor/dynamic-import/server.js").38 ms (2)
| │ │ │ └─ require("./security.js") 35 ms (2)
| │ │ ├─ runJavaScript packages/minifier-css.js…103 ms (1)
| │ │ │ └─ require("/node_modules/meteor/minifier-css/minifier.js").90 ms (1)
| │ │ │ ├─ require(“postcss”) 38 ms (1)
| │ │ │ └─ require(“cssnano”) 50 ms (1)
| │ │ ├─ runJavaScript packages/caching-compiler.js.44 ms (1)
| │ │ │ └─ require("./node_modules/meteor/caching-compiler/caching-compiler.js").30 ms (1)
| │ │ │ └─ require(“babel-runtime/helpers/possibleConstructorReturn”).22 ms (1)
| │ │ │ └─ Npm.require(“babel-runtime/helpers/possibleConstructorReturn”) 22 ms (1)
| │ │ └─ runJavaScript packages/spacebars-compiler.js.32 ms (1)
| │ │ └─ Npm.require(“uglify-js”) 30 ms (1)
| │ └─ PackageSource#_findSources…24 ms (3)
| │ └─ files.withCache 23 ms (3)
| ├─ bundler.bundle…makeClientTarget…1,582 ms (2)
| │ └─ Target#make…1,582 ms (2)
| │ ├─ Target#_runCompilerPlugins 32 ms (2)
| │ └─ Target#_emitResources…1,503 ms (2)
| │ ├─ PackageSourceBatch.computeJsOutputFilesMap…1,403 ms (2)
| │ │ ├─ ImportScanner#_realPath…41 ms (268)
| │ │ │ └─ optimistic lstatOrNull…36 ms (298)
| │ │ │ └─ optimistic lstat 25 ms (246)
| │ │ ├─ ImportScanner#scanImports for modules…68 ms (2)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 53 ms (18)
| │ │ ├─ ImportScanner#scanImports for dynamic-import 22 ms (2)
| │ │ ├─ ImportScanner#scanImports for es5-shim…38 ms (2)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 35 ms (6)
| │ │ ├─ ImportScanner#scanImports for ecmascript-runtime-client.303 ms (2)
| │ │ │ ├─ ImportScanner#_findImportedModuleIdentifiers 151 ms (300)
| │ │ │ ├─ ImportScanner#_resolve…88 ms (950)
| │ │ │ │ ├─ optimistic statOrNull…58 ms (1378)
| │ │ │ │ │ ├─ files.stat 24 ms (526)
| │ │ │ │ │ └─ other optimistic statOrNull 33 ms
| │ │ │ │ └─ other ImportScanner#_resolve 28 ms
| │ │ │ ├─ ImportScanner#_readFile…35 ms (297)
| │ │ │ │ └─ optimistic readFile 21 ms (297)
| │ │ │ └─ other ImportScanner#scanImports for ecmascript-runtime-client 20 ms
| │ │ ├─ ImportScanner#scanImports for base64 158 ms (2)
| │ │ ├─ ImportScanner#scanImports for ejson…35 ms (2)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 29 ms (4)
| │ │ ├─ ImportScanner#scanImports for minimongo…153 ms (2)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 132 ms (16)
| │ │ ├─ ImportScanner#scanImports for check…27 ms (2)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 21 ms (4)
| │ │ ├─ ImportScanner#scanImports for socket-stream-client.24 ms (2)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 20 ms (1)
| │ │ ├─ ImportScanner#scanImports for ddp-client…90 ms (2)
| │ │ │ ├─ ImportScanner#_findImportedModuleIdentifiers 48 ms (10)
| │ │ │ └─ ImportScanner#_resolve 30 ms (47)
| │ │ ├─ ImportScanner#scanImports for mongo 21 ms (2)
| │ │ ├─ ImportScanner#scanImports for jquery…160 ms (2)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 159 ms (4)
| │ │ ├─ ImportScanner#scanMissingModules for the app 23 ms (4)
| │ │ └─ ImportScanner#scanMissingModules for socket-stream-client.71 ms (2)
| │ │ └─ ImportScanner#_findImportedModuleIdentifiers 52 ms (7)
| │ └─ PackageSourceBatch#getResources…95 ms (122)
| │ └─ PackageSourceBatch#_linkJS…94 ms (122)
| │ ├─ optimistic readJsonOrNull 53 ms (122)
| │ └─ linker.fullLink 22 ms (1)
| ├─ bundler.bundle…makeServerTarget…528 ms (1)
| │ └─ Target#make…528 ms (1)
| │ └─ Target#_emitResources…500 ms (1)
| │ ├─ PackageSourceBatch.computeJsOutputFilesMap…467 ms (1)
| │ │ ├─ ImportScanner#_realPath…32 ms (163)
| │ │ │ └─ optimistic lstatOrNull 24 ms (175)
| │ │ ├─ ImportScanner#scanImports for base64 46 ms (1)
| │ │ ├─ ImportScanner#scanImports for minimongo…57 ms (1)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 47 ms (8)
| │ │ ├─ ImportScanner#scanImports for ddp-client 25 ms (1)
| │ │ ├─ ImportScanner#scanImports for webapp 35 ms (1)
| │ │ ├─ ImportScanner#scanImports for ddp-server…29 ms (1)
| │ │ │ └─ ImportScanner#_findImportedModuleIdentifiers 25 ms (5)
| │ │ └─ ImportScanner#scanImports for mongo…57 ms (1)
| │ │ └─ ImportScanner#_findImportedModuleIdentifiers 44 ms (10)
| │ └─ PackageSourceBatch#getResources…31 ms (70)
| │ └─ PackageSourceBatch#_linkJS…31 ms (70)
| │ └─ optimistic readJsonOrNull 23 ms (70)
| └─ bundler writeSiteArchive…378 ms (1)
| ├─ bundler writeTargetToPath…324 ms (3)
| │ ├─ ClientTarget#write…75 ms (2)
| │ │ ├─ bundler writeFile…31 ms (126)
| │ │ │ └─ Builder#write…27 ms (126)
| │ │ │ └─ files.writeFile 23 ms (126)
| │ │ └─ other ClientTarget#write 30 ms
| │ └─ ServerTarget#write…246 ms (1)
| │ └─ JsImage#write…237 ms (1)
| │ ├─ Builder#write…29 ms (101)
| │ │ └─ files.writeFile 21 ms (101)
| │ ├─ Builder#copyNodeModulesDirectory…102 ms (21)
| │ │ └─ files.symlinkWithOverwrite…25 ms (252)
| │ │ └─ files.symlink 20 ms (252)
| │ ├─ optimistic statOrNull 47 ms (584)
| │ ├─ optimistic readJsonOrNull 21 ms (76)
| │ └─ other JsImage#write 38 ms
| └─ other bundler writeSiteArchive 50 ms
|
| Top leaves:
| files.stat…140 ms (3330)
|
| (#3) Total: 3,004 ms (Build App)
2 Likes