Experiences with Dynamic Import so far

I refactored my app a bit to see if I could get the code splitting or dyanamic import working. I did get it to load, with some caveats, but I’m not sure it’s actually splitting anything, and I’m not sure how to check. Is there a tool to generate a graph of some kind?)

The caveats:

  • I had to disable audit-argument-checks as whatever methods are sending out the code modules has arguments which are not being checked using the check method (according to audit-argument-checks).

Exception while invoking method '__dynamicImport' Error: Did not check() all arguments during call to '__dynamicImport'

  • I had to disable browser-policy, as the code is being delivered for dynamically imported modules is executing inside an eval and “unsafe-inline” is disallowed by browser-policy by default.

Refused to load the stylesheet 'https://fonts.googleapis.com/css?family=PT+Sans+Narrow:400,700' because it violates the following Content Security Policy directive: "style-src 'self' 'unsafe-inline'".

Uncaught (in promise) EvalError: Refused to evaluate a string as JavaScript because 'unsafe-eval' is not an allowed source of script in the following Content Security Policy directive: "script-src 'self' 'unsafe-inline'".

  • I can’t get the await method of using import to work at all - it tells me await is a reserved word. I don’t have much going on in babel - here is my .babelrc file contents:
{
  "plugins": ["transform-class-properties"]
}

Removing .babelrc didn’t have any effect, except breaking the code I have which relies on that plugin.

I’m also curious if there is a better way to import multiple modules - I suppose I could stuff all the code into it’s own module file (actually that’s probably what I’ll do instead of) but this is what I have so far:


function handleDeckRoute (params) {
  var React, Provider, AppContainer
  import('react').then(_React => {
    React = _React
    return import('react-redux')
  })
  .then(m => {
    // using `module` as the arg id breaks the following import statement
    Provider = m.Provider
    return import('/imports/containers/AppContainer')
  })
  .then(m => {
    AppContainer = m.default
    return import('react-mounter')
  })
  .then(m => {
    const {mount, withOptions} = m
    const mountWithOptions = withOptions({ rootId: 'r-mount' }, mount)
    mountWithOptions(Provider, {
      store, children: <AppContainer />
    })
    routeDispatcher(params)
  })
}
  • Which brings me to that comment above - I originally used module as the argument identifier, but that broke the following import statement in mysterious ways. I guessed that was the problem, and changed it to m instead, and it fixed it. If there is some better way to write the above, I’d love to see it! (Like I said, I’ll probably migrate that into a separate file, and just use static imports, and import the one whole file that way.)

How can I get a list of which modules are being bundled vs loading dynamically to make sure I’m not accidentally bundling everything in the initial payload (which is what it looks like I’m doing now)?

3 Likes

I think this is all great feedback, make sure to post a link on the PR on GitHub! That’s where all of the feedback is being tracked.

1 Like

One more:

  • Modules loaded dynamically, can load other modules - those can have relative links, but the initial import seems to require an absolute or root relative path.

Should I copy paste it in, or just post a link to this thread?

I think a link to start with! Thanks!

Looks like some of these have already been addressed :slight_smile:

1 Like

Refactoring it so the internals of the function which does the dynamic import is all in it’s own module is much cleaner - then I can use static imports, and just export the function that will be imported later. If I understand the dynamic import stuff correctly I think this will have the same and desired effect?

Here is my new importer function:

function handleDeckRoute (params) {
  import('/imports/routers/deck-route-handler')
  .then(handle => {
    handle.default()
  })
}

And the module:

import React from 'react'
import store from '../store'
import {Provider} from 'react-redux'
import {mount, withOptions} from 'react-mounter'
import AppContainer from '../containers/AppContainer'

export default function handle () {
  const mountWithOptions = withOptions({ rootId: 'r-mount' }, mount)

  mountWithOptions(Provider, {
    store, children: <AppContainer />
  })
}

Oh! This last change actually reduced my packaged js file size! Pretty neat! :slight_smile:

4 Likes

Can you post a reproduction? Is the method you try to use await in a non-top level function that has the async keyword?

I tried it at the top level, and also inside a function - which I didn’t mark as async actually. Let me see if I can make it work, and then break it

You can only use await inside a non-top level function that’s marked with async

Furthermore there’s a bug in the transform-async-to-generator babel plugin, which causes it not to work together with super calls inside class hierarchies ( https://github.com/babel/babel/issues/3930 ), except if you also transform the classes with something like transform-es2015-classes.

Yeah, it looks like I was been using it incorrectly - I changed my handleDeckRoute function thusly:

async function handleDeckRoute (params) {
  const handle = await import('/imports/routers/deck-route-handler')
  handle.default()
}

And it works great!

Just to throw some commentary in there, it would be nice if we could use the static syntax instead of the function syntax. It would make some of the destructuring/default stuff a bit nicer, and more consistent with the top level (static) import statements. The above could then look like:

async function handleDeckRoute (params) {
  await import handle from '/imports/routers/deck-route-handler'
  handle()
}

I’m also curious if there is a better way to load a bunch of modules at once, rather than in serial? Does it matter?

1 Like

import() is a TC39 spec https://github.com/tc39/proposal-dynamic-import and there are numerous reasons (that I don’t claim to know) why an await import foo from 'bar' cannot be done (probably parser related).

There’s also a way to improve your example from above.

Instead of “waterfalling” your imports which will load them sequentially

async function func() {
  let mod1, mod2, mod3;
  import('/mod1')
    .then(m => {
      mod1 = m;
      return import('/mod2');
    })
    .then(... etc ...)
    .then(... etc ...)
}

Retrieve them all in parallel:

async function func() {
  const [mod1, mod2, mod3] = await Promise.all([
    import('/mod1'),
    import('/mod2'),
    import('/mod3'),
  ]);
}
4 Likes

I tried it our today the beta on my project and it works quite nice (reduced inital size from 600kB to 371kB.

However, I am curious if there is a better solution then that:

 FlowRouter.route('/contact', {
        name: 'contact',
        async action() {
            const Contact = await import('./components/contact.jsx');
            mount(MainLayoutCtx, {
                content: () => (<Contact.default />),
            });
        },
    });

What really looks strange that I need to access the default property of the Component like Contact.default.

I tried to implement a helper method for that however then the imports are not resolved anymore:

async function loadDymanic(path) {
    const Component = await import(path);
    return Component.default;
}
FlowRouter.route('/contact', {
        name: 'contact',
        async action() {
            const Contact = await loadDymanic('./components/contact.jsx');
            mount(MainLayoutCtx, {
                content: () => (<Contact />),
            });
        },
    });

This does not work anymore, as the imports are not resolved anymore.

1 Like

@davidsichau As you may have guessed, the reason loadDynamic doesn’t work is that the module identifier you’re importing is no longer statically analyzable, because you’re no longer passing a string literal to the import(...) function.

This is a deliberate design decision. If it suddenly became possible for any client to import(...) any module from the server, even if you (the developer) never explicitly imported the module, then existing Meteor apps would be at risk of exposing sensitive code to malicious clients.

One way to address this risk would be to have some sort of configuration file that whitelisted available modules, but, if you think about it, a much better way of whitelisting modules is to import(...) them using statically analyzable string literals in your code. Instead of having a separate configuration file, all the information about which modules are available is provided by the same code that uses the modules. When/if that code changes, you don’t have to remember to update the configuration file.

With that background in mind, here are two good ways to proceed:

  • Use a destructuring variable declaration:
const { default: Contact } = await import('./components/contact.jsx');
  • Export Contact by name in the contact.jsx module, instead of using export default, and then import it by name:
const { Contact } = await import('./components/contact.jsx');

You can also get loadDynamic to work as written by putting import('./components/contact.jsx') anywhere else in your code, even if that code is not executed:

function neverCalled() {
   import('./components/contact.jsx');
   import('./components/foo.jsx');
   import('./components/bar.jsx');
   ...
}

This works because the modules are once again statically analyzable, which allows them to be imported dynamically anywhere in your app, using any string value that resolves to the same module identifier. However, I think this approach is less appealing because it has the same drawbacks as maintaining a separate configuration file.

Just for fun, here’s one more possibility that might work in the future, but will not work today:

async action() {
  import Contact from "./components/contact.jsx";
  ...
}

While nested import declarations are allowed in Meteor (and this code will work!), they are not (yet) part of the ECMAScript specification. Furthermore, this style of import will cause the contact.jsx module to be included in the initial bundle, because you’re not using dynamic import(...), though it’s conceivable that one day we might treat import declarations inside async functions as if they were dynamic import(...)s, which would make this code do exactly what you want.

7 Likes

This is an edge case that I think we can fix by compiling import(...) to _module$temp0.dynamicImport(...) instead of module.dynamicImport(...), where _module$temp0 is some unique temporary variable that is reliably === to the CommonJS module object. This should also solve a similar problem with other Module.prototype methods like module.import(...) and module.export(...).

We hope to provide this kind of information in a more user-friendly way soon, but for now you can find all dynamically available modules in the .meteor/local/build/programs/web.browser/dynamic/ directory.

Another list of modules that I suspect would be highly useful: all modules in the static initial bundle that are not evaluated during page load (since those modules can probably be made dynamic).

If this is true, it’s a bug. Relative module identifiers should work everywhere. Please cook up a small reproduction if you have time!

3 Likes

Is there some way to figure out what is pulling in certain dependencies? Even if I comment out basically everything in my packages file, I still get jQuery and Blaze in my bundle. What’s causing that?

I ran this command:

for p in `meteor list | grep '^[a-z]' | awk '{ print $1"@"$2 }'`; do echo "$p"; meteor show "$p" | grep -E '^  [a-z]'; echo; done

And it gave me the following output (eventually - very slowly):

accounts-base@1.2.14
  autopublish@1.0.7 (weak dependency)
  blaze@2.1.8 (weak dependency)
  callback-hook@1.0.10
  check@1.2.4
  ddp@1.2.5
  ddp-rate-limiter@1.0.6
  ecmascript@0.5.9
  ejson@1.0.13
  isobuild:isopack-2@1.0.0
  localstorage@1.0.12
  mongo@1.1.14
  oauth-encryption@1.2.1 (weak dependency)
  random@1.0.10
  service-configuration@1.0.11
  tracker@1.1.1
  underscore@1.0.10

accounts-password@1.3.4
  accounts-base@1.2.14
  check@1.2.4
  ddp@1.2.5
  ecmascript@0.6.2
  ejson@1.0.13
  email@1.1.18
  isobuild:isopack-2@1.0.0
  npm-bcrypt@0.9.2
  random@1.0.10
  sha@1.0.9
  srp@1.0.10
  underscore@1.0.10

alanning:roles@1.2.16
  accounts-base@1.2.1
  blaze@2.1.3 (weak dependency)
  check@1.0.6
  mongo@1.1.2
  tracker@1.0.9
  underscore@1.0.4

aldeed:collection2@2.10.0
  aldeed:collection2-core@1.2.0
  aldeed:schema-deny@1.1.0
  aldeed:schema-index@1.1.0

aldeed:simple-schema@1.5.3
  check@1.0.0
  deps@1.0.0
  mdg:validation-error@0.2.0
  random@1.0.0
  underscore@1.0.0

check@1.2.5
  ejson@1.0.13
  modules@0.7.9
  underscore@1.0.10

cordova:cordova-plugin-statusbar@2.1.2
cordova:cordova-plugin-statusbar@2.1.2: not found

deanius:promise@3.1.3+
deanius:promise@3.1.3+: not found

dynamic-import@0.1.0-beta.9
  browser-policy-content@1.1.0 (weak dependency)
  check@1.2.5
  ddp@1.2.5
  ecmascript@0.7.0-beta.9
  isobuild:isopack-2@1.0.0
  modules@0.8.0-beta.9
  promise@0.8.8

ecmascript@0.7.0-beta.9
  babel-compiler@6.15.0-beta.9
  babel-runtime@1.0.1
  ecmascript-runtime@0.3.15
  isobuild:compiler-plugin@1.0.0
  isobuild:isopack-2@1.0.0
  modules@0.8.0-beta.9
  promise@0.8.8

edgee:slingshot@0.7.1
  check@1.0.2
  reactive-var@1.0.3
  tracker@1.0.3
  underscore@1.0.1

email@1.1.18
  underscore@1.0.10

es5-shim@4.6.15
  modules@0.7.7

fastclick@1.0.13

force-ssl@1.0.13
  ddp@1.2.5
  isobuild:prod-only@1.0.0
  underscore@1.0.10
  webapp@1.3.12

fourseven:scss@4.5.0
  caching-compiler@1.1.7
  ecmascript@0.5.8
  isobuild:compiler-plugin@1.0.0
  isobuild:isopack-2@1.0.0
  underscore@1.0.9

ground:db@2.0.0-rc.7+
ground:db@2.0.0-rc.7+: not found

kadira:dochead@1.5.0+
kadira:dochead@1.5.0+: not found

kadira:flow-router@2.12.1
  ejson@1.0.9-rc.1
  meteorhacks:fast-render@2.14.0 (weak dependency)
  modules@0.5.1-rc.1
  reactive-dict@1.1.5-rc.1
  reactive-var@1.0.7-rc.1
  tracker@1.0.11-rc.1
  underscore@1.0.6-rc.1

meteor-base@1.0.4
  ddp@1.2.5
  hot-code-push@1.0.4
  livedata@1.0.18
  underscore@1.0.8
  webapp@1.2.8

meteorhacks:fast-render@2.16.0
  accounts-base@1.1.1
  chuangbo:cookie@1.1.0
  deps@1.0.4
  ejson@1.0.3
  iron:router@0.9.0 || 1.0.0 (weak dependency)
  livedata@1.0.10
  meteorhacks:inject-data@2.0.0
  meteorhacks:meteorx@1.4.1
  meteorhacks:picker@1.0.3
  minimongo@1.0.3
  random@1.0.0
  routepolicy@1.0.1
  underscore@1.0.0
  webapp@1.1.2

mobile-status-bar@1.0.14

momentjs:moment@2.17.1

mongo@1.1.15
  allow-deny@1.0.5
  autopublish@1.0.7 (weak dependency)
  binary-heap@1.0.10
  callback-hook@1.0.10
  check@1.2.4
  ddp@1.2.5
  diff-sequence@1.0.7
  disable-oplog@1.0.7 (weak dependency)
  ecmascript@0.6.2
  ejson@1.0.13
  facts@1.0.9 (weak dependency)
  insecure@1.0.7 (weak dependency)
  isobuild:isopack-2@1.0.0
  minimongo@1.0.20
  mongo-id@1.0.6
  npm-mongo@2.2.16_1
  random@1.0.10
  tracker@1.1.2
  underscore@1.0.10
  webapp@1.3.13 (weak dependency)

seba:minifiers-autoprefixer@1.0.1
  isobuild:minifier-plugin@1.0.0
  minifier-css@1.1.10

shell-server@0.2.2
  ecmascript@0.5.7
  isobuild:isopack-2@1.0.0

standard-minifier-js@1.2.3
  isobuild:minifier-plugin@1.0.0
  minifier-js@1.2.18

static-html@1.1.13*
static-html@1.1.13*: not found

tracker@1.1.2

The only references to Blaze are both marked (weak dependency). jQuery isn’t listed anywhere, but is a dependency of Blaze, which I have assumed is where it comes from.

Are weak dependencies supposed to be bundled in either production or dev mode? (It seems they are bundled in both).

1 Like

can you use the chrome network tab to see if bundles are coming in as you move around to different pages in the app?

I tried relative links again, and it seemed to work. Looks like we are good on that front!

I also re-enabled browser policy and audit-argument-checks and both are working as expected again.

The loaded modules don’t show up in Chrome’s network tab - I’m guessing they come over the socket connection, and are instantiated through eval. I wonder what that does to 'use strict'; code