Meteor generates split bundles if you use dynamic import, is there a way to serve them from CDN like the main bundle, rather than DDP? Same for the main html.
1 Like
can you explain your use case a little bit please. this seems a super interesting topic, and i’d be keen to follow the discussion. sorry - have no idea about answer.
I don’t think this is possible right now, but I’m wondering how useful that would be? I found the fetching of the bundle to be extremely fast if they’re divided into reasonable junks.
Can you elaborate on why do you think this is good to have?
I enabled CDN on our app, the bundle size was ~1.7MB so I thought that doing so would improve load times on mobile, but it doesn’t seem to make a big difference for users in same country as hosting provider.
Next step was to split code into chunks using dynamic imports. But these then get served from the app server via DDP. If you do this in webpack usng common chunks plugin etc, you get a bunch of static assets that can be served from a CDN.
The other advantage to this is once you enable code splitting, a lot of times the chunks won’t change. With a single bundle (the default in Meteor/webpack), any change to your code means new hash/bundle. With splitting, only some of them will change, and the rest can continue to live on the CDN, and be cached on user devices, meaning faster downloads.
Same thing applies to using 3rd party libs - its better to load them from external cdn so they are cached and don’t have to be updated.
At least that’s my thinking. I know there’s only so much that can be done.
3 Likes
Arent they supposed to be cached on user browser after being downloaded once via DDP?
If you split the the code, then the initial bundle size will decrease and it will be cached on CDN. The modules are loaded via sockets when needed and for first time and I think this operation would be extremely fast since you’ve dedicated socket connection established (I mean if serving those files via sockets is slow, then so is the data). The Second time the user request the files, it will use the local storage of the browser.
If you’re too concerned about the initial fetch time of the module then those which are commonly used can be eagerly fetched and cached at the client and only rendered when a page needs it. So I still don’t see how caching the modules on CDN would speed the initial load or the load for returning visitors.
From what I’ve seen the bottleneck seems to be the time it takes to evaluate and initiate the JS scripts, the larger the file the more processing it will require. Even on local host, initiating a react app takes around 2.7 seconds on my machine, the only way I can think of to speed this up is SSR or using something lighter than react, others have wrote about this. With SSR you can achieve a meaningful first paint of less then a second and interactive site within 2.7 seconds.
Aside from the fact that loading assets from a CDN vs. directly from the a server will on average probably be a lot faster, it is also not a scalable solution – as your audience grows, the cost of adding Meteor containers (on Galaxy) is going to be a lot more expensive than using AWS/CloudFront, for example.
It simply isn’t a great use of resources, is literally more costly and is likely to be slower.
1 Like
It seems this should be possible now.
Meteor switched from DDP to HTTP POST for dynamic import requests and Cloudflare’s workers allow you to cache POST requests: https://blog.cloudflare.com/cache-api-for-cloudflare-workers-is-now-in-beta/
Not sure if you can cache POST requests for other CDNs but there’s at least one way!
3 Likes
I can’t seem to figure out how to load a few things via CDN and could use some guidance. From what I can tell the dynamic-import is done as a POST to the Meteor server but the URL is not affected by WebAppInternals.setBundledJsCssUrlRewriteHook(). This seems like it may be a bug as I can’t find this behavior documented anywhere (@filipenevola any guidance here?).
Issue 1: Loading dynamic imports over CDN
I’ve got the app bundle loading over CDN just fine with the standard setBundledJsCssUrlRewriteHook
WebAppInternals.setBundledJsCssUrlRewriteHook((url) => { // eslint-disable-line
return `${baseAssetUrl}${url}&_g_app_v_=${galaxyAppVersionId}`;
});
I can see the app bundle loading via the CDN correctly, i.e.
GET https://my-cloudfront-url.cloudfront.net/18d76ceb79698fad95458bfe175f044ed67097ce.js?meteor_js_resource=true&_g_app_v_=19
But any time I switch to a new route which is loaded with dynamic import() I see a POST request to the server again, i.e.
POST https://myapp.com/__meteor__/dynamic-import/fetch
Shouldn’t that POST url for dyamic-import/fetch be affected by setBundledJsCssUrlRewriteHook and point to Cloudfront instead as well?
Issue 2: Loading atmosphere packages over CDN
Another thing I’ve noticed is that the atmosphere package meteorhacks:zones, which Meteor APM and MontiAPM both recommend you install, is always loaded from the server and not the CDN., For example I see a request to this URL on every page
GET https://myapp.com/packages/meteorhacks_zones/assets/zone.js?1613159377686
Is there any way to get that package loaded from the CDN as well?
3 Likes
Was looking into this today.
Example:
POST to localhost:3000/__meteor__/dynamic-import/fetch
with payload
{"imports":{"ui":{"components":{"shared":{"ScrollToTop.js":1}},"libraries":{"Animate":{"scroll.js":1}}}}}
If :
- you updated ScrollToTop.js, deploy to live and have a new Meteor JS bundle and if
- in some way you would have gotten the dynamic import from a CDN
Then: you now get the same version of ScrollToTop.js that was cached in the CDN (previous bundle of Meteor code).
Pulling dynamic imports from a CDN requires versioning in the same way we have a new version number for the JS bundle every time we push new code. Client side, I know what I am asking for (the POST call to the server). However I am not able to find a version number or so in the response from the server and I am not too clear how a CDN would respond with a certain fetch POST (the new code).
The conversation above was really valuable to my project. Looking into how much code I split I realized that in production I am pulling code bundles at 4x more speed when I include them in the main bundle (and get from CDN) than when I pull them from the Meteor server as split code. I found it more efficient to revert back some of the split code chunks into the main bundle :).
One important factor that affects the Meteor server(s) is that, while both split code and main bundle cache on the client, the CDN delivery (for all clients) kicks in after the first visiting client’s first visit of any page in the app, while with split code, every client’s first visit is being delivered from the Meteor server as well as every first visit to a page that contains split code (e.g. a route that loads a dynamically imported component).
In real life, let’s say you have a social app with 10.000 users. You push a new bundle then announce everyone. All clients -(minus) 1 get the JS bundle from CDN regardless of the page they visit. All clients get the split code from the Meteor servers and keep pulling spilt code on other pages until the local cache is complete. Chances are the service quality/speed is being affected for all.
I am also very interested in options to fetch split code bundles from a CDN.
1 Like
I’ve looked into the Meteor code for dynamic loading; it’s rather complicated, I didn’t penetrate it in all parts. But from what I gathered, the chance of enabling dynamic import from a CDN looks very grim.
The bulk of the server side part is in /packages/dynamic-import/server.js, that’s where /__meteor__/dynamic-import/fetch is also implemented.
It appears that the client is requesting not just a single module, but a tree of modules all at once, see the request payload in the previous post, in which following two modules will be requested:
imports/ui/components/shared/ScrollToTop.js
imports/ui/libraries/Animate/scroll.js
…like this:
{
"imports": {
"ui": {
"components": {
"shared": {
"ScrollToTop.js": 1
}
},
"libraries": {
"Animate": {
"scroll.js": 1
}
}
}
}
}
The exact combination of which modules the client will request is completely dynamic and impossible to predict. It also depends on what is already in the client’s cache (IndexedDB), where entries can be deleted by the browser anytime, e.g. when disk space needs to be freed.
The server (fetch) responds with a similarly shaped document with the actual minified code, e.g.
{
"imports": {
"ui": {
"components": {
"shared": {
"ScrollToTop.js": "function module(e,t,l){let r,n,i,o,a,u,s, ..."
}
},
"libraries": {
"Animate": {
"scroll.js": "function module(t,n,r){(function(t){var e,u;e=this..."
}
}
}
}
}
The above means that a single client request is enough to load any number of modules. The server’s reading (and caching) the requested files from the bundle, and they can be plentiful.
WebAppInternals.setBundledJsCssUrlRewriteHook() redirects the request to a CDN, and there’s just two affected resources, the app.js and the bundled css.
How would the same work with a tree of requests? Say, the client needs to load 20 different modules dynamically in a certain moment (as works now with fetch). The CDN can only serve single files per request, so this would mean 20 requests/responses sent and received, and all need to be waited for until the initial dynamic loading request can finally be satisfied.
My app just loaded a seldom used widget dynamically. The response json has 334 lines and a size of 181 kb. I’m not sure how many requests that would have meant to a CDN, but I guess not less than 100.
When a client pulls a new JS bundle, the IndexedDB is being purged and new content is being cached. Here, things are clear, current code always reflects on the client in IndexedDB.
When we put something in between the server and the client, a CDN cache, what the client requests remains the same: ‘give me my code based on this payload’. The response from the serve is always the same (the tree) for the same request payload. The number of different request payloads is the number of dynamic imports called in the App.
E.g. If on a page (say React component) I have 3 dynamic imports and the code is not in IndexedDB, I generate 3 POST requests, with 3 different payloads and receive 3 responses. All users visiting that page would fall under the same scenario. Ideally I would wan to return those 3 responses from a CDN instead of the server.
Let’s say, the POST request would read the bundle version from the JS bundle and request a specific version. Back to my previous example:
{**"v": "202"**,"imports":{"ui":{"components":{"shared":{"ScrollToTop.js":1}},"libraries":{"Animate":{"scroll.js":1}}}}}
Then I tag the response with the same v number ?!! and cache it through a CDN so that all request with a certain payload and version number are first pushed to/from the CDN … same concept as CDN-ing the Meteor Bundle.
I think what happens on the client side in IndexedDB is managed by the client.js and cache.js of the package you mentioned above and … possibly there are no changes required in that area. We just need to make sure that we receive the proper JSON one time.
1 Like
I can’t relate to most of the conversation ( as per my knowledge)
But need to add one pointer
CDN can cache GET request better, but dynamic imports request is of post type.
Are we going to change it to the GET? Or something similar?
That’s one question I’m qualified to answer 
AWS allow you to cache POST requests:
And I assume other CDNs will have similar options
I vaguely remember there being discussion about using CDN edge services to process the requests and return the correct subtree. I don’t think I’ve heard of anyone successfully doing it however
2 Likes
If that’s the case, that’d be very fortunate! Would there be any change needed on the Meteor core code at all to support such a CDN setup?
1 Like
Hello,
My app is becoming more and more popular 
, and I am thinking of any way to improve load times.
One of them id CDN to avoid assets traffic on galaxy ( which I am running on ), but as I am using dynamic import I see that it is polled from galaxy and not CDN.
Did someone managed to changed that ?