Meteor Galaxy + Cloudfront CDN

kulttuuri · November 28, 2016, 3:59pm

Hey guys,

I’ve enabled the CDN for my meteor application using this package: https://github.com/nitrolabs/meteor-cdn/ which basically just sets the WebAppInternals.setBundledJsCssPrefix() for my app.

As normally public folder gets set cache-control headers to max-age=0 so it won’t get cached in CDN at all and having CDN in front of those would make no sense, I’ve also set this in my server startup file to set longer caching for certain assets:

if (Meteor.settings.public.PRODUCTION) {
	if (Meteor.isServer) {
		WebApp.rawConnectHandlers.use('/', function(req, res, next) {
			if (req._parsedUrl.path.startsWith("/swifticons") ||
				req._parsedUrl.path.startsWith("/gameicons") ||
				req._parsedUrl.path.startsWith("/landing") ||
				req._parsedUrl.path.startsWith("/animal_icons") ||
				req._parsedUrl.path.startsWith("/packages/fortawesome_fontawesome/upstream/fonts") ||
				req._parsedUrl.path.startsWith("/img") ||
				req._parsedUrl.path.startsWith("/largeiconpack") ||
				req._parsedUrl.path.startsWith("/robot") ||
				req._parsedUrl.path.startsWith("/skillicons") ||
				req._parsedUrl.path.startsWith("/tutorial"))
				res.setHeader('cache-control', 'public, max-age=3000');
			next();
		});
	}
}

My problem however is that the main meteor js & css files do set these kind of cache-control headers: cache-control:public, max-age=31536000 which makes the files be cached in cloudfront for long amount of time and whenever I deploy new application to meteor galaxy page shows outdated CSS & JS files from cloudfront for the user and it blows up the whole application for the user, until I go to amazon to purge my cache (and it can take around 10 minutes to fix the view for the user until the cache gets purged).

So for the solution, should I manually change my main meteor js & css files to have these kind of headers by default: cache-control:public, max-age=0 to prevent this error? And how should I match those files using the WebApp.rawConnectHandlers.use script I posted above?

andylash · November 28, 2016, 10:38pm

Are you sure you’re using 1.2.8 of that CDN package? I did a PR to add an internal galaxy version param that fixed this for me (the getting served stale files after a deployment part).

andylash · November 28, 2016, 10:38pm

github.com

Nitrolabs/meteor-cdn/blob/master/lib/server.js#L77-L93


		// Make the CDN_URL available on the client
		// console.log("Setting BundledJsCssPrefix to "+cdnUrl);
		var hasQuestionMark = new RegExp('[\?]');
		WebAppInternals.setBundledJsCssUrlRewriteHook(function(url) {
			// This code fixes an issue in Galaxy where you can end up getting served
			// stale code after deployments
			var galaxyVersionId = process.env.GALAXY_APP_VERSION_ID;
			var rewrittenUrl = cdnUrl + url;
			if (galaxyVersionId) {
				var separator = hasQuestionMark.test(url) ? '&' : '?';
				rewrittenUrl += separator + '_g_app_v_=' + galaxyVersionId;
			}
			return rewrittenUrl;
		});
		// WebAppInternals.setBundledJsCssPrefix(cdnUrl);
		__meteor_runtime_config__.CDN_URL = cdnUrl;
	}

kulttuuri · November 29, 2016, 7:40am

Yes, I have that version. I’ve however got that problem where it serves blank page couple times with the latest version, too. Happened once when deployed new service live with the CDN url set by default to something and then once when deploying new version to galaxy. Purging cache seems to fix it. My extension version when checking through meteor list: nitrolabs:cdn 1.2.8 Serve Meteor content from a CDN

In cloudfront I’ve made these kind of settings

Origin Domain Name: mywebsiteaddress.com (I did not make this point to galaxy ingress url as setting it never did work)
Origin Protocol Policy: Match Viewer
Viewer Protocol Policy: HTTPS Only
Forward Header: Whitelist (I’ve whitelisted Origin)
Object Caching: Use Origin Cache Headers (Customizing this and forcing TTL in cloudfront servers broke lots of things when updating the service, origin is the setting that works)
Forward Cookies: None
Query String Forwarding and Caching: None
Smooth Streaming: No
Restrict Viewer Access (Use Signed URLs or Signed Cookies): No
Compress Objects Automatically: Yes (I’ve enabled this to automatically compress things and thus reduce amount of object sizes)

kulttuuri · November 29, 2016, 7:44am

Are you pointing your cloudfront URL to galaxy ingress? Any differences in your cloudfront settings? When looking at your PR, this seems to be related to my issue. I’ll try to verify if that regexp / conditional matches mine.

Edit: My base js / css files look like this:
https://my.cdn.url.com/8c51d8a80b17e5c5eeb95f4bde7f57601e19d68e.js?meteor_js_resource=true&g_app_v=62 so this should match, right?

and FYI, I didn’t do these steps now that I look at it, what are the functionality of these settings and should I do them? Whitelist the Host and Strict-Transport-Security headers and Set querystring forwarding to “Yes”. Querystring forwarding also has 3 different options, none, Forward all, cache based on whitelist and Forward all, cache based on all, which one to use?

If I get all this working, I can go ahead and improve that readme for the extension explaining more in-depth what settings to do in the cloudfront and why as the current readme is quite lacking in information.

andylash · November 29, 2016, 6:06pm

Cloudfront settings are:

General:
Price Class: Use All Edge Locations
AWS WAF Web ACL: None
Alternate Domain Names:
SSL Cert: Default Cloudfront
Support HTTP Versions: HTTP1.1, HTTP1.0 (maybe i’ll change this to try and support 2)
Default Root Object:
Logging: Off
Cooking Logging: Off
Enable IPv6: Off

Origin Settings:
Origin domain Name: myapp.mydomain.com
Origin path: blank
origin ID: myapp.mydomain.com
Origin SSL protocols: TLS 1.2, 1.1, 1
Origin protocol policy: HTTPS only
HTTP port: 80
HTTPS Port: 443
No custom headers

Behaviors:
Path pattern: Default
Origin: myapp.mydomain.com
Viewer protocol: HTTPS ONLY
Alllowed HTTP Methods: GET, HEAD
Cached HTTP Methods: GET, HEAD
Forward Headers: Whitelist
Whitelist headers:
Access-Control-Allow-Origin
Content-Security-Policy
Origin
Strict-Transport-Security
Object Caching: Use Origin Cache HEaders
Min TTL: 0
Max TTL: 31536000
Default TTL: 86400
Forward Cookies: None
Query String Forwarding + Caching: Forward all, cache based on all
Smooth Streaming: No
Restrict Viewer Access: No
Compress Objects Automatically: Yes

I use Cloudflare DNS. I have a CNAME record for myapp to galaxy-ingress.meteor.com

kulttuuri · November 29, 2016, 8:25pm

I see. The documentation in github states: Whitelist the Host and Strict-Transport-Security headers, however when I whitelist that host, it ceases to work (502 when cloudfront tries to fetch data from my server). Seems like that might be the issue. I’ll make a PR with correct settings.