Treat public folder by nginx instead of meteor


#1

How can Nginx treat my Images from public/assets/images instead of Meteor?

Actually, I am trying to improve my Web App first visit load time.

So I read something nginx treat the images or public folder can help improving the first load of a Meteor based web app.

Is there any good article about this? Any tips would be appreciated. Thanks


Serving static files with nginx
#2

I’m highly recommend to use Phusion Passenger + Nginx setup for any node.js app.
Assuming you have config like this (if you don’t use Phusion, just ignore its settings):

server {
  listen 80;
  server_name myapp.com

  root /var/www/myapp/public;
  passenger_app_root /var/www/myapp;

  location / {
    try_files $uri @application;
  }

  location @application {
    proxy_http_version  1.1;

    passenger_enabled on;
    passenger_app_type node;
    passenger_startup_file main.js;
    passenger_sticky_sessions on;
    passenger_min_instances 1;
    break;
  }
}

Then copy all static assents to /var/www/myapp/public, where ./bundle is build app directory:

cp ./bundle/programs/web.browser/*.css /var/www/myapp/public/
cp ./bundle/programs/web.browser/*.js /var/www/myapp/public/
cp ./bundle/programs/web.browser/*.html /var/www/myapp/public/
cp -R ./bundle/programs/web.browser/app/* /var/www/myapp/public/
cp -R ./bundle/programs/web.browser/packages /var/www/myapp/public/

#3

Also I have found this gist - might be helpful for you too.


#4

Why do I need to copy assets from ./bundle to myapp/public ?


#5

Because nginx is set to look for the files only at: root /var/www/myapp/public


#6

But my images are already places in mymeteorapp/public/assets/iamges.


#7

Even after running meteor build?


#8

I have never run the meteor build command. I use meteor --port 9000 --production


#9

Well, you should do it right way one day, good luck with it. #RTFM


#10

I sympathise with dr.dimitru’s frustration, but I think a link may help: https://guide.meteor.com/deployment.html#deploying :smirk:


#11

Actually following the deploy instructions and building your app will significantly improve your load times. Do that first before trying anything else.

Take heed:


#12

This is close but a little too excessive/unnecessary when you are using passenger, because passenger already understands a node/meteor app’s layout and static asset serving, interfering with static asset requests, giving them the necessary boost and passenger specific optimizations on top of nginx features.

In fact, by all this copying and extra configuration, you are making passenger’s advanced static file handling obsolete and just using stock nginx features.

Check out https://www.phusionpassenger.com/library/walkthroughs/deploy/ for an updated and detailed passenger deployment instruction kit and check out https://github.com/phusion/passenger/wiki/Phusion-Passenger:-Meteor-tutorial for an older version where they actually explain that sttic file handling is automatic, and finally https://www.phusionpassenger.com/library/indepth/meteor/app_autodetection/nginx/ is a short explanation of the why/how of the magic.


#13

Which is good, as Nginx performs much better.

The magic of Phusion Passenger is only in simplicity of initial setup. But when you would like to scale and optimize your server setup it’s easily can be accomplished. Do not take my word for it - take a look on Phusion Passenger sources and run load tests.

And yes, Phusion Passenger can do magic for Meteor app (not bundled) - again it should be used only for small scale apps or tests purposes. Always build you app, and run it as regular node.js app.


#14

I’m quite sure that this argument is not correct.

I think you might be confusing passenger’s unbundled (meteor) mode for passenger nginx integration mode which serves a bundled Meteor app through a tight integration with nginx. You’ll see the reference if you take a peek at those documents I’ve linked to.

Passenger fully utilizes the nginx configuration that you are trying to achieve by your config directives, and adds more on top in terms of performance and scalability.

I am indeed running passenger on a number of servers, even one app running 16 concurrent meteor app instances and 100% sure that it outperforms vanilla nginx due to those aforementioned extra features.


#15

I was talking about (Nginx + Passenger) -> Node.js (not sure how to call a Meteor app after running build command. Bundled? Builded?).

I’m contributor to Passenger’s node-loader and most of the other node.js related code. There is not that much magic.

I agree what integration with Nginx is deep and well done, but it’s still made on top of original Nginx.


#16

“Bundled” is the generally accepted terminology.

Yes it is and that is my point. It correctly utilizes nginx such that the passenger_app_root directive and the selective file copying along with the extra location nesting are excessive and misleading to future readers of this post.

I think as heavy users and advocates of passenger in meteor-self-deployment, you and I, should try to avoid configuration options that are actually provided by the underlying engine when trying to present the ease of passenger to new comers like the OP in this post.

In this case, following a mkdir /home/myappuser/myapp/bundle/public, the following config should “just work”:

server {
  # Standard minimal nginx config
  listen 80;
  server_name myapp.com;
  root /home/myappuser/myapp/bundle/public;

  # Signal passenger that this is a node app (bundled meteor apps are node apps)
  passenger_enabled on;
  passenger_app_type node;
  passenger_startup_file main.js;
  passenger_sticky_sessions on;

  # Meteor specific environment variables
  passenger_env_var ROOT_URL http://myapp.com;
  passenger_env_var MONGO_URL mongodb://localhost:2017/myapp;
}

For a more extensive configuration I use on non-ssl endpoints, check this out:

# Optional more verbose logging
passenger_log_level 4;

# Optional fine grain scaling
passenger_max_pool_size 4;
passenger_max_instances_per_app 3;

server {
  # Standard minimal nginx config
  listen 80;
  server_name myapp.com;
  root /home/myappuser/myapp/bundle/public;

  # Signal passenger that this is a node app (bundled meteor apps are node apps)
  passenger_enabled on;
  passenger_app_type node;
  passenger_startup_file main.js;
  passenger_sticky_sessions on;

  # Meteor specific environment variables
  passenger_env_var ROOT_URL http://myapp.com;
  passenger_env_var HTTP_FORWARDED_COUNT 1;
  passenger_env_var MONGO_URL mongodb://localhost:2017/myapp;

  # Optional Meteor specific environment variables
  passenger_env_var MAIL_URL smtp://user:pass:@smtp.server.com:587;
  passenger_env_var MONGO_OPLOG_URL mongodb://localhost:27017/local;
  passenger_env_var METEOR_SETTINGS '{ "public": { "publicFoo": "publicBar" }, "privateFoo": "privateBar" }';
  
  # Optional fine grain scaling
  passenger_min_instances 1;
  passenger_force_max_concurrent_requests_per_process 100;

  # Optional production flags for libraries potentially seeking this
  passenger_app_env production;
  passenger_env_var NODE_ENV production;

  # Optionally make sure your server times are always utc
  passenger_env_var TZ Etc/UTC;

  # Optionally make sure your app can utilize native server libs, e.g. graphicsmagic when a dependency seeks it in the PATH
  passenger_env_var PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin;
}

# Optionally prestart the app without waiting for the first request
passenger_pre_start http://myapp.com;

And nice work linting that node starter script, I’ve seen other projects with different core languages creating small js scripts, only to neglect to reflect their overall code quality. Not surprized to see that with passenger, either :slight_smile: Kudos to you for noticing and fixing that file’s format! :+1:

I’d also love to hear your opinions on some extra/optional configuration you find valuable in ssl/non-ssl endpoints. Let’s push passenger forward for all the other self-deployers out there!

PS: I still find Galaxy to be by far the best option for apps that can afford it.


#17

You’re right about bundle/public, but at the same time you’re pointing there Nginx with root /bundle/public, moreover it won’t work without copying files from /bundle/programs/web.browser/ into /bundle/public. From the docs you’ve referenced:

Phusion Passenger leverages this property to make your life easier. It assumes this convention by default so that you don’t have to write so much configuration. If your app does not follow the convention then there are configuration options to tell Phusion Passenger about the structure.

According to Phusion Passenger’s convention, you will need two extra directories in your Meteor app directory:

  • public/ - The same directory used by Meteor for static files. All files in this directory will automatically be served by the web server, bypassing Meteor. For example, if there’s a file public/foo.jpg, then any requests to /foo.jpg will be handled by the web server, and never passed to the application.

For simplicity reasons, this tutorial assumes that your app follows this convention. If you prefer to put the public and tmp directories elsewhere then that’s entirely possible. The manuals tell you how to configure their locations.

Nothing said about support of bundle/programs/web.browser/, or what files will be moved into bundle/public. The only case when this will work is meteor run (development mode) where public may exist and has files.

Also at this tutorial, it isn’t mentioned. I’ll send a PR to add this info, good you’ve pointed this issue out.

As you have mentioned should try to avoid configuration options that are actually provided by the underlying engine when trying to present the ease of passenger - perhaps most of the options might be avoided. Overall looks standard. I’m more about tuning Nginx’s options.

I believe using the right tool for the right task. So, yes Galaxy is great when you don’t want/need to care about servers, Linux, MongoDB, etc.
I personally prefer dedicated servers, but always suggest to use complete solutions as Galaxy to my clients, as it will save them from a headache and extra expenses.


#18

Are you absolutely, positively sure about this? This is the way I’m using this (also as described in the official, new version of the docs) and here’s what I’m getting for calling a text file residing in my app’s public directory and I can also see this request in /var/log/nginx/access.log which leads me to believe that it is nginx which is serving.

image

If I’m missing something here, I’d really appreciate your pointer, and of course a PR to fix that on the official docs would also be amazing, and if you don’t have the time to do that, I can do it as well becaue this would be a huge loophole in the docs!

Although, even if that is the case, I would still avoid copying files to /var/www due to security reasons and let passenger utilize automatic user switching to run the app with the app’s own user, and then set up my app as described in the free style node structure configuration article

server {
  ... other options
  passenger_app_root /home/myappuser/myapp/bundle;
  root /home/myappuser/myapp/bundle/programs/web.browser;
  ... other options
}

But then this would also require some further set up due to another mapping for /app directory within the browser bundle.

Would you mind moving this conversation forward with me so that we can get to the bottom of this and come up with a best practice to help official docs to rely on?

Regarding Galaxy, yes, my feelings exactly. I find that Galaxy might not be prefreable only in one or more of these conditions:

  • Money is really tight, time is a lot, and app can tolerate some downtime
  • Database proximity is utterly important, perhaps even preferable within the same box
  • App contains some high-cpu computations

By the way, I was meaning to send a PR to the passenger library for the upcoming 1.6 release, would you like to collaborate on that as well?


#19

To find out if a file was served by Nginx or Phusion Passenger use X-Powered-By header, also Set-Cookie won’t have _passenger_route when served directly by Nginx.

I’ll start classically with new issue at Phusion Passenger repository.

I wouldn’t touch current Phusion Passenger implementation and prefer to limit it with better docs explaining how it works, and what all static files should be moved into the directory where root option is pointing in order to let Nginx serve them. Meteor’s “bundle” structure is more complex than simply pointing to bundle/programs/web.browser/, at it has /app directory and /packages which aren’t “relative” to the used path.

Security is widely discussed at PP docs, I don’t think it needs to be updated.

By the way, I was meaning to send a PR to the passenger library for the upcoming 1.6 release, would you like to collaborate on that as well?

Yes, I would :slight_smile:


#20

Hey I had to be away for a few days but now that I’m back, I’ll try to get the ball rolling on both topics. Watch this space for updates, and I’ll ping you on github, too.