Deploy Meteor to Google App Engine 2017

hayk · May 2, 2017, 10:32am

Hi guys. So I am trying to deploy a simple meteor app to Google App Engine. I’ve tried following this tutorial https://cloud.google.com/community/tutorials/run-meteor-on-google-app-engine
But it resulted in
error: Can't find npm module 'meteor-deque'. Did you forget to call 'Npm.depends' in package.js within the 'meteor' package?
Googling resulted in a few more tutorials but via their comments it seems they are outdate as well.
There is also this one https://medium.com/google-cloud/meteor-google-a-devops-post-b8a17f889f84
However this is about deploying to the compute engine, so this is a plan B.
So I wander if any of you successfully deployed Meteor to GAE recently in 2017 with Meteor 1.4? Can you please share details?

ffxsam · May 4, 2017, 6:24pm

I couldn’t even get mine deployed.

hayk · May 5, 2017, 5:52am

I tried around a few tutorials, most of them were for compute engine though. However even those do not work. Deploying Meteor is really painful.
I’ll let you if I find any solution and please you too let me know if you get any solution.

kiyohiko · May 6, 2017, 1:54pm

Hi guys, I was in the exact same case as you: unsuccessfully trying to deploy my meteor 1.4 app on GAE, and fortunately, I eventually managed to do so (starting from the “how to dockerize meteor” you linked above).

Here is what I did:

app.yaml

env: flex
runtime: custom
threadsafe: true
automatic_scaling:
  max_num_instances: 1
env_variables:
  ROOT_URL: https://<gae-app-name>.appspot.com
  MONGO_URL: mongodb://<mongodb-username>:<mongodb-password>@<gce-ip>:27017/<mongodb-name>
  DISABLE_WEBSOCKETS: "1"
skip_files:
- ^(.*/)?\.dockerignore$
- ^(.*/)?\npm-debug.log$
- ^(.*/)?\yarn-error.log$
- ^(.*/)?\.git$
- ^(.*/)?\.hg$
- ^(.*/)?\.svn$

Dockerfile

FROM launcher.gcr.io/google/nodejs
RUN install_node v4.8.2
COPY . /app/
RUN (cd programs/server && npm install --unsafe-perm)
CMD node main.js

Steps to deploy

$> meteor build ../ --directory --architecture os.linux.x86_64 --server-only
$> cp app.yaml ../bundle/ && cp Dockerfile ../bundle/
$> cd ../bundle && gcloud app deploy --verbosity=info -q

I also removed bcrypt from my project dependencies, it was always crashing because of that.

Some explanations:

I added skip_files to ensure the ‘node_modules’ directory generated inside ../bundle/programs/server/npm/ was uploaded during the process. I think that “Can’t find npm module ‘meteor-deque’” errors might come because of that.
I added RUN install_node v4.8.2 to force the right node version on GAE.

(I used the Bitnami mongodb from GCE, if you have some issues working with this, I can’t give you some answers on how I did.)

Hope this helps you!

a.com · May 6, 2017, 2:49pm

Any reason you want to use Google App engine in particular? Now/Zeit is an okay alternative using meteor-now

husayt · May 6, 2017, 6:16pm

@kiyohiko thank you very much, I was struggling with this for a long time. It now worked.

Although i did not have to change node version, so i skipped docker file.

here is the .bat file i ended up with.

SET depDir=.deploy
rmdir %depDir% /s /q
meteor build .%depDir% --architecture os.linux.x86_64 --server-only
copy .extra* %depDir%\bundle
cd %depDir%\bundle
gcloud beta app deploy -q --verbosity info

Note, i placed app.yaml and package.json in .extra folder

ffxsam · May 7, 2017, 6:32am

I still can’t get this working. Below are my app.yaml and Dockerfile:

app.yaml

env: flex
runtime: custom
env_variables:
  ROOT_URL: https://xx.appspot.com
  MONGO_URL: mongodb://xxx
  DISABLE_WEBSOCKETS: "1"

Dockerfile

# Dockerfile extending the generic Node image with application files for a
# single application.
FROM gcr.io/google_appengine/nodejs
RUN /usr/local/bin/install_node 4.8.2
COPY . /app/
RUN (cd programs/server && npm install --unsafe-perm)
CMD node main.js

It bombs at the RUN step:

Step 4 : RUN (cd programs/server && npm install --unsafe-perm)
 ---> Running in 36160b426d9a
npm WARN package.json meteor-dev-bundle@0.0.0 No description
npm WARN package.json meteor-dev-bundle@0.0.0 No repository field.
npm WARN package.json meteor-dev-bundle@0.0.0 No README data
npm WARN package.json meteor-dev-bundle@0.0.0 No license field.

> fibers@1.0.15 install /app/programs/server/node_modules/fibers
> node build.js || nodejs build.js

`linux-x64-46` exists; testing
Binary is fine; exiting

> meteor-dev-bundle@0.0.0 install /app/programs/server
> /nodejs/bin/node /app/programs/server/npm-rebuild.js

events.js:141
      throw er; // Unhandled 'error' event
      ^

Error: spawn npm ENOENT
    at exports._errnoException (util.js:907:11)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:189:32)
    at onErrorNT (internal/child_process.js:363:16)
    at nextTickCallbackWith2Args (node.js:511:9)
    at process._tickCallback (node.js:425:17)
    at Function.Module.runMain (module.js:443:11)
    at startup (node.js:140:18)
    at node.js:1043:3

npm ERR! Linux 4.4.0-75-generic
npm ERR! argv "/nodejs/bin/node" "/nodejs/bin/npm" "install" "--unsafe-perm"
npm ERR! node v4.8.2
npm ERR! npm  v2.15.11
npm ERR! code ELIFECYCLE
npm ERR! meteor-dev-bundle@0.0.0 install: `/nodejs/bin/node /app/programs/server/npm-rebuild.js`
npm ERR! Exit status 1

But there are no details anywhere, I can’t tell what’s missing. If I run docker build . manually (even on a Linux box), it builds the image just fine.

kiyohiko · May 7, 2017, 10:00am

Hi, did you run gcloud app deploy with --verbosity=info to see if there are any skipped files that should be uploaded? (if there are, you should add skip_files: [...] to avoid the default skip regex.

kiyohiko · May 7, 2017, 10:03am

For me, it’s mostly about the pricing, the scalability and the evolution options available on GAE, but if you’re happy with Now/Zeit (about which I don’t know much), I guess it’s fine

ffxsam · May 7, 2017, 2:41pm

Ahhh, thank you! Yes, it was ignoring all the node_modules files. I used your skip list instead, and now it actually built properly and is in the process of deploying.

ffxsam · May 7, 2017, 2:48pm

Spoke too soon. It pushed up, but now I see the “can’t find meteor-deque” error. Argh! Though it seems more like a warning, maybe. My test app is running ok so far.

Side note: is it normal for a plain Meteor app that does pretty much nothing to consume nearly 700MB RAM??

And on top of that, I’m getting tons of these errors:

kiyohiko · May 7, 2017, 4:26pm

Mine is using around 350 Mo RAM

I solved the errors you have (POST xhr for sockjs with 404 response code) by adding:

threadsafe: true
automatic_scaling:
  max_num_instances: 1

in app.yaml

ffxsam · May 7, 2017, 4:27pm

But doesn’t limiting the max instances to 1 kind of defeat the purpose of using App Engine for auto-scaling?

kiyohiko · May 7, 2017, 4:28pm

Yes, you’re right, I just found those lines in another “how to”, didn’t try with just threadsafe maybe it works

ffxsam · May 7, 2017, 4:38pm

I’m not sure what I’m doing wrong. I’m still getting tons of the 404 sockjs errors. I’d reach out to Google support but I doubt they’d have any clue since this is probably Meteor-specific.

I’m also seeing console.log calls on the server show up way more times than they’re supposed to! A simple button click on the client is supposed to dump a message once. Hosted on Google App Engine, one button click dumps out 5-6 instances of the same message. All this, plus the 700MB RAM consumption… I’m pretty disappointed.

 3005 ?        Ssl    0:00 /opt/google-fluentd/embedded/bin/ruby /usr/sbin/google-fluentd --log /var/log/google-flu
entd/google-fluentd.log
 3103 ?        Sl     0:11 /opt/google-fluentd/embedded/bin/ruby -Eascii-8bit:ascii-8bit /usr/sbin/google-fluentd -
-log /var/log/google-fluentd/google-fluentd.log --under-supervisor

There’s a ruby process running on each instance that seems to be taking up a bit of memory:

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND                                              
 3103 root      20   0  726m  87m 9520 S   5.7  8.7   0:12.19 ruby

hayk · May 8, 2017, 6:17am

Hi guys!
@kiyohiko thanks for your contribution. I was just able to deploy the simple tutorial app with the help of your first post without any problems.
Just one thing to note, you are using node 4.8.2 I used 4.6.2 since it’s the version mentioned in the official docs https://guide.meteor.com/deployment.html#custom-deployment
Now I’ll try to deploy my real app which is using react also and update the results here.

SkyRooms · May 8, 2017, 6:31am

You need Meteor Galaxy. That’s the entire purpose, because, YEP it’s complicated to deploy.

I have my app running perfectly after a LOT of frustration. Maybe I’ll do a video on how I deploy it.

hayk · May 8, 2017, 6:59am

Hey guys it seems my real app is building well too. Just one problem I am running my meteor with meteor --settings settings.json but in the gae it seems that the app runs without the settings because I am getting errors about that. Is there anyway I can connect the settings.json in the gae?

ffxsam · May 8, 2017, 2:02pm

I guess I didn’t understand App Engine’s pricing for their Flexible option. If you had two instances (2 vCPU each), it would cost $153/mo. I really don’t see how that’s an attractive price compared to Galaxy. And if I’m not mistaken, you can’t use runtime: custom in App Engine Standard.

There goes that!

SkyRooms · May 8, 2017, 2:31pm

Make sure you compress this file to ONE line.

{this:{
wont:{work}
}}

Here’s a copy of my settings.js that DOES work. It’s silly and there aren’t any errors to say “please fix your settings.json by compressing”.

{“public”: { “ga”: {“id”: “xx-xxx-xxx-xx”} }}