"Server Error" Deploying to Galaxy [SOLVED] (9 days later?!)

deployment
1

I’m really disappointed that in my experience, Galaxy is still not ready for production use for two reasons:

  1. The platform itself has unresolved, critical technical problems
  2. The Meteor Galaxy support infrastructure is unable to adequately address problems in a timely manner

Read my story below and let me know if you think my expectations are unreasonable

March 4th, 2017 - Deployed my Meteor app on Galaxy in HA mode. Yay! it works!

Rerun the deploy script every few days to deploy code updates, no problems.

March 11th, 2017 - Run the script again. FAIL! with the following error (same script that worked before)

Talking to Galaxy servers at https://galaxy.meteor.com
Error deploying application: Server error (please try again later)

Nothing in the Galaxy logs to hint at what the problem might be.

Hmmm, problem with my code? Apparently not, since I can still build and deploy to my other staging server on DigitalOcean.

Submit a Support Ticket on Saturday.

Yeah, I have the Basic Support, so I need to wait until Monday…

March 13th - Monday rolls around, and it takes all day for Meteor Support to determine that I’ve done everything correctly:

Meteor Support: After reviewing your information, I wasn’t able to spot any errors; … I’ll escalate this to the team; once they’ve had a chance to review your case, we’ll update you.

OK, grrr, it’s been 3 days now guys…

March 14th - Another day goes by with no update, so I ask for one:

Meteor Support: I’m happy to update. I’ve escalated this to the team, and I’m awaiting a response; once that’s available, we’ll email you again with more information.

Really?

March 15th - Yet another day goes by, no resolution, no updates, unable to deploy code updates for 5 days, so I’m getting a little peaved:

Paul Asks:
1 - I’m on Day 5 of being unable to deploy updates to my app on your servers because of your “Server error (please try again later)”.
2 - The feedback I got from “Meteor Support” (two days ago) was “After reviewing your information, I wasn’t able to spot any errors” (hardly helpful)
3 - I’ve had no progress update for 48 hours…
4 - This app needs to go live into full production in 21 days.

This inadequate level of support and communications from Meteor Support re Galaxy hosting has left me with these questions:

1 - If I was paying $399/mo for support would this issue have been taken care of already?
2 - Can you provide me the official terms of Galaxy’s SLA?
3 - I’m unable to deploy updates, so I consider the current status of your service as currently operating in “crippled” mode. Is this considered as not being a problem by your SLA?
4 - When can I expect this problem to be fixed?
5 - When can I expect a progress update?

The response I got from Meteor Support was as follows:

Hi Paul,
I apologize for the formatting of my response; for clarity, I’m including it again, with clearer numbering this time. If you have any questions about this, please let me know.

  1. The support you are currently receiving, as part of the Galaxy Base plan, is very different from the support you would received on our enhanced plans, as described on the page below. You can received support where our on-call engineers will troubleshoot your app within 15 minutes or 1 hour of an issue (depending on your plan), at a level of detail including stack traces, for example; I can share more details upon request.
    http://galaxy-guide.meteor.com/support.html
  1. Our SLA is described in the same article (linked again, for convenience). To summarize, support on the Galaxy Base plan entitles you to at least one update per business day.
    http://galaxy-guide.meteor.com/support.html
  1. We don’t believe this is a systemic issue stemming from the Galaxy side at this time; we believe it is compliant with your SLA. Once an engineer is able to review your case, they’ll be able to explain the issue in more detail.
  1. We do not commit to ETA’s for issues when some action may be required on your end to fix this (which would be out of our control).
  1. While you can expect an update daily, our engineer should be able to contribute a more substantive update tomorrow, after he’s had the time to review your case.

Cheers,
Julian

:astonished: Am I reading that correctly?

  1. If I want “real” support, I need to pay $399/mo, otherwise, you just get what you get.

  2. Even though it was fairly obvious that I was asking what the Galaxy platform SLA was, the rep answered with what the Galaxy support SLA was. And that’s “we only need to provide you a daily update”, with “eventually, an engineer will look at it” being adequate as a daily update?!

  3. Even though an engineer apparently hasn’t even looked at my case after 5 days, the fact that since they “believe” it’s not a “systemic error”, there’s been no violation of their SLA?! Also, despite the fact that THEIR code is throwing an error that says “Server error (please try again)”, they don’t believe it’s a problem on their end?! (Don’t forget, this same code builds and deploys successfully on a non-Galaxy platform.)

  4. They felt the need to reiterate that since they don’t think the problem is on their end, they don’t have to commit to an ETA, even though they admitted that I am doing everything correctly on my end.

  5. They also effectively confirmed that an engineer HASN’T reviewed my case, and yes, it can take 6 or more days to help solve “Galaxy Server Errors” if all you have is a basic support contract.

Are my product support expectations too high? I love what Galaxy promises, but it looks not ready to deliver…

2

Just informed at 9pm on Day 6 that an engineer has not yet looked at my problem…

1 Like
3

As far as my experience with Galaxy goes, I never had problem deploying any app, never. Everytime I needed support they replied in one day, but it was never something like you are seeing.

I dont know if it exists, but try to execute it in a verbose mode, because if everyone but you can deploy, the problem is problem not on galaxy.

4

I can successfully build and deploy my app to a DigitalOcean Droplet.

The error I’m getting says:

Talking to Galaxy servers at https://galaxy.meteor.com
Error deploying application: Server error (please try again later)

There is no “verbose” switch for deployment:

Without any logs or additional output to analyze, I’m (still) dead in the water…

5

People gotta pay rent. How many clients do they need at $399/mo to support a single engineer? How many service tickets can that engineer service in a day, considering that a single ticket might take a day or more to trace and debug and fix.

Your expectations may be too high.

6

No, not really. Even on a mass host provider like OVH or some cheaper ones you’ll get responses within 1 business day. That’s the way it is and the way it should be. If I pay a higher price for Galaxy I expect that everything is working and I’ll get a response within a common time if something went wrong - but 6 days is absolutely not okay.

1 Like
7

As someone else with the Basic Support, I don’t expect for them to drop what they’re doing to debug an Error deploying application error. That’s on me or the dev team I’m working with.

When I get those errors (and it has happened plenty of times), I go back to my code and remove components until it deploys correctly. If I have to, I go back to a boilerplate, and copy code over file by file.

Invariably, I’ve added some exotic bit of code that I thought was reasonable; but wound up having dependencies that required C compilers, python hooks, etc. I’ve had it happen with node-sass, imagemagick, samtools, etc.

Just because his app deploys on Digital Ocean doesn’t mean it should necessarily deploy on Galaxy. That’s an incorrect and unfair assumption, and it sets up unreasonable expectations.

2 Likes
8

Yeah, I’m not saying that MDG is committed to solve an issue, but they should give a response to it. That’s not the case if I’ve understand the author right.

9

When I see that kind of error message, I can only assume the problem is on their end…

I did finally get an actual “engineer” looking at my issue on Friday (after 6 days) but still no fix… (Day 7)

10

Finally solved after 9 days, and such a simple solution! In case anyone runs into similar issues deploying to Galaxy, I’ve copied the full text of the support engineer’s final troubleshooting instructions. It turns out that a simple 'meteor logout" and ‘meteor login’ is all it took to fix my problem.

Galaxy Support:

It seems like your current failure is happening early in the bundling process, possibly before you are even authorized properly with the Galaxy servers. The error message is particularly interesting though as it would appear that your computer isn’t able to complete the handshake with our Galaxy servers. I’ve confirmed that they are working properly for me (from a Windows computer) to the same region that you are deploying to, so something else seems to be awry here.

I’m curious if you would be able to try logging out of Meteor and logging back in again using the following two commands:

meteor logout

and then

meteor login

At which point you should be prompted for your username and password again.

Once you are re-logged-in successfully, please try deploying again with the meteor deploy command. Also, you should be able to remove the --owner flag from your deploy command as that should only be necessary the very first time you deploy an app in order to specify which account you would like linked to the deployment. After the initial deploy (which creates the app), this argument is no longer used. I’d recommend trying to deploy with this argument removed.

Lastly, if you continue to have problems deploying at this point, if you could try enabling additional debugging by running the following on your Windows prompt before running meteor deploy it should provide some additional information:

SET METEOR_LOG=debug

And then running your meteor deploy command as usual. Please provide the additional output provided when this command is enabled in an update to this issue. Hopefully we can get this resolved for you soon!

Cheers,
Jesse

1 Like
11

I guess this is something that should be added to the FAQ’s for Galaxy combined with the error message.

12

For those curious, this issue ended up being that @cormip accidentally had a space in the DEPLOY_HOSTNAME environment variable, which was quite subtle to figure out over email! Fortunately we’re going to fix that for everyone in the future via https://github.com/meteor/meteor/pull/8508 .

I personally don’t recommend that people set DEPLOY_HOSTNAME at all when using Galaxy, except when initially setting up their app when DNS doesn’t exist yet. It used to be the case that you always had to set DEPLOY_HOSTNAME when using Galaxy and many of our docs still suggest it, but if you’ve already set up your app’s main hostname to point at the Galaxy ingress, meteor deploy will automatically find the right Galaxy server to deploy to, no matter what region you’re deploying to.

13

@glasser, your assessment of the problem cause is incorrect. The problem stated at the start of this thread was solved by logging out and logging back in again as I explained in my earlier comment. I discovered the separate, unrelated problem of having a trailing space in DEPLOY_HOSTNAME after having solved the issue and simply alerted MDG Support by replying to my previously opened ticket. No doubt having this separate issue reported on the same ticket caused some confusion. I have not heard that a cause for my original problem was discovered or addressed.

14

Oops, sorry for misinterpreting there.

15

regardless, thanks for adding that comment. I had a space at the end of my DEPLOY_HOSTNAME and the issue was driving me crazy.

16

The most troubling part of all this is an official MDG staffer (@glasser) revealing information about an issue not volunteered by the customer (@cormip) himself (space is hostname). This is seriously unprofessional.

17

Hi @nadeemjq. It’s important for us that forum members following problems posted publicly by our users can see the ultimate solution in case they run into similar issues. An extra space in an environment variable is not personally identifiable information and is relevant to the issue raised by the user in this discussion; you can email support@meteor.com if you’d like to discuss our terms of service in more detail.

2 Likes