@jkatzen thanks. I am still getting random issues, so perhaps my setup is messed up.
UPDATE to the info below:
I think I know what happened and it had to do with swapping the IPs.
eg Server 1 - IP-addr-A (AWS dyamic address)
swap Server 2 to IP-addr-B (Elastic IP)
Somehere in the nginx settings it’s still referring to IP-addr-A which now no longer exists as it was originall a dynamic AWS IP.
Could that be possible?
------------orginal below ---------
Folk are experience sporadic “disconnection” and I see the following in the logs:
May 06 09:49:03 Buzzy-Docker-buzzy1 nginx-proxy: nginx.1 | 2015/05/05 23:49:03 [error] 26#0: *5152 upstream prematurely closed connection while reading response header from upstream, client: <someones IP>, server: *.buzzy.buzz, request: "GET /_timesync HTTP/1.1", upstream: "http://<and internal amazon IP egg 172.something>/_timesync", host: "buzzy.buzz", referrer: "https://buzzy.buzz/l"
I don’t have a server with the internal ip of <and internal amazon IP egg 172.something> and I don’t see it listed in my cluster database.
The weird things is, I don’t think I have changed any of my app settings when I deploy. So not sure why it’s not working?
I am not sure if this is related, but sometimes in my deploy logs I see the following:
================================================================================
Error executing action `run` on resource 'bash[docker-logspout]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/tmp/chef-script20150505-15749-r7i4yu" ----
STDOUT: 14a0ebf9bd7700085849e058a658fe1f8afd345b5e5321c2f7dac3a58b0c1780
STDERR: time="2015-05-05T12:42:57Z" level="fatal" msg="Error response from daemon: Cannot start container 14a0ebf9bd7700085849e058a658fe1f8afd345b5e5321c2f7dac3a58b0c1780: Error getting container 14a0ebf9bd7700085849e058a658fe1f8afd345b5e5321c2f7dac3a58b0c1780 from driver devicemapper: Error mounting '/dev/mapper/docker-202:1-172158-14a0ebf9bd7700085849e058a658fe1f8afd345b5e5321c2f7dac3a58b0c1780' on '/var/lib/docker/devicemapper/mnt/14a0ebf9bd7700085849e058a658fe1f8afd345b5e5321c2f7dac3a58b0c1780': no such file or directory"
---- End output of "bash" "/tmp/chef-script20150505-15749-r7i4yu" ----
Ran "bash" "/tmp/chef-script20150505-15749-r7i4yu" returned 1
Cookbook Trace:
---------------
/var/lib/aws/opsworks/cache.stage2/cookbooks/lxc/libraries/monkey.rb:20:in `monkey_shell_out!'
Resource Declaration:
---------------------
# In /var/lib/aws/opsworks/cache.stage2/cookbooks/logspout/recipes/install.rb
33: bash "docker-logspout" do
34: user "root"
35: code <<-EOH
36: docker run -d --restart=always -h #{hostname} --name logspout -v /var/run/docker.sock:/tmp/docker.sock gliderlabs/logspout syslog://#{deploy[:environment_variables][:PAPERTRAIL_URL]}:#{deploy[:environment_variables][:PAPERTRAIL_PORT]}
37: EOH
38: end
39: Chef::Log.info('docker-logspout stop')
40: end
Compiled Resource:
------------------
# Declared in /var/lib/aws/opsworks/cache.stage2/cookbooks/logspout/recipes/install.rb:33:in `block in from_file'
bash("docker-logspout") do
action "run"
retries 0
retry_delay 2
command "\"bash\" \"/tmp/chef-script20150505-15749-r7i4yu\""
backup 5
returns 0
user "root"
code " docker run -d --restart=always -h Buzzy-Docker-buzzy1 --name logspout -v /var/run/docker.sock:/tmp/docker.sock gliderlabs/logspout syslog://logs2.papertrailapp.com:<SOMEPAPERTRAILPORT>\n"
interpreter "bash"
cookbook_name "logspout"
recipe_name "install"
end
[2015-05-05T12:42:58+00:00] INFO: Running queued delayed notifications before re-raising exception
[2015-05-05T12:42:58+00:00] ERROR: Running exception handlers
[2015-05-05T12:42:58+00:00] ERROR: Exception handlers complete
[2015-05-05T12:42:58+00:00] FATAL: Stacktrace dumped to /var/lib/aws/opsworks/cache.stage2/chef-stacktrace.out
[2015-05-05T12:42:58+00:00] ERROR: bash[docker-logspout] (logspout::install line 33) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
If I run it again, it usually just works, no error
