Step by step for third party log processing?

help
1

My app has been running for some time on meteor galaxy. Its been generally good!

I’ve been having some log errors which I would like to start being alerted about so it is time to setup something more like ELK (Elastic search, Logstash, Kibana).

The meteor galaxy page lists very brief instructions on getting galaxy logs into elastic search. I’ve been trying to get it going but wish there was a step by step guide to get this done.

For example, setting up elastic search on AWS was pretty straight forward. Configuring security so that galaxy can access the elastic search instance I setup is more involved. The built in security options available do not match the galaxy options.

I am now looking to just use a logstash approach and put the code in my app but that will miss the node.js errors such as Unhandled Promise Rejection Exceptions which I would like to capture.

My Ask:

  1. Step by step instructions to setup galaxy logging with AWS elastic search that is current (2020)
  2. The same for another service I’ve used in the past such as logit.io (another hosted ELK stack provided)

Or event better (but obviously a lot more work)
3) Galaxy supporting ELK stacks for hosted apps! That would be truly amazing!

2

Update - I was able to setup a solution. I added code to my app to capture my app logs and any other console log in the app itself. It then sends it via https to a logstash http receiver on ec2. Logstash then feeds it into aws elastic search with all the security signing required.

If you have a need for something similar, let me know.

I wanted to suggest that galaxy devs provide an HTTP web hook mechanism for the logs instead of the current elastic search interface. This way the logs can be sent to anything for processing. Using logstash to process this raw web hook traffic and ingest it into elastic search/kibana is straight forward.