Meteor metrics reporter package

announce
1

Hi everybody, today I have completed developing a new package which will collect and report Meteor metrics. You can use it with any reporter and any metrics software: signalfx, graphite, prometheus, no matter.

Links:
Atmosphere
GitHub

You can use it in conjunction with meteor-elastic-apm where apm will trace code execution, while meteor-measured will collect important metrics.

Please let me know what you think about this package. Thanks!

Copying Readme here:

Meteor-measured

The library will collect meteor specific metrics using node-measured library.

It’s metrics platform agnostic so you can use it with any metrics software you want: signalfx, graphite, prometheus, etc…

Install

meteor npm --save install measured-core
meteor npm --save install measured-reporting

meteor add kschingiz:meteor-measured

Requirements

The library was well tested on Meteor 1.8 version, maybe it will work on older versions.

Usage

You will need to build measured reporter and pass it to meteor-measured:

Step 1:

import { SelfReportingMetricsRegistry, LoggingReporter } from 'measured-reporting';
const registry = new SelfReportingMetricsRegistry(
  new LoggingReporter({
    defaultDimensions: {
      hostname: os.hostname()
    }
  })
);

Step 2:

import meteorMeasured from 'meteor/kschingiz:meteor-measured';

meteorMeasured(registry);

meteorMeasured arguments

meteorMeasured(registry, customDimensions, reportingInterval);

Reported metrics

Those metrics collected and reported

Sockets Metrics:

  1. Open sockets: 10 sockets open
  2. Live data sockets: 8 sockets uses livedata

Session Metrics:

  1. Sessions count: 8 meteor sessions

Pub/sub Metrics:

  1. Subscriptions count: 100 subscriptions
  2. Subscriptions count for each publication: testPub: 20 subs, notTestPub: 80 subs
  3. Published documents: 20 docs published
  4. Published documents for each collection: 10 docs of TestCollection published, 10 docs of NotTestCollection published

Observer Metrics:

  1. Number of observers: 20 observers created
  2. Number Observer drivers: Oplog drivers: 10, Polling drivers: 10
  3. Number of documents for each driver: TestCollection published by oplog driver: 10

Not reported metrics

Please note that the library will not collect and report those metrics:

  1. Node process metrics: CPU usage, RAM usage, etc…
  2. OS metrics: average free memoty, etc…

They are NOT Meteor specific, if you want to collect them, install and use measured-node-metrics:

meteor npm i --save measured-node-metrics

Reporters

Those reporters are available for usage:

  1. LoggingReporter
  2. SignalFx Reporter

You will need to develop your own reporter implementation if you want to report to graphite, prometheus, etc…

TODO

  1. Tests
16 Likes
2

Is there a way to get this to talk to Elastic so we could have all the APM data and these metrics displayed in one place ? Or are you already doing that ?

1 Like
3

Yes, elastic apm already uses node-measured library to send node metrics to the apm-server. You have 2 ways to integrate with elastic:

  1. You will need to get elastic metrics registry from agent and pass it to meteor-measured:
    https://gist.github.com/kschingiz/7ab422d577bcae49f4bd9b421190ebc2
    The solution is a little bit tricky, but at least it works =)
  2. Implement your own elastic reporter, you can just copy paste their one:
    https://github.com/elastic/apm-agent-nodejs/blob/master/lib/metrics/reporter.js
    Please note that their reporter takes transport as an argument, you can retrieve that transport from agent itself:
agent._transport

Please let me know if one of the solutions works for you. Thanks!

PS: I am not kibana expert, I was not able to visualize data I have sent with meteor-measured, but I see that at least data is taken and stored in elasticsearch. If you provide me steps to draw diagrams, that would be cool.

1 Like
4

Ok, I think I follow. So elastic-apm (not your meteor-apm) is already sending the basic metrics to elastic but if we want your extra meteor specific metrics we need to tie the two of them together ? I’ll see if I can find some time to investigate later.
I’m a noob with the entire ELK stack so can’t really help right now but I’ll be trying to do some basic tutorials to get a nice dashboard in Kibana at some point.

5

Yes, elastic-apm already sends basic metrics like CPU, RAM, pool, etc… you can find full list of sent metrics here:

My library sends only meteor specific metrics.

1 Like
6

Would it make sense to integrate the two packages then ? I can’t think of a case where people would not want these extra metrics if they are installing the apm package. Or if the apm package detects the metrics is installed it hooks it up like in your gist ?

7

Yes, I am planning to integrate them, I wanted to develop generic metrics library first which can be used with any software not only elastic. Next version of meteor elastic apm will come with meteor-measured by default.
Just wait for it =) As an workaround you can for now use the solutions above.

1 Like
8

That’s great! Nice work :+1:

1 Like
9

@marklynch and eveybody who was interested in meteor-elastic-apm integration with meteor-measured: Meteor-elastic-apm with meteor metrics