Running a own Kadira instance Update: now with a guide!

No one else has faced this issue?

Their pricing is something to look out for. I was quite excited about it, but I also use auto-scaling with Heroku, and if you peak above a certain number of hosts for even a second, you’ll pay for it for the rest of the month. And this isn’t mentioned anywhere in their docs, nor is it how the original Kadira pricing worked, so just keep this in mind.

Hey thanks a lot for the contributions guys, this was very helpful. Spun up an instance this morning using the docker images and now I’m getting some useful metrics in. :grinning:

What did u use, there are too many links in this thread. Did you use a mongo hosting service or are you running mongo yourself?

Also does anyone know how to limit Kadira to keep only last X days of logs - the db quickly becomes huge.

I followed the guide by markoshust/vyvegard, database is hosted on compose.io

Also FYI if anybody’s interested in using Kadira Debug, it’s available at /debug on your Kadira instance.

1 Like

I really recommend to think twice before using Compose.io or mLab.com, because it will get very expensive, since Kadira eats a lot of disk space in the database. I am running a Droplet with mongo now instead, which is on 20GB which is plenty for 3 kadira apps.

Thank you, that was really helpful! Did not find this myself.

yeah, i’ve had it running for a week across a few instances and it’s up to 1GB already. will probably either move it to something self hosted or figure out a strategy for pruning it every month or so. having that historical data is probably part of the benefit of running it though.

Yes I do the same now. How do you purge old database records? I only want to keep 1 week’s data around. The real Kadira must’ve had some scripts to do this since I don’t see anything in the UI to do this.

Are the required indexes documented somewhere? The codebase apparently does not create indexes by itself.

@vladgolubev Great Job, with your docker-compose getting kadira up and running is really straightful!

But I cannot find anywhere the repos about your modification, I’m really interested in the ui repo, because the other two are quite unmodified.

Can you publish your kadira-ui sources anywhere? Thank you!!

Hi, I’ve had the same issue with kadira monitoring used for several apps after a month of usage. I’ve checked and noticed following things:

  1. Size of the DB (kadiraData) has become quite large (> 2.6 GB).
  2. Current operations analysis shown a big numYields values.

From that I’ve concluded that all DB data cannot be kept in RAM (vm has only 4 GB of RAM) and some kadira queries have become very slow due to multiple disk read operations.

I haven’t found any ready solution for cleaning up old data, so I’ve made two things:

  1. Changed billing plan in kadiraApps db from business to standard (to keep history for 1 week only). However this might not work if the app has more than 5 hosts
  2. Manually removed records older than 1 week (approx) from each collection in kadiraData DB.

Solution worked for me. Now monitoring is fast again.

good advice! :slight_smile:
i had to reboot my db instance yesterday because it became unresponsive. seems alright since then. running it on a super cheap lightsail…

Ah one thing that might be useful to note for others is I noticed today if you reboot your mongo db, you have to restart Kadira Engine or your app will fail trying to connect to it. Kadira UI continues working fine but the Engine will stop logging…

cool, thank you for the update. Very useful

FWIW, I made a more detailed docker-compose.yml that includes an nginx-proxy to provide https access to the kadira-engine (which is necessary if your Meteor app is running on https), and includes a built-in MongoDB server:

Thanks to @vladgolubev for the dockers that makes this possible!

4 Likes

( @edemaine: Wow! I was just watching your MIT lectures in algorithms! How cool that you’re a Meteor dev! :slight_smile: )

thanks a lot for this thread. i really would like to learn more on how to do that, after all, that’s why i’m here now. i’ve been sitting home doing nothing but searching for hgh reviewer because of health issues and i thought about learning more so thanks guys. i appreciate it a lot and hopefully i can ask my questions here later if i would have some…

So I’ve had some issues where over time my Mongo instance will just choke out. Previously I was able to sort it by just restarting the Mongo service, but now it’s basically timing out.

I’ve got it running on a Lightsail instance with a single vCPU and it’s just thrashing the CPU.

Anybody else having issues running Mongo on a relatively small instance, particularly in relation to whatever data crunching Kadira is doing? I haven’t investigated yet whether it’s the map/reduce stuff or the UI querying directly…