loses oplog for 12+ hours if you want to downsize. Alternatives?

Last month we decided to take some data out of our Compose database and archive it separately. This dropped that database down from a few GB to ~30MB. We also executed the resync command remotely from the console of one of our computers. This drop in size was enough to go down a pricing tier, but we noticed Compose was still charging us the higher tier pricing, so we submitted a support request, titled “BILLING BUG? Database size wrong”

This was their response:

Hi Larry,

The way MongoDB works, unless you perform a resync, size on disk is not reclaimed even if data is removed from a database. This is a technical limit imposed by the database engine itself. As a result, if you drastically reduce your data size you’ll need to perform a resync through our UI. As a result of how MongoDB handles permissions, you’ll not be able to perform a resync through the application layer or the mongo shell; only through our UI. If you plan to have large data sets that scale down in size frequently, you may want to look into using capped or TTL collections to avoid this scenario.

You can resync your replica set from within the UI by going to a deployment, clicking the “settings” tab, and looking for the “Resync Secondary” section. The one caveat is that after you resync a secondary, you have to trigger a stepdown for your replica set which means that your new primary has no oplog. Our safety mechanism will not allow a resync to be triggered on a secondary through the UI if the oplog has less than 12 hours of data. As a result you’ll need to wait 12 hours or a little more for the oplog on the new primary to regain enough play-time before triggering the final resync.

Here’s the outline for the steps to resync your replica set:

  • Click resync, and your current secondary pulls fresh data from the primary. You can watch your live logs on the secondary to see how that process is progressing.

  • When the secondary is done syncing, click stepdown to elect a new primary. You can check server status in the UI to see if the secondary is in a startup / recovering state or is in a healthy secondary state to determine if you can perform a stepdown.

  • Wait 12+ hours for the oplog on the new primary to fill with data.

  • Click the resync button again and your new secondary (the former primary) will be resynced and your entire deployment will be reduced to its smallest possible size on disk.

Let us know if we can help with anything else. Remember, it is not enough to resync once. You’ll need to resync -> stepdown -> resync to completely reduce your deployment’s billable amount of space.

Of course, before submitting the ticket we knew about MongoDB’s resync quirk and ran the resync command directly into the mongo shell… and that’s what got compose to finally display the reduced database size (just wasn’t effecting the pricing), but apparently we have to do it through their UI and then go through some other hoops and lose oplog.

I feel like we’re being locked into a higher pricing than we need and the only way to minimize the database costs going forward is to lose oplog for half a day, and enforce some strict cap. Neither sound all that ideal. Sure we could do this once but unless we create a capped collection or completely rewrite the way our app stores data, we’d have to go through this process again.

At Compose’s pricing, which starts at $31/mo, this is kind of not what I’d expect. Sure you can go with their MongoDB classic that starts at $18/mo, but that uses an older version of Mongo and doesn’t take advantage of the new features like atomic operations. And even at $18/mo starting price there are competitors who could be offering something better.

Is there a better database host? How’s MLab on this?