Good evening,
I’m using Meteor’s Mongo to store hundreds of documents (eventually thousands). each document consists of a title, a link and a content field, the content consisting of at least 100 word, at most a thousands (or several). So far so good. But imagine my surprise when, trying to port my app to mobile I notice it is strangely slow or unresponsive. After some poking around I discover that having around 300 documents my collection size is 65mb!, and I am still ways from the number I intended.
Hence my question: Isn’t Mongo’s management of space a little too permissive? Meaning isn’t there some way to compress data, or at least to chalk up some of it so that my collections can be safely stored offline (using GroundDB) on the end-product mobile app?
Just to give some context: The App I’m developping is meant to function almost entirely offline. The once in a while opportunity where the user is online, collections will simply be synced again; meaning I need my collection(s) to be stored on the client-side, and they need to be as small as possible for obvious reasons.
It seems to be exactly what I need, yet when I try: db.runCommand( { compact : 'mycollectionname' } )
I get an error message:
{
“ok” : 0,
“errmsg” : “will not run compact on an active replica set primary as this is a slow blocking operation. use force:true to force”
}
I can’t make sense of it. Google isn’t helpful much either.
First thing I tried, prints {ok: 1} but when typing db.mycollection.totalSize(), the size remains the same.
I have also tried http://stackoverflow.com/a/24723068 answer, to no avail. Might it be because I’m running my app on Cloud9 and not locally? (I had to run bandwidth-consuming tests, that’s why)
Yeah - I’ve just run a test and compact doesn’t really do a lot for me. However, I’m a little puzzled regarding the original issue. If documents are copied to a mobile device, the memory used will be representative of the document size(s), not the database size. So, @unforgiven: how is your data getting onto your device?
It will but only if you’re using WiredTiger - from the docs:
Rewrites and defragments all data and indexes in a collection. On WiredTiger databases, this command will release unneeded disk space to the operating system.
I’m still somewhat new to the whole of Meteor, so you’ll forgive me if I’m wrong, but I believe that by subscribing to my collection (in which my documents are stored) these will be copied to my mobile device, and in so doing the size of my collection is directly representative of the size that the documents occupy on my mobile device, or am I wrong?
As for db.repairDatabase(), sadly it didn’t have the intended effet.
All in all, is storage on mobile device simply doomed by the fact that Mongo is used, and that there is no apparent way to make it use less space? If someone here has some experience with developing a mobile app App using large amounts of offline data, I would be more than happy to listen to his/her thoughts on the subject.
What you get back on the client is effectively a bunch of POJOs and while there is an overhead to any object, it’s not going to be the same as the database overhead (which as we’ve seen includes unused, preallocated space).
If you want to bring the client side usage down even more, but at the expense of transcoding, you could store JSON/EJSON strings in (say) application storage. It’s even more obvious what the size of a string is, making the storage requirements much clearer.
