Any reason not to store images in Mongo?

[aaabbbccc]

I have code like this:

MeteorCamera.getPicture(function (error, data) {
            if (error == null) {
                var url = 'url(' + data + ')';

                $(".photo").css('background-image', url);
            }
   
            Meteor.call("newPhoto", {error: error, data: data});
        })

The newPhoto method just inserts the data object.

The method on the server inserts the new record into Mongo.

It’s half-baked as you can see (for instance, it does not check for errors).

What are the pros and cons of inlining an image like this vs. storing it in a filesystem or S3?

[garrilla]

AFAIK there are no show-stoppers, but, off the cuff, some things to consider:

MongoDB has a 16mb limit for docs - you can get around this with gridFS or you could limit users to less than 16Mb but this introduces a checking stage

Processor cost - your server has to process the image, add to this if mongdb is also on the same server

Storage cost - if you’re buying dedicated MongoDb storage space or even dedicated VM storage space, it usually costs a little more than using a storage bucket on AWS/Google/etc

[aaabbbccc]

Thank you! All good points.

Currently, I don’t need to store images larger than 16M, but that can change; and yes, I need to check.

Will the processor cost be greater than with CollectionFS?

At present, I am running mup with Mongo on the same tiny E2 instance.

Storage cost - is S3 cheaper than EBS, and if so, why?

Edit: About S3 vs. EBS, found this: http://www.tomsitpro.com/articles/cost-of-the-cloud-book,2-694-2.html

Perhaps the biggest drawback of the S3 storage system is lack of support for file systems. Also, S3 retrieval times can be slow when compared to EBS and can vary between requests. S3 will, however, support up to 5 TB of data in a single object and users can store as many objects as they like. EBS volumes are limited to 1 TB and can be attached to only a single EC2 instance. If you want to use the same EBS volume on multiple EC2 instances, you will have to replicate the EBS volume and attach the replicas to the other instances. This is a reasonable solution for applications that support primarily read operations, such as business intelligence applications

[garrilla]

I can’t say if processor cost will be more - its a test you might want to run… but logically speaking, if everything end-to-end is on your processor, the cost is likely to be greater.

The reason Sole storage is cheaper is that 1mb image, costs 1mb. A 1mb image stored in Mongodb costs 1Mb + data structure and addressing space. This might be minimal on 1 image, but over several thousand it builds up a cost differential. Like i say, its something to consider.

[jorgeer]

Read this article a while back. Found the arguments against it pretty compelling.

revsys.com

[wickedmuso]

I went and had a quick read of the suggest post and I think it only skims the surface and doesn’t look at potential underlying technologies that can support storage of “files” (e.g. blobs) in a database.

The Mongo GridFS is designed for this and I have been using it via the Meteor File Collection package for a production application to store composed documents and, now, for ALL the files in the system that can be requested (including branding images, replaceable documentation - like Privacy Statements, etc.) and find it performs well.

vsivsi/meteor-file-collection

[Matteo_Saporiti]

The Mongo GridFS is designed for this and I have been using it via the Meteor File Collection package for a production application to store composed documents and, now, for ALL the files in the system that can be requested (including branding images, replaceable documentation - like Privacy Statements, etc.) and find it performs well

Quoting from MongoDB site:

In some situations, storing large files may be more efficient in a MongoDB database than on a system-level filesystem.

• If your filesystem limits the number of files in a directory, you can use GridFS to store as many files as needed.
• When you want to access information from portions of large files without having to load whole files into memory, you can use GridFS to recall sections of files without reading the entire file into memory.
• When you want to keep your files and metadata automatically synced and deployed across a number of systems and facilities, you can use GridFS. When using geographically distributed replica sets, MongoDB can distribute files and their metadata automatically to a number of mongod instances and facilities.

Basically they wrote GridFS and they still say that if you don’t fit one or more of these three use cases you’re better off storing the files in the filesystem,
That’s a pretty compelling argument against using GridFS just because you can.

[wickedmuso]

In my case, the system has to be able to work across geographically distributed replicas and I really didn’t want to deal with some files as part of the Meteor build, others sitting out on an S3 instance somewhere, and then others (that had to be in the replica) all having completely different maintenance mechanisms.

So, I decided my longer-term use-case fits the 3rd point. It wasn’t a random decision but “need” and “convenience” based. I was previously rolling my own “store the blob in the database” system and GridFS was a clear ‘out-of-the-box’ plugin replacement.

[rashmimhatre100]

DO NOT STORE IMAGES TO MONGO. Your DB will become heavy at production site. Data will take more time to come on production. Users will get frustate.