Suggestions on structuring an images collection

#1

Lots of things to consider here. For instance, I’ll probably use this package:

https://peterpalau.github.io/react-bnb-gallery/#getting-started

It’s going to be used in a real estate application.

Probably some other considerations I’ll add as i think of them.

I’m thinking

  • imageUrl
  • thumbnailUrl
  • title
  • caption
  • fileType (png, jpeg, etc)
  • base64 (maybe a fuzzy image version to do progressive image loading?)
  • homeId (references a collection of homes, or maybe store the image _ids on the home)?

any other ideas while I’m at it?

Also, I was going to store these in firebase storage. In the past I used cloudinary which made grabbing thumbnails a little easier… anybody think I should bail on firebase storage an go back to cloudinary?

#2

I’ll tell you what I use:
AWS S3 to store the original images and their thumbnails.
A collection UserFiles in mongodb where I save the s3 key, the userId of the owner, creation date and some metadata like ‘purpose’. To determine if it’s used for an avatar or a post.
(That base64 might be a good idea. I’ll consider it)
I would use this exact same model if I was creating a real estate app.

3 Likes
#3

Do you generate the thumbnail client side?

#4

I’ll second @bluray’s use of S3, and add to it cloudfront, so you can secure access based on cookies.

Our approach differs slightly, we use slingshot for direct S3 upload, lambda to generate thumbnails (though we could do this client side too).

We don’t have an “images collection”, instead we store an imageKeys object on any entity (about 15 different entity type) that has an image, this tracks the original and any thumbnail versions, then we have a helper that takes an imageKeys object and returns either the URL for the requested size, or the URL for the original if the requested size isn’t available.

We originally used an images collection for this (based on MeteorFiles) but found it unnecessarily messy

2 Likes
#5

To secure access with cloudfront and cookies, do meteor and cloudfront need to talk to each other?

I’ll probably store a images field on entities (an array of objects) instead of a separate collection.

#6

Not exactly talk to each other. You create a signing key pair, your private key resides on your Meteor server, which also specifies an “access policy”, this can be a generic policy, or can be user specific on login - we have multiple companies using our system, so we ensure that every company’s images reside at /companyId/... then a user’s access policy is that they can only access files within that “folder”. You sign the access policy with your private key, when the user requests an image resource, it sends the cookie with the request and AWS uses the public key to verify the requset.

1 Like
#7

We opted not to track the thumbnail versions, as we generate them on-the-fly with a Lambda function via API gateway. Instead, we put each image in a unique folder along with all it’s differently sized thumbnails.

If the image is removed, we just wipe that folder from S3.

#8

Depending on your resources, imgix is an amazing solution for images. If you want to optimize for a different thumbnail size, instead of having to update a lambda or other function, you just pass a different param string into your url.

Tons of other great things about it also!

We use (in general) a similar approach: get a signed url, upload straight from the client to s3, get the key and save that into a property on a document in the Photos model.

#9

How do you remove items within a folder in S3 - do you have a function that fetches all keys for a given prefix, then deletes those?

#10

Pretty much yes. First I list all objects with a prefix I get from the “master” file (prefix = object key without the filename in the end).

For a file…

blaaa/blooo/foo/bar/xkSzmMr5/cat.jpg

the prefix becomes

blaaa/blooo/foo/bar/xkSzmMr5/

and in return for the list call I get objects with keys:

blaaa/blooo/foo/bar/xkSzmMr5/cat.jpg
blaaa/blooo/foo/bar/xkSzmMr5/520x/cat.jpg
blaaa/blooo/foo/bar/xkSzmMr5/1400x/cat.jpg
blaaa/blooo/foo/bar/xkSzmMr5/2500x/cat.jpg

Then I just delete all those keys.