Currently my group runs a meteor app with hundreds of thousands of images that are all very large full size. We should have done this a long time ago but we are in need of a way to optimize them to help loading times. I’m looking for a solution to be able to save the image in several sizes when the user uploads it from our app (ex: full-size, medium, thumbnail) and also auto rotate and allow user to rotate. We use Amazon S3 to host all of our images. We also need a way to convert all of the existing images into these size formats from server side.
I tried implementing something a while back and it was unsuccessful. I set up imagemagick on our server but had trouble getting this to work in production because the image was being saved temporarily on server memory to process but this was causing crashes because of limited amount of memory. I have little experience with this kind of thing.
My second thought was to use HTML canvas to resize the images. This would work, I think, for newly uploaded images. But I am still searching for a way to process the existing images as well.
I have considered:
Maybe AWS has a built in way to process them. I wouldn’t mind doing it that way.
Some sort of Meteor/node package that can help with this.
Setting up another server just to process images.
Using some third party image processing.
If someone can give me some advice so I can get the ball rolling that would be so helpful!
just today i had the same question and i decided to give https://github.com/thumbor/thumbor a try.
It works nice so far, i could host it on our kubernetes-cluster. It should also be easy to install it on some VM.
It’s service where you can specify resizing, filters, etc. in the url and pass the original url, you will then get the transformed file (it also caches the results of course).
Also keep in mind, that your server might have a limit on maximum upload size and that that mobile users might want to save bandwidth. It is therefore also important to limit the file-size prior to upload and resize the images before upload. E.g. you can use something like: https://github.com/nodeca/pica
1 Like
we use a lambda function for this - technically you can configure S3 to trigger the lambda function whenever you have a file uploaded, but if you’re storing the modified images in the same bucket it gets confusing depending on how you’ve organised (because uploading the thumb might trigger the lambda function again) we decided to just trigger this manually whenever the user uploads an image with a meteor method.
Depending on how you track these images (e.g., in a mongo collection) you’d also need to track either whether a thumb exists, or what the S3 key of the thumb is
To convert all existing images to this structure you’d just trigger the lambda function with the required arguments - probably something along the lines of (key, width, height, suffix/outputKey).
1 Like
We’ve been using Cloudinary for asset storage and manipulation across all of our apps for awhile now - it might not be the best solution for very large scale applications, but their api allows you to upload and manipulate image sizes, generate additional sizes, shapes etc. very quickly and easily.
1 Like
This looks extremely interesting. I wonder, do you know if I can run this from a galaxy server where I currently host my app? Or would this need to be installed on a separate server?
This sounds like it may be the perfect solution to transform the existing images. I kind of had something like this in mind. TY for pointing it out to me C: I will look more into this for sure.
Not sure we would want to transfer to a new image hosting service, do you happen to know if Cloudinary can be used strictly for image manipulation and then stored in S3? This sounds like a really neat service
To be honest, I kind of doubt it - I think you have to upload an image to then have the ability to alter it. You may be able to leverage it as a temporary storage and then reupload the modified images to S3? Just an idea.
Not the best solution, but one that is acceptable in some use case, use the client for processing the image as most of the computers and smartphones are more powerful than servers. I use Slingshot to load to S3 and use the onBeforeUpload to resize before loading.