Cloud storage and Image manipulation (cross origin problem)


#1

A while ago I experimented with different upload packages for Amazon S3 and thanks to other peoples work everything was working just fine.

Then I discovered cropper and the meteor package jonblum:jquery-cropper and thought I can integrate the two. So I had my demo upload images to S3 and in a later step crop the image to upload a thumbnail of choice.

This solution has stopped working, however, due to the cross origin restrictions between the browser and image store (S3 in my case) somewhere in-between browser updates. I am on latest Chrome, Safari and Firefox and neither browser allows the canvas.toDataUrl method on the image coming from S3.

Does anybody here now how to deal with this or how to solve it?


#2

Do the image transformation on the server.

You can use:


#3

I use edgee:slingshot to allow direct client-S3 uploads

on AWS, I have a lambda worker listening to S3 event (file uploaded), which in turn creates a thumbnail

you could add a cropping stage by uploading extra metadata to the S3 along with the file, so the lambda worker can interpret the cropping parameters and act accordingly.


#4

I use the fs package - https://github.com/CollectionFS/Meteor-CollectionFS which supports s3. Slingshot is great because you don’t incur the upload bandwidth but (could be wrong) it doesn’t look like it provides a private/secure download stream option. You can achieve that by making your s3 files public and that excludes a lot of private storage use cases.

The fs package is nice because it offers a private file access (download/view) via fs http handle other than making the s3 file public. The downside is, you incur download bandwidth cost between your meteor server and the browser.


#5

I am using edge:slingshot, and am transforming the image during upload so I have an original and a derivative.

The problem is if I want to change the derivative after upload.


#6

my use case is public files so I’ve never delved into making them private, but this is a matter of S3 which can be handled regardless of Meteor

I started off with CollectionFS, but:

  1. I didn’t need a whole db collection just for keeping a reference to some file in storage, which forced me to “join” collections for simple cases (i.e. publishing a user’s property and its photo)

  2. my intention was to get the file in the cloud at the first place, so why bother with a package which inherently stores files locally (I know there’s a S3 extension)

just a final note - I think this package is a great effort by @aldeed, it just didn’t serve my needs in this case.


#7

then why don’t you try Lambda?


#8

OK, tying to figure out how to give an end user the freedom to choose how to crop an image and to have AWS Lambda do the work.


#9

Correct me if I’m wrong but he wants the user to decide how to crop the image, not just create a smaller thumbnail of the original image. Lambda can’t do that.

OP, this used to work? I’d say your problem is either on your S3 bucket configuration (it needs to be setup to allow CORS) or possibly a configuration in the browser-policy if you’re using that package.

I use my own package (mrt:s3policies). It allows you to create both upload and download policies on the server and pass them down to the client, allowing the file to be private in S3 without having your server incur the bandwidth.


#10

sure it can, if you add metadata to your S3 upload
Lambda can read this metadata when an S3 event invokes it


#11

Correct, automatically cropping and uploading is not the issue here. I had been experimenting with different buckets, and one of them had the right CORS but was missing the policy. Additionally I fell victim to autocomplete in my editor, where I had defined a function getDataUrl but the canvas object has a method getDataURL, so I was dealing with exceptions. Everything works again as desired.


#12

I stand corrected. Thanks, I’m by no means an AWS expert. Haven’t played with Lambda yet.


#13

Hey @jamgold,

can you share the code for how you’re doing cropping while uploading to s3 using slingshot package,
I tried this before but no luck.


#14

+1 Yes, that would be very helpful for learning. I too had bad time with the uploads


#15

I am planning to create a package for this. Currently I am using meteor-uploader and not slingshot, though. The other tool is the amazing jQuery library cropper.


#16

+1 for new package, 20char


#17

@jamgold, +1 for a package :- )


#18

I have a working package with Slingshot and Cropper, but suddenly Safari is acting up and doesn’t load cross origin image URLs, Firefox and Chrome have no problem. I don’t know what has changed, because similar code worked earlier this week, und now suddenly it no longer does. Pulling my hair out.


#19

when I was working with uploadings too using slingshot, everything works fine in chrome and firefox but safari had some issues


#20

The weird thing is that uploading is not the issue. It works great. My problem suddenly is with Cropper.