Upload large file via Meteor Methods

help
1

I am developing a chat application similar to WhatsApp.
I need to send Video & Audio messages.
Currently I am encoding the selected file using base64 and then decoding it on server side and saving that file on server.
This way it takes lot of time to upload that file, and the base64 string gets too large.

Here is the current way I am doing this:

Meteor.methods({

	sendVideoMessage: function (params) {

		try {

			if(!params || !Meteor.userId()){

				return serverHelperClass.prepareResponse(5, 'UnAuthorized!', {}, 'sendVideoMessage');
			}
		
			if(!params.videoBase64 || !params.videoFormat){

				return serverHelperClass.prepareResponse(3, 'invalid file', {}, 'sendVideoMessage');
			}

			if(!helperClass.validVideoExtensions[params.videoFormat]){

				return serverHelperClass.prepareResponse(3, 'invalid video format', {}, 'sendVideoMessage');
			}

			// Defining Path where video will be uploaded
			var path	 = serverHelperClass.getVideoDirPath();  

			// Preparing file names
	        var fileName = helperClass.generateNewFileName() + '.' + params.videoFormat;
	        
	        // Saving Images to directory
	        serverHelperClass.saveBase64Video(params.videoBase64, path, fileName, true);
	        
	        params.videoBase64 = fileName;

			Messages.insert({ userId: Meteor.userId(), message: params.fileName });
			
			var url = helperClass.prepareUrl(params.videoBase64, 'getVideoDirUrl');
			
			return serverHelperClass.prepareResponse(200, 'Success!', url, 'sendVideoMessage');
		}
		catch (e) {

			console.dir(e);
		}
	}
})

What is the proper way to handle large files?
Is there anyway I can upload as multipart or chunks?
What could be the possible efficient way to upload large file using Meteor methods?

2

For file uploads, better to use a slingshot technique wherein the file is uploaded directly to your static file host like AWS S3 (the file will not pass through your meteor server)

Using evaporatejs for this

1 Like
3

So you mean an Android App will be directly uploading files on AWS S3?

1 Like
4

Yes. We have done it numerous times. A meteor android or ios app is still just a normal web app with the Cordova wrapper

5

Well I am using Native android & ios apps via Ddp libraries, so i hope there won’t be any difference in uploading files directly to AWS S3?

6

Amazon has a native sdk for uploading directly to S3 from native apps so mobile phones can upload directly to S3 without passing through the app server

Technically speaking, the slingshot technique is language-agnostic

7

If you need confidentiality and GDPR compliance etc … you need to use Meteor Files (from Ostrio) or signed/encrypted URLs with a CDN (Like S3/Cloudfront).

For a server signed uploads with 0 confidentiality (public URLs) you can use https://github.com/Lepozepo/S3 or a newer but more customized version which adds Cache and Expiry - https://github.com/activitree/s3up-meta (you would need to take your own fork from it)

Both packages feature: upload 1 file, multi file delete files, and the second one (from Activitree) gets ‘abort upload’ in a day or two, perhaps tomorrow. For chat, I’d go with Meteor Files.

1 Like
8

Does Meteor Files upload files in chunks or multipart? How would be the performance if Android Native App uploads a large video file?

9

I am not sure you can still upload non-multipart with S3. Anyway the answer is yes, it is multipart. You will need a policy on the S3 storage to delete incomplete multipart uploads every x days.