Deploying meteor app with file storage capability

I’m building an app targeted at mobile devices with the awesome Meteor-files library. So far everything is working fine in my machine and on android studio.

I did deploy the app to heroku early on in the process and it worked fine. But when I added and configured the Meteor-files library, the app errors out when deployed to heroku.

A look at the logs showed that this line storagePath: '/Meteordata/uploads/files is the problem.

The relevant portion of the logs is

meteor://💻app/packages/ostrio_files.js:412
2019-02-08T20:28:12.605095+00:00 app[web.1]: throw new Meteor.Error(401, `[FilesCollection.${self.collectionName}] Path "${this.storagePath({})}" is not writable! ${error}`);
2019-02-08T20:28:12.605097+00:00 app[web.1]: ^
2019-02-08T20:28:12.800766+00:00 app[web.1]: Error: [FilesCollection.audios] Path "/Meteordata/uploads/Audios" is not writable! Error: EROFS: read-only file system, mkdir '/Meteordata' [401]
2019-02-08T20:28:12.800770+00:00 app[web.1]: at FilesCollection.fs.mkdirs.error (packages/ostrio:files/server.js:274:15)2019-02-08T20:28:12.800773+00:00 app[web.1]: at mkdirs (/app/.meteor/heroku_build/app/programs/server/npm/node_modules/meteor/ostrio_files/node_modules/fs-extra/lib/mkdirs/mkdirs.js:43:19)
2019-02-08T20:28:12.800774+00:00 app[web.1]: at mkdirs (/app/.meteor/heroku_build/app/programs/server/npm/node_modules/meteor/ostrio_files/node_modules/fs-extra/lib/mkdirs/mkdirs.js:43:19)
2019-02-08T20:28:12.800776+00:00 app[web.1]: at xfs.stat (/app/.meteor/heroku_build/app/programs/server/npm/node_modules/meteor/ostrio_files/node_modules/fs-extra/lib/mkdirs/mkdirs.js:55:43)
2019-02-08T20:28:12.800777+00:00 app[web.1]: at /app/.meteor/heroku_build/app/programs/server/npm/node_modules/meteor/ostrio_files/node_modules/graceful-fs/polyfills.js:284:29
2019-02-08T20:28:12.800779+00:00 app[web.1]: at FSReqWrap.oncomplete (fs.js:152:21)
2019-02-08T21:30:33.031657+00:00 heroku[web.1]: Process exited with status 1
2019-02-09T01:28:59.961593+00:00 app[api]: Deploy 82b3a2e2 by user orjichidi95@gmail.com
2019-02-09T01:29:01.415395+00:00 heroku[web.1]: State changed from crashed to starting
2019-02-09T01:43:53.299057+00:00 heroku[web.1]: State changed from crashed to starting
2019-02-09T01:44:21.097337+00:00 app[web.1]:
2019-02-09T01:44:21.097351+00:00 app[web.1]: meteor://💻app/packages/ostrio_files.js:412
2019-02-09T01:44:21.097353+00:00 app[web.1]: throw new Meteor.Error(401, `[FilesCollection.${self.collectionName}] Path "${this.storagePath({})}" is not writable! ${error}`);
2019-02-09T01:44:21.097354+00:00 app[web.1]: ^
2019-02-09T01:44:21.292652+00:00 app[web.1]: Error: [FilesCollection.audios] Path "/assets/app/uploads/Audios" is not writable! Error: EROFS: read-only file system, mkdir '/assets' [401]
2019-02-09T01:44:21.292657+00:00 app[web.1]: at FilesCollection.fs.mkdirs.error (packages/ostrio:files/server.js:274:15)2019-02-09T01:44:21.292683+00:00 app[web.1]: at mkdirs (/app/.meteor/heroku_build/app/programs/server/npm/node_modules/meteor/ostrio_files/node_modules/fs-extra/lib/mkdirs/mkdirs.js:43:19)

I believe the log is clear enough that I cannot write to Heroku

But it seems that the only way to use the library is to save temporarily on local file system prior to moving to AWS. How can I resolve this issue?

I deployed as described by this tutorial https://medium.com/@leonardykris/how-to-run-a-meteor-js-application-on-heroku-in-10-steps-7aceb12de234

I don’t know if that has anything to do with why mine is failing.

Thanks

Well, I should put this here just in case it trips up someone else.

My problem was that I was trying to access the root of the heroku drive /Meteordata/uploads/files. The leading / is a problem. I changed my destination to assets/Meteordata/uploads/files and everything works fine.

1 Like

I’m not sure if it’s a good idea to store files on a Heroku “whatever they call it” 's file system. Isn’t it ephemeral? If you update your code, won’t those files be lost?

Sure. Thanks. I immediately move the file to S3 upon successful upload.

Ah, that’s good. You may want to give users a federated (temporary) token and use S3’s uploader in the client. If these are big files, it should go a lot faster for the client using S3’s bandwidth instead of your Heroku inbound bandwidth.

I see that now. I think you shouldn’t use this library :slight_smile:

I’d really love to get pointers on how I can accomplish this even if I end up not using it. I’ll be uploading files about 20MB in size. But what’s to keep them from getting larger later on?

I’d really love to learn more. I’m always open to learning.

In fact, one of my problems is that I read about a lot of things that I have almost absolutely no idea about, no matter how far I think I’ve gone in my studies. Can be pretty frustrating.

I await your reply. Thanks

I couldn’t find a good tutorial online, I just remember doing something similar for Meteor video uploads in 2013! It’s unfortunate that it’s so complicated, despite being so good. There’s a whole github thread about the solution I’m describing here: https://github.com/aws/aws-sdk-js/issues/1603. Here’s an outline:

  1. Import the AWS Javascript SDK in the client.
  2. On the server, you generate temporary credentials that make it possible for the client to upload (“put”) files to a specific prefix on your S3 bucket.
  • In the past, I used Amazon’s “STS” to generate these credentials because it gives you a single string you just pass to the client that’s compatible with all AWS SDK authorization functionality.
  • But people also use something called “presigned” URLs, which I am dubious of.
  1. Pass the credentials back to the client. This can be as simple as writing the credentials to a field on the user document (don’t be one of those people returns data in a method :slight_smile: ).

  2. Upload the video to the S3 bucket + prefix + file path using the AWS Javascript SDK on the client using the token generated from STS or the “presigned URL”. s3.upload is what you’re going to be looking for.

  3. Notify the server that the upload is complete, which will then create the entries in your database for finding the files, etc.

  • The most durable way of doing this is with S3 events (https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html). I believe this integrated with the media transcoding service Amazon provides which is great for making immediate playing videos compatible with <video> element, so it’s worth looking into to get the tip top best experience.
  • Otherwise, just call a method. There are a variety of reasons the end user’s device can finish the upload but fail to make a method call or vice versa. However it will be so much simpler.

Asking people to upload files in a mobile browser on Android is going to be horrible. Like someone’s Boost Mobile connection in some 1 bar service area in ironically the most dense populated places in the world just won’t ever succeed in uploading a large file. So be prepared for a constant stream of bugs from people through no fault of your own!