Not Able to Create Directory for uploading Images in EC2 AWS (MUP JS)

Hi Everyone,

I am trying to upload the image to my EC2 instance itself, for this I created a post API and used fs package. I want to upload images in public folder ( which already exists) so I could get the image using the IP address.

This is the code I use,

Router.route('/users',{where: 'server'})
    .get(function(){
        var response = {"message":"get api"};
        this.response.setHeader('Content-Type','application/json');
        this.response.end(JSON.stringify(response));
    })
    .post(function(){
	var response = this.response;
	var fs = require('fs');
	var data = "data:image/gif;base64,R0lGODlhPQBEAPeoAJosM//AwO/AwHVYZ/z595kzAP/s7P+goOXMv8+fhw/v739/f+8PD98fH/8mJl+fn/9ZWb8/PzWlwv///6wWGbImAPgTEMImIN9gUFCEm/gDALULDN8PAD6atYdCTX9gUNKlj8wZAKUsAOzZz+UMAOsJAP/Z2ccMDA8PD/95eX5NWvsJCOVNQPtfX/8zM8+QePLl38MGBr8JCP+zs9myn/8GBqwpAP/GxgwJCPny78lzYLgjAJ8vAP9fX/+MjMUcAN8zM/9wcM8ZGcATEL+QePdZWf/29uc/P9cmJu9MTDImIN+/r7+/vz8/P8VNQGNugV8AAF9fX8swMNgTAFlDOICAgPNSUnNWSMQ5MBAQEJE3QPIGAM9AQMqGcG9vb6MhJsEdGM8vLx8fH98AANIWAMuQeL8fABkTEPPQ0OM5OSYdGFl5jo+Pj/+pqcsTE78wMFNGQLYmID4dGPvd3UBAQJmTkP+8vH9QUK+vr8ZWSHpzcJMmILdwcLOGcHRQUHxwcK9PT9DQ0O/v70w5MLypoG8wKOuwsP/g4P/Q0IcwKEswKMl8aJ9fX2xjdOtGRs/Pz+Dg4GImIP8gIH0sKEAwKKmTiKZ8aB/f39Wsl+LFt8dgUE9PT5x5aHBwcP+AgP+WltdgYMyZfyywz78AAAAAAAD///8AAP9mZv///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH5BAEAAKgALAAAAAA9AEQAAAj/AFEJHEiwoMGDCBMqXMiwocAbBww4nEhxoYkUpzJGrMixogkfGUNqlNixJEIDB0SqHGmyJSojM1bKZOmyop0gM3Oe2liTISKMOoPy7GnwY9CjIYcSRYm0aVKSLmE6nfq05QycVLPuhDrxBlCtYJUqNAq2bNWEBj6ZXRuyxZyDRtqwnXvkhACDV+euTeJm1Ki7A73qNWtFiF+/gA95Gly2CJLDhwEHMOUAAuOpLYDEgBxZ4GRTlC1fDnpkM+fOqD6DDj1aZpITp0dtGCDhr+fVuCu3zlg49ijaokTZTo27uG7Gjn2P+hI8+PDPERoUB318bWbfAJ5sUNFcuGRTYUqV/3ogfXp1rWlMc6awJjiAAd2fm4ogXjz56aypOoIde4OE5u/F9x199dlXnnGiHZWEYbGpsAEA3QXYnHwEFliKAgswgJ8LPeiUXGwedCAKABACCN+EA1pYIIYaFlcDhytd51sGAJbo3onOpajiihlO92KHGaUXGwWjUBChjSPiWJuOO/LYIm4v1tXfE6J4gCSJEZ7YgRYUNrkji9P55sF/ogxw5ZkSqIDaZBV6aSGYq/lGZplndkckZ98xoICbTcIJGQAZcNmdmUc210hs35nCyJ58fgmIKX5RQGOZowxaZwYA+JaoKQwswGijBV4C6SiTUmpphMspJx9unX4KaimjDv9aaXOEBteBqmuuxgEHoLX6Kqx+yXqqBANsgCtit4FWQAEkrNbpq7HSOmtwag5w57GrmlJBASEU18ADjUYb3ADTinIttsgSB1oJFfA63bduimuqKB1keqwUhoCSK374wbujvOSu4QG6UvxBRydcpKsav++Ca6G8A6Pr1x2kVMyHwsVxUALDq/krnrhPSOzXG1lUTIoffqGR7Goi2MAxbv6O2kEG56I7CSlRsEFKFVyovDJoIRTg7sugNRDGqCJzJgcKE0ywc0ELm6KBCCJo8DIPFeCWNGcyqNFE06ToAfV0HBRgxsvLThHn1oddQMrXj5DyAQgjEHSAJMWZwS3HPxT/QMbabI/iBCliMLEJKX2EEkomBAUCxRi42VDADxyTYDVogV+wSChqmKxEKCDAYFDFj4OmwbY7bDGdBhtrnTQYOigeChUmc1K3QTnAUfEgGFgAWt88hKA6aCRIXhxnQ1yg3BCayK44EWdkUQcBByEQChFXfCB776aQsG0BIlQgQgE8qO26X1h8cEUep8ngRBnOy74E9QgRgEAC8SvOfQkh7FDBDmS43PmGoIiKUUEGkMEC/PJHgxw0xH74yx/3XnaYRJgMB8obxQW6kL9QYEJ0FIFgByfIL7/IQAlvQwEpnAC7DtLNJCKUoO/w45c44GwCXiAFB/OXAATQryUxdN4LfFiwgjCNYg+kYMIEFkCKDs6PKAIJouyGWMS1FSKJOMRB/BoIxYJIUXFUxNwoIkEKPAgCBZSQHQ1A2EWDfDEUVLyADj5AChSIQW6gu10bE/JG2VnCZGfo4R4d0sdQoBAHhPjhIB94v/wRoRKQWGRHgrhGSQJxCS+0pCZbEhAAOw==";
	var buf = new Buffer(data, 'base64');
	var refrence = this;
	var image = 'image_'+Math.floor((Math.random() * 2465789) + 1) +".png";

		var dir = "/var/www/html/special/public/uploadd/";  // Case 1 (**ENONT**)
		var dir = "/var/www/html/special/public/uploads/"+image;   // Case 2 (**ENONT**)
		if (!fs.existsSync(dir)){   // Case 3 (**Got the server Error**)  
		    fs.mkdirSync(dir);
		}

		var path = process.env['METEOR_SHELL_DIR'] + '/../../../public'; // **process.env['METEOR_SHELL_DIR'] = undefined**
		var path = require('fs').realpathSync( process.cwd() + '/../../../../../public/'); (SERVER ERROR)
		var path = path.join(__dirname, '..', 'public', 'uploadd', image); (**ENONT**)
		var path = process.env["PWD"]+"/public/" +image; (Server Error)
        fs.writeFile(, buf,function(err,success){
		if(err){
				var response = {
					"p":p,
	                "result" : err,
	                "message" : "data"
	            };
		}else{
				var response = {
	                "result" : image,
	                "message" : "data"
	            };
		}
		console.log("file Created");
		
        refrence.response.setHeader('Content-Type','application/json');
        refrence.response.end(JSON.stringify(response));

			});
	});

I have hardcoded base64 string for testing purpose. All of these methods are working fine in Local, but not working on AWS EC2 instance. I am using Mup js on the AWS and manually deployed meteor on it.

After facing the ENONT Error,I changed the owner using sudo chown user:user / /var/www/html/project_name /
But this does not help.

Also, using this process.env[‘METEOR_SHELL_DIR’] returns undefined some people are using the same. Is there anyway I could set the process.env[‘METEOR_SHELL_DIR’]

Is there something missing, Please let me know. It would be great help.

I’m not sure what exactly the issue is. It probably is a permission issue like you said. However, its not recommended to use EC2 instances for storage at least not instances that are configured to act as application instances. I would recommend storing the files in S3. That’s both cheaper, more reliable and also a lot easier to access the files from anywhere else. Simply use this command:

It should look like this (instead of fs)

  const base64data = new Buffer(data, 'binary');

  const s3 = new AWS.S3();
  s3.client.putObject({
    Bucket: 'banners-adxs',
    Key: 'del2.txt',
    Body: base64data,
    ACL: 'public-read'
  },function (resp) {
    console.log(arguments);
    console.log('Successfully uploaded package.');
  });

HI @cloudspider, thanks for your suggestion but I can’t have another AWS instance due to issues with the client. Please let me know if there is any possible way by which I could have the desired results. Also, I knows that its not recommended but still I need to find a way to do this.

S3 is not an instance. Its a service so you don’t need to spin up something. Just create a bucket.

If that’s not possible either, you might try the below command after being logged in with SSH:

sudo chmod a+x -R /var/www/html/special/public/uploads/

Also, make sure that the folder exists. If it does not exist already, you need to create it manually.

For this API,

 var response = this.response;
      var fs = require('fs');
      var data = "**data:image/gif;base64 String**";
      var buf = new Buffer(data, 'base64');
      var refrence = this;

      var image = 'image_'+Math.floor((Math.random() * 2465789) + 1) +".png";
      // var path = require('fs').realpathSync( process.cwd() + '/../../../../../public/');
      var path ="/var/www/html/special/public/uploads";
      fs.writeFile(path+"/"+ image,buf,function(err,success){
      		if(err){
      				var response = {
                        "p":path,
                        "result" : err,
                        "message" : "data"
                    };
          }else{
              var response = {
                        "result" : image,
      				        	"path":path,
      	                "message" : "data"
      	            };
      		}
      		// console.log("file Created");
      		
              refrence.response.setHeader('Content-Type','application/json');
              refrence.response.end(JSON.stringify(response));

      			});

I got this in response

I execute the command above.

Here is my server path, lets have a look at this also
Screenshot%20from%202018-06-28%2022-27-26

All Folder exists but got the problem again. Is there still anything wrong?

Are you able to use the command line to SSH into the EC2 instance? I think you app does not have permission to create any file.

Yes, I am able to create/delete any file and I have the SSH access.

(just thinking out loud here) It might be that your app is not aware of the file structure of the machine. This would mean that linking to /public/uploads will actually link to:

{dir-of-your-app}/public/uploads

And /var/www/html/special/public/uploads will link to:

{dir-of-your-app}/var/www/html/special/public/uploads 

Meaning it actually links to: /var/www/html/special/var/www/html/special/public/uploads which ofcourse does not exist.

Try to do this: var path ="/public/uploads";

Got this error. is there any way I can give the relative path like ./…/public/uploads` instead of /public/uploads

{
“p”: “/public/uploads”,
“result”: {
“errno”: -2,
“code”: “ENOENT”,
“syscall”: “open”,
“path”: “/public/uploads/image_553257.png”
},
“message”: “data”
}

Hi @cloudspider,

I returned process.cwd() and it returned this path. “path”: “/built_app/programs/server”, , is there something wrong with this ??
On Local : /home/bird/Downloads/Meteor/rest/.meteor/local/build/programs/server
On Server (AWS) : I am not able to locate this, also searched .meteor folder.

Command: locate built_app
Output: /var/lib/docker/overlay2/3231e406e1e783c861846a0e0b9543aa60d3ef46953b8d3e2f364678faeb5b96/diff/built_app

There is some internal folders created under docker folder. and Can I change this.
Is there any way to specify public folder in mup.js file.

This makes sense! Meteor is being built to make it production ready. It means that it gets converted back to a plain nodejs application. What you see in your EC2 server is essentially a built version of your Meteor app. There is no .meteor folder.

It should be located here: programs/web.browser/app/

If you can’t find it, you might want to try this:

var path = process.env['METEOR_SHELL_DIR'] + '/../../../public';

I tried this earlier but it’s also giving undefined for process.env[‘METEOR_SHELL_DIR’].

Also I tried to set the process.env[‘METEOR_SHELL_DIR’] = “/var/www/html/special/public/”
but this also does not works

Correct me if I am wrong.

The problem is if I am giving it the path from starting it also not taking it. It should accept the path when giving from starting like /var. All the solutions basically redirects to giving the right path.

I know that we will eventually be able to solve the problem. However, I would definitely dis-recommend you from going this path. Is it no option to use s3 or any other proper storage at all? Since the real problem is that you are trying to do something against how both Meteor and Ec2 are intended to operate.

Meanwhile I will test a similar case myself. Brb…

Actually, there are some Costing issues for using AWS Services. You continue to do the needful. I will wait till then.
Thanks

Storing files via S3 is WAY cheaper. No upfront costs aswell:

I know its cheap but please consider this is not available for us.

Hi @cloudspider,

Have you created the test case?

Mup runs your app inside of a docker container, which has it’s own file system. The changes to the file system are lost when the container is stopped or removed, which happens when deploying the app or the app crashes.

To store the files outside of the docker container on the server, you can use a volume. In your mup config, add app.volumes:

module.exports = {
  app: {
    volumes: {
      '/var/www/html/special/public/uploads/': '/var/www/html/special/public/uploads/'
    }
  }
};

This will make /var/www/html/special/public/uploads/ available in your docker container at the same path.

If you are using a docker image that runs the app with a non-root user (abernix/spaceglue or zodern/meteor), to avoid permission problems /var/www/html/special/public/uploads/ should be owned by a user with the same user id as the one in the docker image. If you are using zodern/meteor, run chown -R 1000:1000 /var/www/html/special/public/uploads/.