Meteor HTTP.get() : how to work with the obtained file?

help
1

When I work in basic node, I can easily get a file and save it on my server :

let fs = require( 'fs' );
let https = require( 'https' );
var file = fs.createWriteStream( process.env.PWD + fileName );
let request = https.get( fileURL, function ( response ) {
	response.pipe( file );
} );

But when I try to use Meteor HTTP, I cannot find how the content is encoded :

let file = HTTP.get( fileURL );
fs.writeFileSync(
	process.env.PWD + '/imports/' + fileName,
	new Buffer( file.content, 'utf8' )
);

I tried binary, usc2and other friends…

For a jpeg, the content looks like that :

���\u0000\u0018Exif\u0000\u0000II*\u0000\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000��\u0000\u0011Ducky\u0000\u0001\u0000\u0004\u0000\u0000\u0000<\u0000\u0000��\u00031http://ns.adobe.com/xap/1.0/\u0000<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?> <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c111 79.158325, 2015/09/10-01:10:20        "> <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"> <rdf:Description rdf:about="" xmlns:xmp="http://ns.adobe.com/xap/1.0/" xmlns:xmpMM="http://ns.adobe.com/xap/1.0/mm/" xmlns:stRef="http://ns.adobe.com/xap/1.0/sType/ResourceRef#" xmp:CreatorTool="Adobe Photoshop CC 2015 (Macintosh)" xmpMM:InstanceID="xmp.iid:F0133203CE5511E5A010FA2CA91463FD" xmpMM:DocumentID="xmp.did:F0133204CE5511E5A010FA2CA91463FD"> <xmpMM:DerivedFrom stRef:instanceID="xmp.iid:F0133201CE5511E5A010FA2CA91463FD" stRef:documentID="xmp.did:F0133202CE5511E5A010FA2CA91463FD"/> </rdf:Description> </rdf:RDF> </x:xmpmeta> <?xpacket end="r"?>��\u0000\u000eAdobe\u0000d�\u0000\u0000\u0000\u0001��\u0000,.........

For All the text based file (json, xml…) it works well.

What did I miss ?

1 Like
2

The encoding used for the response should be in your file.headers object.

3

The list of headers I obtain

x-amz-id-2
x-amz-request-id
date
content-disposition
last-modified
etag
accept-ranges
content-type
content-length
server

The solution was to pass npmRequestOption width encoding null as parameter

let response = HTTP.call( 'GET', fileURL, {
	npmRequestOptions: {
		encoding: null
	}
} );

Now the content is a Buffer and not a string with a weird encoding.

4 Likes
4

The encoding is in content-type, (for example "image/png").

By setting a requirement of null encoding you’ll get a Buffer because you’re effectively requesting raw binary data (which will be fine as long as you know what to expect).

5

Hey Rob, how would I know the encoding from the content-type? If, for example, I take 'content-type': 'application/pdf', how would I know what encoding the data is send in?

6

You need to look for the Content-Transfer-Encoding header. For example:

Content-Transfer-Encoding: base64
7

Thanks @robfallows for answering so quickly. Unfortunately, the encoding is not specified in the header. What I get concerning the content is:

'content-type': 'application/pdf',
'content-disposition': 'attachment; filename=somenamepdf',
'content-length': '59503',

and that’s it. This is why I was wondering how to infer it from the content-type. Because, I also tried binary, base64 and the likes in new Buffer(content, encoding). None worked. For the time being, I just went with the npmRequestOptions solution specified above, but would be good to know nonetheless.