Meteor HTTP.get() : how to work with the obtained file?


When I work in basic node, I can easily get a file and save it on my server :

let fs = require( 'fs' );
let https = require( 'https' );
var file = fs.createWriteStream( process.env.PWD + fileName );
let request = https.get( fileURL, function ( response ) {
	response.pipe( file );
} );

But when I try to use Meteor HTTP, I cannot find how the content is encoded :

let file = HTTP.get( fileURL );
	process.env.PWD + '/imports/' + fileName,
	new Buffer( file.content, 'utf8' )

I tried binary, usc2and other friends…

For a jpeg, the content looks like that :

���\u0000\u0018Exif\u0000\u0000II*\u0000\b\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000��\u0000\u0011Ducky\u0000\u0001\u0000\u0004\u0000\u0000\u0000<\u0000\u0000��\u00031\u0000<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?> <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.6-c111 79.158325, 2015/09/10-01:10:20        "> <rdf:RDF xmlns:rdf=""> <rdf:Description rdf:about="" xmlns:xmp="" xmlns:xmpMM="" xmlns:stRef="" xmp:CreatorTool="Adobe Photoshop CC 2015 (Macintosh)" xmpMM:InstanceID="xmp.iid:F0133203CE5511E5A010FA2CA91463FD" xmpMM:DocumentID="xmp.did:F0133204CE5511E5A010FA2CA91463FD"> <xmpMM:DerivedFrom stRef:instanceID="xmp.iid:F0133201CE5511E5A010FA2CA91463FD" stRef:documentID="xmp.did:F0133202CE5511E5A010FA2CA91463FD"/> </rdf:Description> </rdf:RDF> </x:xmpmeta> <?xpacket end="r"?>��\u0000\u000eAdobe\u0000d�\u0000\u0000\u0000\u0001��\u0000,.........

For All the text based file (json, xml…) it works well.

What did I miss ?


The encoding used for the response should be in your file.headers object.


The list of headers I obtain


The solution was to pass npmRequestOption width encoding null as parameter

let response = 'GET', fileURL, {
	npmRequestOptions: {
		encoding: null
} );

Now the content is a Buffer and not a string with a weird encoding.


The encoding is in content-type, (for example "image/png").

By setting a requirement of null encoding you’ll get a Buffer because you’re effectively requesting raw binary data (which will be fine as long as you know what to expect).


Hey Rob, how would I know the encoding from the content-type? If, for example, I take 'content-type': 'application/pdf', how would I know what encoding the data is send in?


You need to look for the Content-Transfer-Encoding header. For example:

Content-Transfer-Encoding: base64


Thanks @robfallows for answering so quickly. Unfortunately, the encoding is not specified in the header. What I get concerning the content is:

'content-type': 'application/pdf',
'content-disposition': 'attachment; filename=somenamepdf',
'content-length': '59503',

and that’s it. This is why I was wondering how to infer it from the content-type. Because, I also tried binary, base64 and the likes in new Buffer(content, encoding). None worked. For the time being, I just went with the npmRequestOptions solution specified above, but would be good to know nonetheless.