How to `fetch()` from Meteor server

xareelee · December 8, 2016, 7:26am

I have http requests in the front-end for CORS, but I can’t change that server headers to support CORS.

const url = "http://121.42.154.150:8081/chart/h?cf=AVERAGE&end=1481009206&graph_type=h&id=24022&start=1480922806&sum=off";

fetch(url).then(res => {
  console.log('res', res);
})

Is there any way to fix this without changing server side?

My approach

I find a way for workaround:

make Meteor as a REST API server;
make a http request to Meteor server from Meteor client;
forward the query from the client to the target API server and return the results back to the Meteor client.

But I get stuck to build a REST API service on the Meteor server.

1. using `fetch()` to the same origin (Meteor server) will always return a success

// client side
fetch('/hello/world').then(res => {
  console.log('res', res);
})

I didn’t implement any server side code to response /hello/world, but it will get the response with status 200.

2. try to use express on the meteor

I tries to build the API service with express by implementing meteor-express:

// server side
var app = express();
WebApp.connectHandlers.use(Meteor.bindEnvironment(app));

app.get('/hello/world', function (req, res) {
  res.send('Hello World')
});

It still doesn’t work. I can’t get the data . The fetch() in the client side will always get the same results no matter what route I query.

3. try using Restivus to build a REST server on the Meteor

Negative. I failed to install restivus by meteor add nimble:restivus. It shows:

=> Errors while adding packages:             
                                              
While loading package npm-bcrypt@0.7.8_2:
error: Command failed: /Users/Xaree/.meteor/packages/meteor-tool/.1.4.2_3.1fwg30s++os.osx.x86_64+web.browser+web.cordova/mt-os.osx.x86_64/dev_bundle/bin/npm rebuild
--update-binary
gyp ERR! configure error 
gyp ERR! stack Error: read ETIMEDOUT
gyp ERR! stack     at exports._errnoException (util.js:911:11)
gyp ERR! stack     at TLSWrap.onread (net.js:558:26)
gyp ERR! System Darwin 16.1.0
...

I have no idea how to achieve my goal now.

Please give me some advices (or sample code that would work). I’m really stuck.

robfallows · December 8, 2016, 10:29am

As you have correctly determined, you’ll need to make the http call on the server. There are a number of ways you could get the result back to the client. However, for a simple “one-shot”, the easiest way is to call a server-side method from the client.

server code

Note that if you are returning JSON from your endpoint, Meteor will correctly return this as a standard JavaScript object. You will need to meteor add http to your project and import { HTTP } from 'meteor/http'; in the files where HTTP is referenced.

Meteor.methods({
  getData() {
    try {
      return HTTP.get('http://121.42.154.150:8081/chart/h?cf=AVERAGE&end=1481009206&graph_type=h&id=24022&start=1480922806&sum=off');
    } catch (error) {
      throw new Meteor.Error('oops', 'something broke');
    }
  }
});

client code

Where you put this (and how you handle the returned data, which is asynchronous) is determined by your application requirements. However, the basic syntax is:

Meteor.call('getData', (error, result) => {
   if (error) {
    // handle the error
  } else {
   console.log(result);
  }
});

xareelee · December 9, 2016, 1:06am

Thanks @robfallows. It works!

I have further questions here.

modifying the results before sending back

If I want to modify the results of the request in the server side before sending it back to the client, how to do it?

Meteor.methods({
  getData(a, b) {
    HTTP.get('http://falcon.chaoke.ren:8081/chart/h?cf=AVERAGE&end=1481009206&graph_type=h&id=24022&start=1480922806&sum=off', {}, (error, response) => {
      const obj = JSON.parse(response.content);
      const data = obj['series'][0]['data'];
      // send the data back to client side
    });
  }
});

multiple HTTP requests

If I need to call multiple HTTP requests each depending on the results of the previous request, how to do it?

For example: (1) asking the id for the type I want to search (2) use the id to fetch the results (3) return the results back to client.

ffxsam · December 9, 2016, 1:35am

Don’t use a callback when using HTTP on the server side. (Follow Rob’s example above) I’d recommend doing this:

Meteor.methods({
  getData(a, b) {
    let modifiedResult;

    try {
      let result = HTTP.get('http://falcon.chaoke.ren:8081/chart/h?cf=AVERAGE&end=1481009206&graph_type=h&id=24022&start=1480922806&sum=off');
      modifiedResult = '?'; // do something with result
    } catch (e) {
      // Do whatever you want with handling the error object (e)
      throw new Meteor.Error('some-error-code', 'Something bad went down');
    }

    return modifiedResult;
  }
});

tathagatbanerjee · December 9, 2016, 1:54am

@robfallows, @ffxsam - gents, simple question. If, in the above example, there was a delay in the return of the http request. So

(1) make http request
(2) http request is expected to come back with info after say a few hours (4 hours or whatever). User needs to trigger call from UI. Once done, this goes to server and request is handled as a promise.
(3) The promise polls the http service for response every hour (? or some time interval). Once successfully completed, response from http service is written to somewhere in the db. If no successful response for a set time (12 hours?), different part of mongo db written to.

Assuming low usage (not that interested in how it scales at moment), would it make sense to use a promise in this context. or is one better of trying to write a node micro service or some other architecture to handle this? pseudo-code appreciated

Thanks so much.

Tat

ffxsam · December 9, 2016, 2:07am

I think this is across the board, but generally you cannot do async operations in Meteor methods. It’s a server-side operation that expects to run, and return something. By its nature, async funcs can’t run a process, wait, and return the result.

If you really think the HTTP call will take more than a few seconds, be sure to use unblock:

Meteor.methods({
  getData(a, b) {
    this.unblock();

    let modifiedResult;
    // ...

xareelee · December 9, 2016, 2:10am

@ffxsam, thanks for your reply.

Your recommendation is like sync code. Will it block the runloop of the node.js (Meteor) when it sends a request over synchronous HTTP?

tathagatbanerjee · December 9, 2016, 3:37am

hey mate - wouldn’t the promise be the thing that gets returned though?

ffxsam · December 9, 2016, 3:59am

Honestly I’ve never tried returning a promise from a Meteor method. Give it a shot.

That’s what this.unblock() is for.

xareelee · December 9, 2016, 6:10am

@ffxsam

It seems like Meteor.method only supports sync code for returning value. If I want to implement ‘retry’ feature with a time interval for ‘sleep’, is there any good idea to sleep in JS? Using the following code would not be good in the production, right?

function sleep(delay) {
  var start = new Date().getTime();
  while (new Date().getTime() < start + delay);
}

ffxsam · December 9, 2016, 7:19am

I’d say this is going down a bad road. A method is the wrong place for a repeating cron type process. If you want to fetch data via HTTP every X seconds, you could just have a Meteor.setInterval on the server side, and have it dump data into the DB so the client can reactively pick it up.

But I’m making assumptions about your goals and architecture, so I could be way off.

robfallows · December 9, 2016, 11:38am

True.

Not strictly true (see comments below) …

Meteor methods fully support Promises, however the Promise is resolved before data is sent over the wire to the client. That means it’s quite in order to use this sort of code:

Meteor.methods({
  async myAsyncMethod() {
    const a = await someAsyncFunction();
    const b = await someOtherAsyncFunction(a);
    return b;
  },
});

From the perspective of the client that method (myAsyncMethod) looks no different than a standard Meteor method - you can call it in the exact same way:

Meteor.call('myAsyncMethod', (error, result) => {
  if (error) {
    // do something with the error
  } else {
    console.log(result);
  }
});

Equally, you can Promisify the Meteor.call independently of whether or not the method is async. I use something like this:

const callWithPromise = (method, myParameters) => {
  return new Promise((resolve, reject) => {
    Meteor.call(method, myParameters, (err, res) => {
      if (err) reject('Something went wrong');
      resolve(res);
    });
  });
}

and then use it something like this:

async function blah() {
  const myValue1 = await callWithPromise('myMethod1', someParameters);
  const myValue2 = await callWithPromise('myMethod2', myValue1);
};

Interestingly (or not), you don’t need this.unblock() for async methods. They are “unblocked” naturally.

You could use a Promise for this in a method. However, I’m not sure of your expected use case here. If I make an http get (for example) on a REST endpoint, the connection will time out long before the 4 hours (or whatever) is up. Basically, this is not how REST endpoints are supposed to work.

Well, yes, if this isn’t a normal REST endpoint you could do something like that. However, if the use case is “request some long-running action”, a better way to to do this is to have that request return a 200 to say “okay, that’s under way” and then use a webhook (or similar device) to signal completion - polling is not really very good for that. If the endpoint is completely under your control then you could write it as a Meteor microservice and use inter-server method calls and pub/sub.

Me too!

ffxsam · December 9, 2016, 6:01pm

Yes, for sure. I meant in a case where a dev is calling a Meteor method that uses a blocking HTTP.call that might take 20 seconds. As I understand it (correct me if I’m wrong), the method in question will not lock up the Meteor server, but it will prevent that user from making any other method calls until the HTTP request is complete. (unless this.unblock() is used)

robfallows · December 9, 2016, 6:45pm

No, I mean it doesn’t block the same client, without using this.unblock()

ffxsam · December 9, 2016, 8:10pm

But for synchronous operations in a Meteor method, each client can only execute one at a time. So for example:

Meteor.call('someLongOperation'); // takes 5s to return
Meteor.call('someShortOperation');

What would happen is:

someLongOperation method called
5 second delay
someShortOperation method called

Is that not the case? I might be misunderstanding the documentation for this.unblock():

On the server, methods from a given client run one at a time. The N+1th invocation from a client won’t start until the Nth invocation returns. However, you can change this by calling this.unblock. This will allow the N+1th invocation to start running in a new fiber.

Or I just misunderstood your post