How do Fibers and Meteor.asyncWrap work?

So I understand why I want to use it.

b = function(callback) {
  callback(null, 10)
}
f = Meteor.wrapAsync(b)
f()
// > 10

But then for some reason, this doesnt work:

a = function(b, callback) {
  b(callback)
}
f = Meteor.wrapAsync(a)
f(b)
// > nothing

Could someone help me understand why this doesnt work?

3 Likes

Meteor.wrapAsync requires a thisArg

I believe its optional.

I just found this btw. Starting to read it now:

https://meteorhacks.com/fibers-eventloop-and-meteor

This has been one of the tougher areas for me to understand within Meteor (and in general). I’m sure I’m not the only one out there who has leapt with joy upon seeing a json response finally show up in their browser console.

I wish I had some sage advice, but for me it was a lot of trial and error. I wish I could say that I’ve had luck with wrapAsync, but I just haven’t been able to wrapMyHeadAroundIt. I have had luck with fibers/future, and the code snippet below has helped me immensely in making http requests available to the client. Hope it helps!

https://www.discovermeteor.com/patterns/5828399

2 Likes

I went through a lengthy debug chore trying understand what you can and can’t do with Meteor.wrapAsync(). Looking at what I went through might help : Am I pushing Meteor.wrapAsync() beyond what it can do [solved]?

Reading the code was very useful. It’s wonderfully compact, really :

// wrapAsync can wrap any function that takes some number of arguments that
// can't be undefined, followed by some optional arguments, where the callback
// is the last optional argument.
// e.g. fs.readFile(pathname, [callback]),
// fs.open(pathname, flags, [mode], [callback])
// For maximum effectiveness and least confusion, wrapAsync should be used on
// functions where the callback is the only argument of type Function.

/**
 * @memberOf Meteor
 * @summary Wrap a function that takes a callback function as its final parameter. On the server, the wrapped function can be used either synchronously (without passing a callback) or asynchronously (when a callback is passed). On the client, a callback is always required; errors will be logged if there is no callback. If a callback is provided, the environment captured when the original function was called will be restored in the callback.
 * @locus Anywhere
 * @param {Function} func A function that takes a callback as its final parameter
 * @param {Object} [context] Optional `this` object against which the original function will be invoked
 */
wrapAsync: function (fn, context) {
  return function (/* arguments */) {
    var self = context || this;
    var newArgs = _.toArray(arguments);
    var callback;

    for (var i = newArgs.length - 1; i >= 0; --i) {
      var arg = newArgs[i];
      var type = typeof arg;
      if (type !== "undefined") {
        if (type === "function") {
          callback = arg;
        }
        break;
      }
    }

    if (! callback) {
      if (Meteor.isClient) {
        callback = logErr;
      } else {
        var fut = new Future();
        callback = fut.resolver();
      }
      ++i; // Insert the callback just after arg.
    }

    newArgs[i] = Meteor.bindEnvironment(callback);
    var result = fn.apply(self, newArgs);
    return fut ? fut.wait() : result;
  };
},

That’s from meteor/packages/meteor/helpers.js

2 Likes

(EDIT: What I wrote here is probably not correct or at least not quite correct. @warehouseman is describing the actual, immediate issue below.)

I believe the issue is about b’s callback not being wrapped in a Meteor execution context, and so Meteor can’t know that there’s something else to wait for/handle. Something of that sort. (see Meteor.bindEnvironment)
But I’m not super sure, haven’t had the need to dig into these things that far. Definitely share what you learn so we can all understand this better!

Here’s how wrapAsync handles the arguments of the function it is wrapping ::

for (var i = newArgs.length - 1; i >= 0; --i) {
  var arg = newArgs[i];
  var type = typeof arg;
  if (type !== "undefined") {
    if (type === "function") {
      callback = arg;
    }
    break;
  }
}

The argument “b” in the code fragment of @ccorcos . . .

a = function(b, callback) {
  b(callback)
}

. . . is a function. Therefore, wrapAysnc treats it as a the callback and breaks out from the loop. The next passed function named “callback” is ignored.

1 Like

Oh! Ha, that makes sense. Well, in terms of “how does wrapAsync handle this”, not in terms of “what one would expect”. Thanks for spelling it out that clearly :smile:

Isn’t this pretty much a bug in wrapAsync? It’s not sticking to the (nodejs) convention of “callback argument last” and is instead doing a super weird search for function args from the beginning? Wtf? Right? Or am I not getting something here?

Actually, it breaks out from the loop on undefined. However, until it gets undefined it will continue to overwrite the callback variable every time it comes across a function.

It may better to use:

if (type === "function" && 'undefined' === typeof callback) {
  callback = arg;
}

But even this won’t constrain the final argument to be the callback, just the function nearest the end.

@sashko: does this make sense?

[Edit: I’m talking nonsense, see @warehouseman’s post further down]

As soon as it has found any function, it exits the loop. So its rule seems to be “callback argument first and last”.

I have the problem of wrapping a 3rd party NodeJS module that breaks the convention in its own sweet way. First callback for “success”. Second callback for “error”. :confounded:

1 Like

Yes, that’s true. Not enough sleep last night. It is already breaking out on the nearest function to the end.

I wonder if it is worth raising a GitHub issue on it? I don’t like to, because I don’t have a clear understanding of all the forces that impinge on the design.

1 Like

This does look like an issue. Or at least I think there is an issue here, but I’m not sure it’s your issue.

The documentation is clear:

Wrap a function that takes a callback function as its final parameter. On the server, the wrapped function can be used either synchronously (without passing a callback) or asynchronously (when a callback is passed). On the client, a callback is always required; errors will be logged if there is no callback.

So, we have the possibility of no callback in the final parameter, but if there is a callback it must be the final parameter. Or, to turn it on its head, “if the final parameter is a function, then it’s the callback”

My analysis of the code runs as follows (code section first, explanation following):

wrapAsync: function (fn, context) {
  return function (/* arguments */) {
    var self = context || this;
    var newArgs = _.toArray(arguments);
    var callback;

So, we begin with a copy of the arguments in an array (indexed from [0]) and an undefined callback variable. Then:

    for (var i = newArgs.length - 1; i >= 0; --i) {
      var arg = newArgs[i];
      var type = typeof arg;
      if (type !== "undefined") {
        if (type === "function") {
          callback = arg;
        }
        break;
      }
    }

We iterate over the arguments in reverse order (starting with the last and going to the first). As soon as we find a function, we break out of the loop with i at the position of the function and callback set to that function reference. If we don’t find a function, the loop runs to the end, and i has the final value of -1, with callback still undefined. Then:

    if (! callback) {
      if (Meteor.isClient) {
        callback = logErr;
      } else {
        var fut = new Future();
        callback = fut.resolver();
      }
      ++i; // Insert the callback just after arg.
    }

If we did find a function (callback), skip the above section (i will be at the position of the function in the newArgs array).

Otherwise, i will be -1 by this time and we could have any number of non-function parameters. Clients must use a callback, so we need to register an error. On the server, no callback means use a future. In all cases increment i (which means set to 0). Then:

    newArgs[i] = Meteor.bindEnvironment(callback);
    var result = fn.apply(self, newArgs);
    return fut ? fut.wait() : result;
  };
}

What’s going on here? The i-th parameter is overwritten with the callback (may be the actual callback, or the registered error callback, or a future). That’s fine if there were no parameters or if the final parameter was the callback. But what if this is on the server and there are several non-function parameters? In that scenario we don’t need to specify a callback, because a future will be provided for us. However, it looks as if newArgs[0] will be given the future - but newArgs[0] is one of our non-functional parameters and has just been destroyed. I think that line ++i; // Insert the callback just after arg. is incorrect and should be: i = newArgs.length; // Insert the callback just after final parameter.

What aren’t I seeing?

1 Like

Getting to the bottom of your issue now, @ccorcos.

In traditional node world, an on-completion callback has two arguments, either/both of which may be null: error and result in that order.

In your first definition, you define your wrapped function as having no parameters except for the final callback. When you invoke the callback you follow node law and execute it with (null, 10): null error and result of 10. Executing that wrapped function therefore returns a result of 10 as expected.

In your second definition, you define a function with one parameter b (the original function) and a callback. However, in a's function body you do not follow node law and execute callback with its two arguments. Instead, you execute the first parameter, your original b function. This means that a's callback is never executed within a's context and so the wrapped result is undefined.

[Edit: to get OP right!]

1 Like

Uhmmmm. The issue ain’t mine it’s from @ccorcos

(Have you considered visiting a sleep lab? :stuck_out_tongue_winking_eye: I should talk! I didn’t notice that the loop was reverse order until you pointed it out.)

1 Like

DOH!

Sleep would be good. Bring it on.

I’ll just edit that post first … zzzz

When you’re awake again, you might want to publish a Meteor Pad example of the failure you expect to result from replacing arg 0.

@vigorwebsolutions, thats a nice example. I suppose fibers are using Futures a bunch under the hood? I recall about a month ago, Meteor was switching everything from Futures to Promises… Not sure the difference really…

Well, b has the proper callback with two arguments, so passing the callback to b should ensure the callback gets the right two argument callback.

I’m not entirely sure what but you’re speaking of, but it sounds like they take the first function in the args to be the callback. So maybe it thinks b is the callback?

Also, it looks like this may have to do with the answer…

I’ve been trying to find out what Meteor.bindEnvironment does. Any ideas?