Am I pushing Meteor._wrapAsync beyond what it can do? [Solved]

help
1

This question is rather long, sorry! I think any shortening would make it ambiguous, though…

I have a requirement to wrap many dozens of asynchronous functions into synchronous functions.

The functions are generated “on-the-fly”, in accordance with specifications in a JSON file, by a module I access with Npm.require().

The functions are grouped by entity like this :

 {
      "user" : ["getUserByName", "foo", "etc"]
   ,   "pet" : ["getPetById", "bar", "etc"]
   , "store" : ["getOrderById", "bloo", "etc"]
 }

I need to be able to call them like this …

var aPet = {petId : 486635645};
var thePet = asyncProxy.pet.getPetById( aPet, mimeType );

… not like this …

var callbackGetPetById = function(pet) {  console.log('pet', thePet);  };
syncProxy.pet.getPetById( aPet, mimeType, callbackGetPetById );

To create synchronous functions from the asychronous ones, I run them through Meteor._wrapAsync one at a time, in an outer/inner loop like this :

 for (entity in entities) ...
   for (method in entityMethods) ...
         : 
     collectedMethods[entity][method] = Meteor._wrapAsync( 
       function (arguments, headers, success, error) {
         syncProxy[entity][method](
             arguments
           , headers
           , function ( theResult ) {  success(null, theResult);  }
           , function (  theError ) {    error(null, theError );  }
         )
       }
     );
         : 
  return collectedMethods;
}

I then test one of them with :

Tinytest.add('Get "Fido" ::  Try as bulk pre-wrapped function !', function (test) {
  var synchronous_functions = collectMethods(swagger, "sync");
  var jsonPet = synchronous_functions["pet"]["getPetById"] ( aPet, mimeType );
  var pet = JSON.parse(jsonPet.data)
  console.log('pet', pet); 
  test.equal(pet.name, "Fido");
});

Unfortunately, neither of the callback’s are ever referenced. The correct data is written to a log file by the 3rd party module, but my attempted synchronous function returns jsonPet “undefined”.

Meanwhile, there is no problem if I wrap the functions individually like this :

var getPetById = Meteor._wrapAsync(  
  function (arguments, headers, success, error) {
    syncProxy["pet"]["getPetById"](
        arguments
      , headers
      , function ( theResult ) {  success(null, theResult);  }
      , function (  theError ) {    error(null, theError );  }
    )
  }
);

Tinytest.add('Get "Fido" ::  Try with Async.wrap()!', function (test) {
  var jsonPet = getPetById ( thePet, mimeType );
  var pet = JSON.parse(jsonPet.data)
  console.log('pet', pet); 
  test.equal(pet.name, "Fido");
});

Result: pet.name is indeed equal to “Fido”.

Evidently, the success and fail functions are not getting passed in correctly. The 3rd party module dumps to console.log when no success handler is provided and the Travis listing shows that the correct result is obtained, just unavailable since the success handler is not provided correctly.

So … I would like to ask, what is the correct way to make the success handler available to the methods during the process of wrapping them?

Travis results. The source.

  • wrapping in a loop is done between lines 162 & 168
  • failing tests begin at line 231
  • the several tests preceding line 140 show my efforts to ensure I am building on solid ground.
2

I believe the source of your error in the loop version of your wrap async call is what you are trying to reference the outer context in your function which you cannot do. You have to pass the context along with the function for wrap async, i.e.

 for (entity in entities) ...
   for (method in entityMethods) ...
         : 
     collectedMethods[entity][method] = Meteor._wrapAsync( 
       function (arguments, headers, success, error) {
         syncProxy[entity][method](
             arguments
           , headers
           , function ( theResult ) {  success(null, theResult);  }
           , function (  theError ) {    error(null, theError );  }
         )
       }
    , this); // <--- This second argument
         : 
  return collectedMethods;
}

This is because wrap async requires you to specify the context if you wish to form a closure.

3

I am not sure I understand.

The only context from the outer loop I need is the name of the entity, but the error log shows that both the entity name and the method name are used correctly. It’s only the function calls that aren’t working.

Note : I deliberately pseudo-coded the outer/inner loop; the actual code tested is here

4

I really would suggest making a much much simpler test case / some experimentation code in order to be able to fully and specifically grasp how _wrapAsync works and what it expects from you. You’d do yourself a great favor. :smile:

Also, I think this might be a mistake here:

var getPetById = Meteor._wrapAsync(  
  function (arguments, headers, success, error) {
...

The standard nodejs convention for callbacks is not to have 2 callbacks, one for the success case, one for the error case, but rather it is to pass a single function as the last parameter, the callback function, which itself takes 2 parameters, error and result.
You are violating that convention. In order to rectify that you would have to write something like:

var getPetById = Meteor._wrapAsync(  
  function (arguments, headers, callback) {
//...

function callback(error, results) {
  if (error) { ... handle error ... return; }
  // do something with result here
//...

I hope it’s clear, even though I’ve left out a lot of code. If it’s not clear then go have a look at the nodejs standard library/API over at http://nodejs.org/api/index.html and look at some of the examples, e.g. in the fs module. You’ll quickly see the general pattern here with callbacks.
And _wrapAsync adheres to nodejs convention, and it expects you to as well.

And the other thing, and that’s, iiuc, what @entropy alluded to, is that you have to make sure you carry over any this bindings you need for your functions manually. I.e. if your methods need a certain this in order to function properly, make sure to use _wrapAsync-s last argument to pass that this binding, or “context”, along. Especially third-party libraries often expect that you keep their this/context intact. Example:

let ppCreate = Meteor.wrapAsync(paypal.payment.create.bind(paypal.payment))

Real-world code. PayPal NodeJS SDK. You need to call a method on paypal.payment, do it synchronously / in a Meteor Fiber, while keeping the this binding of the method intact. One way to do that is using the last param to wrapAsync, another one is to use Function.bind.

HTH!

5

Oh and to answer your original question: No, you are not. (pushing wrapAsync beyond what it can do)
You’re just tripping over (your own) complexity and need to understand more clearly what’s happening and what is expected. And then you’ll be just fine and will be able to accomplish what you’re trying to do there! :smile:

6

Dear @seeekr

I really appreciate it that you took the time for such a detailed reply. Where possible I do try to keep tests extremely simple. It’s why the first half of the test script shows incremental “baby-steps” towards the issue I am trying to solve.

It also seemed to me that the “one call back” rule is not being respected. Unfortunately, this is not within my control. My package calls the node module : swagger-client. Their code creates javascript functions and executes them “on-the-fly” from a swagger spec. This also limits my chances for simplifying the test case any further than I have already,

I am fairly sure I tried that already, but I’ll have another go at it in the morning.

Thanks very much again for offering assistance.

7

Ok. I fixed it.

I spent the better part of a day picking over this, simplifying things and working with node-inspector to get to the root of the problem. The symptom of the problem has changed and is more understandable now. Here it is :

 for (entity in entities) ...
   for (method in entityMethods) ...
         : 
     collectedMethods[entity][method] = Meteor._wrapAsync( 
       function (arguments, headers, success) {
         asyncProxy[entity][method](
             arguments
           , headers
           , function ( theResult ) {  success(null, theResult);  }
         )
       }
     );
         : 
  return collectedMethods;
}

In the pseudo-code above. When a caller tries to execute any one of the synchronous methods, for example, collectedMethods["pet"]["getPetById"]({petId : 234564556}) the asynchronous method called is asyncProxy["user"]["getUserByUserName"].

It does this regardless of which synchronous method is referenced. Not surprisingly, user and getUserByUserName are the last entity and method referenced in the outer/inner loop !!

So, the context used by Meteor._wrapAsync is not saved at each loop iteration; all iterations get passed the last state of the context of the loops.

The complexity of this requirement is that, while Meteor._wrapAsync() expects to wrap a function that wraps a function, I need it to wrap a function that wraps an array of arrays of functions.

So the solution must lie in ensuring that the async wrapping gets the correct context with each iteration. Bingo!

All I had to do was execute Meteor._wrapAsync() inside a function called from the inner loop to ensure a fresh persistent context for each iteration.

function wrapItUp(syncs, nameEntity, nameMethod, proxy) {
     syncs[nameEntity][nameMethod] = Meteor._wrapAsync( 
       function (arguments, headers, success) {
         proxy[nameEntity][nameMethod](
             arguments
           , headers
           , function ( theResult ) {  success(null, theResult);  }
         )
       }
     );
}

for (entity in entities) ...
  for (method in entityMethods) ...
        : 
    wrapItUp(collectedMethods, entity, method, asyncProxy);
        : 
  return collectedMethods;
}

Thanks to all for helping me narrow the range of possibilities to something comprehensible.

2 Likes