Fibers and meteor-promise npm pacakge

core
1

Last week I read @benjamn’s presentation from GOTO Chicago on “Why Fibers Make Sense for Meteor” and got a much better sense of why Fibers are still needed in a Javascript world with Promises, Iterators, and Generators. (Great slides Ben, thanks for posting them!) I was surprised and happy to find out that Ben already made the meteor-promise npm package that wraps Promises with Fibers, and gives us new abilities with Promise.await and Promise.async. I’m also curious and excited to see that it’s a vanilla Node package and not just a Meteor one. The more we can share with everyone else the better :two_men_holding_hands:

I was wondering what the recommendation is to start using this today? I tried to drop it into an existing application with Npm.depends and browserify but I couldn’t get it working after about an hour or so. I think it was from conflicts between native Promises/Babel & meteor-promise, but I may be wrong.

Anyway, it’d be great to start experimenting with the package. Dropping all of the Meteor.bindEnvironment calls every time I use a Promise would be SO nice :smile: Guidance either here or on the README would be :fireworks:

1 Like
2

Ben should also respond here, but he is also working on a special Babel package which uses this package to transpile async/await: https://github.com/meteor/babel/blob/master/options.js#L37

I believe the plan is to use this package for ES6 transpilation in Meteor (It’s already being used in the preview jsx package: http://react-in-meteor.readthedocs.org/en/latest/jsx/)

1 Like
3

@louis I’m glad we agree async/await, Promises, and Fibers are important and exciting :smile: I’m not sure exactly what you’ve tried already, but maybe I can say some things that will help you get unstuck.

Here’s an unreleased package I built that polyfills Promise on both client and server (though of course Fibers only work on the server). That commit is part of an experimental branch that replaces all uses of Future with Promise in the code for the command-line tool.

Using that package (or something like it) should give you a reliable Promise constructor that you can use without any special compilation steps, including Promise.async and Promise.await and a few other Meteor-specific methods.

If you want to use actual ES7 async and await syntax, transpiled to Promise.async and Promise.await, that’s a little trickier, but basically it’s a matter of creating a package that registers a source handler using Plugin.registerSourceHandler, similar to the jsx plugin that @sashko mentioned. I’m planning to release a package for that soon, and I’ll be sure to comment on this thread about my progress.

The package I have in mind will use the meteor-babel NPM package, which includes a special transform that compiles async and await syntax to Promise.asyncApply and Promise.await. For that to work, you have to be using the Promise constructor provided by meteor-promise, and you have to enable async and await using a special option:

var meteorBabel = require("meteor-babel");
var babelOptions = meteorBabel.getDefaultOptions({
  // This option tweaks the default options to enable parsing async/await
  // and also apply the meteor-async-await transform.
  meteorAsyncAwait: true
});
// Modify babelOptions however you like here, before passing them to
// meteorBabel.compile.
var result = meteorBabel.compile(source, babelOptions);
var transpiledCode = result.code;

One small note, since you’ve read the slides: the Babel parser currently does not allow await expressions to appear outside async function bodies, but you can use Promise.await(argument) instead of await argument anywhere you like, as long as your code is running in a Fiber.

1 Like
4

Internally, the core team at Meteor has started using ES6+ via meteor-babel in code that implements the command-line tool. This is a safe and convenient environment because it’s all Node/CommonJS code, and we can experiment there without changing how app or package code behaves.

In case you’re curious how that works, the most important logic is here:

function babelRegister() {
  require("meteor-babel/register")({
    babelOptions: require("meteor-babel").getDefaultOptions(
      require("./babel-features.js")
    )
  });
}

babelRegister(); // #RemoveInProd this line is removed in isopack.js

// Install a global ES6-compliant Promise constructor that knows how to
// run all its callbacks in Fibers.
global.Promise = require("meteor-promise");

Much like require("babel/register"), the require("meteor-babel/register") call adds a require.extensions[".js"] hook to compile all non-NPM .js files using Babel. This technique works for Node code, but it won’t work for Browserify or WebPack code, since those bundlers don’t support the require.extensions API, as far as I know.

2 Likes
5

@benjamn, very cool! Great response, definitely helped get me unstuck.

A Babel plugin to transpile async and await is clever, I didn’t know it had such deep hooks that you could check while parsing whether you were in a Fiber or not. We were talking the other day about whether it’d be possible to parse files for Meteor.isServer blocks and remove them at compile time before sending them to the client. It sounds like that would definitely be within the realm of possibility, but without it baked into core you’d have to register another build plugin on a separate file extension. Unless batch processing is on the horizon and we can process a Javascript file multiple times…

So I’m assuming that past 1.2 we’re not going to have to explicitly bring in meteor-babel as a package, that it’ll be a stage of the build process by default? And probably the unreleased promise package as well? And the babel packages in the meteor/react repo are for if you want to use the actual Babel API in your own package/app. If that’s true, for those of us who are already using ES6 with grigio:babel what’s the best way to start transitioning to using the built-in stuff short of cloning the Meteor repo and running from source?

Jealous you guys get to hack around and play in the tools while we’re still waiting. I wish there were Meteor nightly releases or betas or something that we could use and contribute on. I’m excited for 1.2 to say the least :wink:

6

@benjamn I tried using the meteor-promise package and it works great!.. except inside of a meteor method… Is there a way to return a value to the method?

The best I idea that I could come up with was using a future to grab the 2nd promise:

getUrlTest: function() {
    var future = new Future();

    hellosign.signatureRequest.createEmbedded(options)
      .then(function(response){
        var signatureId = response.signature_request.signatures[0].signature_id;
        return hellosign.embedded.getSignUrl(signatureId);
      })
      .then(function(response){
        future.return(response.embedded.sign_url);
      })
      .catch(function(err){
        console.log(err);
      });

    return future.wait();
  }        
});
7

There’s a Promise.prototype.await method on the Meteor version of the Promise constructor, which should allow you to avoid using Future. Try something like this?

getUrlTest: function() {
  var promise = hellosign.signatureRequest.createEmbedded(options)
    .then(function(response){
      var signatureId = response.signature_request.signatures[0].signature_id;
      return hellosign.embedded.getSignUrl(signatureId);
    })
    .then(function(response){
      return response.embedded.sign_url;
    });

  return promise.await();
}

Now that you’ve brought my attention to it, I’m inclined to make it possible to return a Promise from a method, and the method will automatically wait for the promise to be settled before returning to the caller (but that’s not how it works right now).

1 Like
8

Wow thanks!

Yea I think if you automatically resolved it that would be great (I actually tried to do that just in case lol).

When I run the above I don’t get a promise with an await, do I need to be running the promises branch to get that? Here’s what the returned promise:

{
  "_handler": {
    "consumers": [],
    "resolved": false,
    "state": 0
  }
}

Exception while invoking method 'getUrlTest' TypeError: Object #<Promise> has no method 'await'

I’m using the pollyfill first (and tried after) the hellosign-sdk require:

Promise = Meteor.npmRequire("meteor-promise");
var hellosign = Meteor.npmRequire('hellosign-sdk')({key: '1234'});

Using a Future isn’t the end of the world but I really appreciate you looking into this :smile:

9

There’s now an official promise package that you should be able to install with meteor add promise. The code lives here, and the Atmosphere package can be found here. The package exports the Promise variable on both server and client, though the fiber/await-related methods will only be available on the server version.

Let me know if that does(n’t) work for you!

2 Likes
10

@benjamn
Hmmm same thing. I added the promise package and then tried it again (copy/paste the code above). The server side code runs and logs the url but the meteor method returns the same error:

Exception while invoking method 'requestIframeURL' TypeError: Object #<Promise> has no method 'await'
I20150722-12:27:50.581(-4)?     at [object Object].Meteor.methods.requestIframeURL (app/server/esignature.js:38:20)

and if I console.dir the response of just returning the promise:

{
  "_handler": {
    "consumers": [],
    "resolved": false,
    "state": 0
  }
}

Also running METEOR@1.1.0.2

If it’s helpful for you/MDG I can create a minimal repo to send over.

11

A repro would be great, though one last thing you might consider is that some other code may be replacing the Promise polyfill with its own implementation. I mention this only because that { _handler: ... } structure doesn’t look like the implementation Meteor uses, but definitely looks like it could be some other non-native implementation.

1 Like
12

Maybe the core.js Promise polyfill from grigio:babel?

1 Like
13

Ah good call! I looked into the source and they have when as a dependency used here.

On this when issue they decided to overwrite the native promise and may not in the future when they get to 4.0.

Did you still want a repro or is this issue outside of the scope for the promise package?

14

@benjamn - I’ve been looking at this stuff for a while too, and made this package: https://atmospherejs.com/deanius/promise To be honest, I don’t fully understand your goals of fibers/promises well enough to decide whether your package (and the official meteor Promise package) subsume my functionality or not… Could you take a gander and provide some feedback? Basically my goals are to make client-side callback-accepting functions (HTTP, Meteor.call) work with promises. Yours seem a little more ambitious, but I don’t follow.

15

@deanius There are two main parts to the project of introducing Promises into the Meteor framework:

  • providing a reliable Promise constructor that works with Fibers, works in Node and all browsers, reuses the native implementation if available, does not require Meteor.bindEnvironment, etc.
  • rethinking various APIs like Meteor.call to take full advantage of Promise support.

So far, I have only been working on the first part, and it looks like you’ve started tackling the second part. That’s great, and I hope that the new core package allows you to worry less about the way Promise is implemented.

For the sake of backwards compatibility, it may be difficult for us (the framework team) to change the return type of Meteor.call, or at least it may be easier to provide a companion method that returns a Promise. I see you’re calling that method Meteor.promise. I think I would like to adopt a convention that generalizes to other existing methods, like maybe appending -Async to the original method name. Meteor.call would have a Meteor.callAsync alternate form, and likewise Meteor.apply would have Meteor.applyAsync.

Another interesting possibility for Meteor.call is to allow the function that implements the method to return a Promise, and instead of returning that Promise (which is impossible when you’re trying to send the result from server to client), simply wait for the Promise to be fulfilled before invoking the callback (or un-yielding the Fiber on the server). This is what @SkinnyGeek1010 was asking about above. It’s a more conservative change than altering the return type of Meteor.call, which makes it more likely to ship in Meteor 1.2.

Maybe we should start a new forum thread to brainstorm use cases for Promises in Meteor app code?

1 Like
16

A new thread is fine. Is there already a Trello card or Hackpad?

Yes, it was my hope that with a core package providing a promise polyfill, i wouldn’t have to provide my own.

Just a thought, but the concern about the return type of Meteor.call changing on the client may not be a big. Nobody could have depended on the return type for Meteor.call, since it didn’t return anything… In my latest rev of deanius:promise, Meteor.call without a callback returns a promise. Other libraries I use have been structured in that style (callback omitted=> return promise), and I appreciate its simplicity

I like that better than calling a different method (ala CallAsync) for a few reasons.

  • More Isomorphic to keep the same method name
  • Makes use of a currently non-existent return type (on the client)

I may be missing the use case for having a promise-based implementation of Meteor.call (or HTTP.call etc) on the server. “Meteor gives you ‘sync’ with FIbers on the server, and returns promises or lets you use callbacks on the client”. That hits the use cases I have.

But already this is words enough for its own thread. Thanks for your thoughts, and let me know where we might move the discussion :slight_smile:

17

One more thing- have you seen Sashko’s ReactiveMethod ? There’s something about its implementation that bugs me, w/r/t how easily infinite loops may be inadvertantly introduced. I’m sure if the promise/await/call issue can be settled, there’d be a way to avoid this type of issue:

1 Like
18

Here’s the result of a couple hours of playing around, using async/await - it looks Promising (pun intended haha): http://bit.ly/1S4qFsU

2 Likes
19

@SkinnyGeek1010 I know this is from a while back but I’m wondering if you ever got this to work. I’m taking a crack at trying out the Promise packages and finding a few working usage examples would be a big help.

For what it’s worth at least part of the reason the above doesn’t work is because promise is just a straight variable. There is now reason promise would get the .await() prototype method with the above code, and why you were getting:

TypeError: Object #<Promise> has no method 'await'

Unfortunately I don’t have the answer but I would expect needing to do something that includes creating a new object (var promise = new Promise) or wrapping an existing object with Promise(wrapped promise).

If you do have it figured it out a working example or help with my question here : How to wrap a function wtihin an NPM lib that returns a Promise with the official promise package? would be really appreciated.

20

My project had a change in plans and that feature was delayed so I haven’t had a chance to try this out but the following should work.

The gist is that the promise library was overwriting the global promise which removed the await method.

Meteor.methods({
  getUrl: function(){
    var promise = hellosign.signatureRequest.createEmbedded(options)
        .then(function(response){
          var signatureId = response.signature_request.signatures[0].signature_id;
          return hellosign.embedded.getSignUrl(signatureId);
        })
        .then(function(response){
          return response.embedded.sign_url;
        });

    // promise is from when.js
    // coerce it to a Meteor promise so we know for sure
    // it will have an await() method that does the right thing
    return Promise.resolve(promise).await();
  }
})