Introduction
Hello,
I’m Daniel and our company has built a big Blaze - Everything-and-the-kitchen-sink - App in the last years.
This post is meant to become a “clearinghouse” of the current state of Blaze & I’ll likely update it going forward as new information arrives.
Please feel free to add any kind of information or constructive feedback! It’d be nice if you’d include to which part of the post you’re responding to in your replies to keep the discussion focused on the different issues.
Our Stack
Think “classic meteor” + kitchen sink.
Think Meteor & MongoDb. Blaze. Flow-Router. Pub/Sub and validated methods.
Think Cordova & Desktop. Android. iOS. Push Notifications. Video Calls. Fotos & Audio recording & sending. Custom / patched Cordova packages abandoned by others many years ago
Think multiple Admin-, Back- and Frontend-Interfaces for users in different roles. Think lots of different views. Custom & Handcrafted UI & Styling, all built on a fork of a CSS / UI Framework abandoned years ago (Meteor-Ionic, Based on Ionic Framework 1 I think ). SCSS, obviously.
All of this with full-stack-reactivity, and End-to-End - testing to go.
We have 50+ atmosphere packages in our packages
file.
Current Situation with migrating to Async & preparing for Meteor 3
Since the beginning of the year we’ve been working on trying to figure out a way to navigate the maze from Meteor 2.0 to Meteor 3.0
We’ve been in touch with a handful of Projects & Teams who’re also basing their Stack on Blaze.
We’ve also been supported by @grubba and @radekmie to get the first versions of a workable version of Blaze running.
I wouldn’t say we’re there yet, but
![:fire: :fire:](https://forums.meteor.com/images/emoji/twitter/fire.png?v=12)
I hope we’re now finally approaching something useful - as a stop-gap measure! ![:fire: :fire:](https://forums.meteor.com/images/emoji/twitter/fire.png?v=12)
![:fire: :fire:](https://forums.meteor.com/images/emoji/twitter/fire.png?v=12)
The sentence above is designed to induce terror in anybody having built anything substantial on Blaze, to get in gear to get more eyeballs to look at this & finally start converting their apps to ASYNC!
The state of Blaze, in general
Blaze, in it’s entirety, conceptually foots strongly on a) Tracker for reactivity and b) synchronous code to control the entire process of rendering the entire page & data.
Also its data-available-everywhere model when using the minimongo liberally leads to really big issues with data not being fetched centrally, but things being re-run granularly in all different kinds of places in more complex projects / templates.
Which makes it unfortunately really difficult to just wrap access to reactive data sources in a little Tracker.withComputation
block and be done with it… Access to the Tracker.currentComputation
- objects is lost as soon as one async function is being await
ed.
Making it fully Async-Aware would be a major undertaking, bordering on a re-write and possibly a re-integration with all areas it touches within meteor. I don’t think it’s going to happen, there seems to be too little interest an resources available to take a real stab at it.
The state of Tracker
Tracker and ASYNC / Await: Cat, meet Dog…
Tracker & Async: they’re unfortunately conceptually basically incompatible. The one (Tracker) builds on a global state being accessible (Tracker.currentComputation
) to enable reactivity.
ASYNC / Await builds on the foundation of “it doesn’t matter when this code gets executed, as long as it gets executed in a plan-able way”, unfortunately not having a mechanism for storing & restoring global state.
This means, our Tracker loses control of the global variable and can’t control it anymore for ASYNC code as the code execution becomes asynchronous.
TL;DR: ASYNC & TRACKER: TRACKER FUNDAMENTALLY BROKEN.
There are measures to work around this, but they all orbit around the idea of trying to store the current computation somewhere, locally, and passing it into and around all reactive code, around the async/await callbacks, mostly using lexical scoping.
This means we now have to become very conscious of where reactivity comes into play, which autoruns are essential, and which code they call.
How’s tracker / async going for the non-blaze projects going by the way?
It looks like all the integrations into the major frontend frameworks are based on Tracker / autorun, right?
Are there maybe negative effects which might come from helper code suddenly becoming ASYNC there too (and thus breaking the reactivity)?
Or are the patterns there in general unproblematic? The recommended / default react-meteor - package, react-meteor-data seems to rely on Tracker to synchronize & fetch its information & keep it reactive - how’s the state of the union over there, guys?
(The same is true for vue & svelte too I think?)
The problem with Blaze - there’s nobody having real “ownership” I guess?
I mean, there are maybe people still using it, and it’s getting patched and prodded from multiple sides, but really, there’s no plan here, no central communication & no leadership.
If I’m wrong here and you or someone you know OWNS Blaze and WANTS to lead it into the next years, I’m all ears!
Anyways… here’s our current “solution” stack: ![:syringe: :syringe:](https://forums.meteor.com/images/emoji/twitter/syringe.png?v=12)
![:hocho: :hocho:](https://forums.meteor.com/images/emoji/twitter/hocho.png?v=12)
![:pirate_flag: :pirate_flag:](https://forums.meteor.com/images/emoji/twitter/pirate_flag.png?v=12)
First and foremost: These are basically all bandages around a very sick patient who is bleeding from multiple major wounds… Blaze isn’t looking good and probably isn’t feeling well either!
WITH THESE MEASURES we mean to be able to convert out app to ASYNC to become ready for Meteor 3.0.
But Blaze itself isn’t really ASYNC aware and probably will never be.
So for many this might be the stopgap measure to convert your existing codebase, and start saying goodnight, sweet prince… You were great while you lasted.
and I’ll miss you.
So, on to the meat - the current state of ASYNC Blaze, to the best of my knowledge:
I. Spacebars & Async Data
What would a frontend framework be without it’s friendly templating language?
Here a lot of progress has been made, although not everything has been resolved. @radekmie did most of the heavy lifting, though he did make it look easy Cool stuff @radekmie !
Is there any documentation about what works (and what still doesn’t) in Blaze? Well of course not!
Let me give you the low-down, as far as I have it:
a) Implemented async bindings in #let
Thank you @radekmie !
(This is already in Blaze if you’re on the current 2.x branch.)
Thanks to this PR, you can fetch whatever code you want if it returns a promise & stuff it in a variable for the current block:
<template name="example">
{{#let name=getName}}
Hi, {{name}}!
{{/let}}
</template>
Works with both synchronous and asynchronous getName
:
Template.example.helpers({
// Synchronous value.
getName: 'John',
// Asynchronous value.
getName: Promise.resolve('John'),
// Synchronous helper.
getName: () => 'John',
// Asynchronous helper.
getName: async () => 'John',
});
Also provides additional Async state information in the new global helpers @pending
, @rejected
and @resolved
, if you’re into that kind of thing, and it can also accept multiple parameters, eg. {{#let name=getName shoeSize=getShoeSize}}
.
BAM! one thing done, this is a good as a catchall for other things which might not work in the template.
Again, The docs are here: Implemented async bindings in #let. #412
b) Added support for Promise
s in Spacebars.call
and Spacebars.dot
. #413
Again, thank you @radekmie !
(This is also already in Blaze if you’re on the current 2.x branch.)
In his words:
“In this pull request, I made Spacebars.call
and Spacebars.dot
helpers aware of Promise
s. The former resolves all of the arguments before calling the function, while the latter wraps property accesses in Promise
s.”
That means that this will now work whenever some part of your hpefully soon-converted-to-ASYNC API (any day now!) starts handing out promises and not result in flames and errors:
From this:
{{#if someFunc ((await calcParamA) (await someOtherparamB)) (await calcParamB (someOtherparamC) (someOtherparamD)) }}
(...)
{{/if}}
This will have to unwrap the Promise
, but the entire computation will be simple:
{{#let result=(someFunc (calcParamA someOtherparamB) (calcParamB someOtherparamC someOtherparamD))}}
{{#if result}}
(...)
{{/if}}
{{/let}}
Of course, it’ll be possible to use the @pending
, @rejected
, and @resolved
helpers to check the progress of calculating result
as well.
Ah, isn’t that just beautiful?
Response: No.
But at least it’s something!
c) Added support for Promise
s in #if
and #each
. #424
NOTE: Not in Blaze / your Meteor 2.13 or what have you yet (see below for how to put together my / our current stack):
Third cheers to @radekmie here! He’s the real hero Blaze needed, he took the time to make this work to help out us poor Blazers.
Again, see the PR for more details: Added support for Promise
s in #if
and #each
. #424
Basically, Radek is I following up on #412 and #413 above by supporting Promise
s in #if
/#unless
and #each
. The idea is basically the same as how it works in #let
except for where we actually store the ReactiveVar
s.
So that means, you can use your helper / data / whatever in {{#if}}
's and {{#unless}}
and {{#each}}
's again - I mean it would have been a sad thing if we couldn’t have used those going forward anymore, right?
TODO: This should get checked and merged soon-ish for the transition to continue.
d) What doesn’t work (without a little help from your new friend, the await
template)?
Aside: Thank you again, @radekmie! for this idea
!
So what doesn’t work here?
Hahaha… outputting async values / promise results in templates, for example!
Like this:
{{myAsyncHelperResult}}
Sorry, can’t do that!
But fortunately, our new friends, the await
and the awaitRaw
- templates can help us here, by creating a view which can then react once the data arrives:
{{> await myAsyncHelperResult}}
(You could also use a {{#let }}
of course, but I like the simplicity of not having to wrap too many things.)
Where you can find these awesome helpers? Just put them in a .html file in your project & import it, here they are (- Gist here, same thing):
<!--
Display potentially async data source results in the template. Unsafe Content / HTML will be escaped like with regular {{ someValue }} blaze syntax.
Use like this:
{{> await someValue}}
-->
<template name="await">{{#let value=.}}{{value}}{{/let}}</template>
<!--
Display potentially async data source results in the template. Unsafe Content / HTML will NOT be escaped, similar to using the triple bracket {{{ someValue }}} blaze syntax.
Use like this:
{{> awaitRaw someValue}}
-->
<template name="awaitRaw">{{#let value=.}}{{{value}}}{{/let}}</template>
Ta-Daa!
e) What doesn’t work Pt II: Mostly things in attributes, basically:
None of these will work as soon as there’s some async going on:
<div class="{{#if getHighlight}}highlight{{/if}}"></div>
<div class={{getAwesomeClassList}}></div>
etc jadda jadda. Which is a bit sad.
f) - Bonus why is all / some of this so difficult?
If a spacebars language expression doesn’t create its own view, then it’s difficult to handle. That’s the main point why we can’t just create new await
helpers and such, because there’d be no point in the rendering process where the template could actually be rerun once the async data has been resolved / rejected.
So the fixes & hacks from above all work because there’s some place in the template where there’s a view which can be re-rendered when its state changes.
Everything else and you start to have to touch the spacebars compiler, and the whole framework, basically, and it’s all really complicated. See the red/blue problem: What Color is Your Function? – journal.stuffwithstuff.com …
g) what are further workarounds to keep in mind?
→ Stuff stuff into reactive vars & fetch from there
So, basically everything you want to do in templates you can still do - if you go the detour & use reactive vars & share those with the template & use those.
For example, I’ve enjoyed using the template-controller
- package a lot in the last years, which allows you to add stuff to a reactive dictionary called state and access that in the template, I’m sure there many other ways too to do this without adding extra helpers for everything.
For example for the attribute
helpers above you could do something like this:
<template name="coolStuff"
<div class="{{#if getHighlight}}highlight{{/if}}"></div>
<div class={{getAwesomeClassList}}></div>
</template>
Add this to the template.js:
TemplateController('coolStuff', {
state: {
hasHighlight,
awesomeClassList
},
async onCreated() {
this.state.hasHighlight = await myCollection.findOne().highlight
this.state.awesomeClassList = await awesomeClassesCollection.findOne().awesomeClasses
}
})
state
is an automatic helper in all TemplateController templates, so we can use spacebars
in our attributes again now:
<template name="coolStuff"
<div class="{{#if state.hasHighlight}}highlight{{/if}}"></div>
<div class={{state.awesomeClassList}}></div>
</template>
“But, but, that won’t work in my {{#each}}'es and what have you out of the box!?!”
I know, I’m crying too…
We’ll have to deal with it. There are probably many ways to slice this, we just can’t have the promises end up in the attributes of our HTML elements.
II. Babel Plugin to enable autorun across async/await yields
Basically this: Babel Plugin to enable autorun across async/await yields
Here’s a link to a gist of the plugin: Meteor Async Transition: Babel Plugin to keep Tracker Computation alive after "await"s
It’s a Babel plugin we wrote to allow for rudimentary automatic passing of Tracker.currentComputation
in computation contexts across async function calls and awaits contained therein.
You can see more details & examples in the link above.
The good:
- works for simple code, async calls and simple assignments from async calls, mostly
- doesn’t break complex code (I think!)
- is only executed for client code / skips server code
- seems to be working in our relatively big & complex Blaze app
- Can be used as a stopgap measure to transition codebases into the ASYNC world
Caveats:
- could become more optimized
- could be extended for slightly more complex cases
- adds build & runtime overhead
Whether this is a good idea & should be used in wider circulation remains to be seen But it’s there in case you want to give it a go!
By the way: This could be used outside of Blaze for any other Tracker code as well, Hint Hint to the react / vue / svelte gang ![:slight_smile: :slight_smile:](https://forums.meteor.com/images/emoji/twitter/slight_smile.png?v=12)
III. - Allow await-ing autoruns by adding firstRunPromise to computation objects
This PR: # Allow await-ing autoruns by adding firstRunPromise to computation objects adds a new, await
-able field, Computation.firstRunPromise
to newly created computations to allow autorun blocks with async autorun functions to be run in a predictable manner.
Note: This hasn’t been merged yet into Meteor, but you could still try it out anyhow… I’ll get to finishing the open tasks on the PR as soon as I can. I think functionality-wise it’s pretty final.
This allows you to await
autorun functions until they’ve been finalized:
Template.onCreated(async function() {
// autorun 1
await this.autorun(async(c) => {
// 1 - 1
await Tracker.withComputation(c, () => { ... })
// 1 - 2
await Tracker.withComputation(c, () => { ... })
// 1 - 3
// (...)
}).firstRunPromise
// autorun 2
await this.autorun(async(c) => {
// 2 - 1
await Tracker.withComputation(c, () => { ... })
// 2 - 2
await Tracker.withComputation(c, () => { ... })
// 2 - 3
// (...)
}).firstRunPromise
// autorun 3
await this.autorun(async(c) => {
// 3 - 1
await Tracker.withComputation(c, () => { ... })
// 3 - 2
await Tracker.withComputation(c, () => { ... })
// 3 - 3
// (...)
}).firstRunPromise
})
Without awaiting the firstRunPromise for all autoruns, each autorun function would “yield” / leave the current thread on the first await
& the thread would zig-zag / ping-pong between autoruns as the individual.
Without the await this.autorun().firstRunPromise
, the order of the comments being reached up top would be:
1-1; 2-1; 3-1; (Now all bets are off, but maybe): 1-2, 2-2, 3-2, 1-3, 2-3, 3-3
etc etc.
Which could lead to a) lots of unnecessary re-runs and b) to actual race conditions where autoruns would run and complete in different orders, thus leading to unpredictable and random behaviours.
But - await
the firstRunPromise
s - and at least you can force your autoruns to run in a repeatable and predictable fashion again.
Some final points: Blaze was never made for this - async vs. lifecycle callbacks; Blaze rendering in general; What next.
Have a look at chapter III above…
I said “now you can await
your autoruns”.
But that means you have to turn your lifecycle function into an async
function.
Blaze doesn’t await
its lifecycle functions. And it’s also not easy to make it wait for it because it continues to render all children as soon as one call returns…
So turning you onCreated
into an async function means that now your onRendered will be called before the async onCreated function has been completed.
**I actually don’t know how many issues still slumber below the surface ** besides the ones we have patched & tried to mitigate, see above.
Where do we go from here?
As I mentioned in the beginning,
I think there are basically two ways to go here -
a) a rewrite / total reevaluation of blaze - which probably will not happen.
b) we can continue to try to mitigate issues the best we can as they pop up and try to convert our existing codebases to ASYNC - for the short term. But Blaze will probably never become a fully supported citizen of the ASYNC world of Meteor 3.0 .
The latter is what I’ll continue to try to be doing. It’s a bit sad because it means accepting that Blaze will never be “fixed” for Async and will live in a continuous state of “known brokenness” until the end of its days.
Our little demo / template project
Here is a little sample project which contains all the code mentioned above in one package - plus some sandbox code which isn’t really guaranteed to do anything for you
But you can use it as a grab-bag to get all the mentioned code, merged or not, in one handy package right now, here: - Blaze Async Sandbox
To use this “stack” in your own project, the easiest thing to do would be to
- clone the project above
- copy the following packages from the
packages
directory to your own projects’packages
directory in theconvert-to-async
- branch of your repository:blaze
tracker
Copy the babel plugin plugin-trackerAsyncMitigation.js
from the babelPlugin
folder into a cozy place in your project and update your .babelrc
like eg. this:
{
"presets": ["@babel/preset-env"],
"plugins": ["./babelPlugin/plugin-trackerAsyncMitigation.js"]
}
NOTE: you have to delete your meteor build cache once (eg. run meteor reset
once) after adding the babel plugin in order to get its effects applied to your codebase.
Calling all Canaries / Blaze users
These are all the people I know who have shown or expressed an interest in Blaze recently. I’ll mention them here to GET YOU IN HERE!!! HELLOOO!!! NICE TO SEE YOU GUYS!
Hi @fredmaiaarantes @TatiBarros @grubba @radekmie!
Hi @sebastianspiller and @thebarty !
Hi @alimgafar, @jkuester, @storyteller and @dr.dimitru !
This one has been a long time in the making, thank you all for your help so far.
But now I’d need your help again: Please reply, be heard, tell us about your current status re: Blaze & Async! Your news, your updates, your interests going forward, and all feedback in general! Anything. Please! (Just do it!!!
)
And, very impotant: WHOEVER YOU MIGHT KNOW, WHO WORKS WITH BLAZE, FOR WHOM IT MIGHT BE IMPORTANT TO GET INTO THE KNOW HERE, PLEASE PLEASE PLEASE MENTION THEM IN THE COMMENTS BELOW!
OR SHARE THIS POST WITH THEM, VIA EMAIL, VIA CHAT, TWITTER, X, Y, THREADS, LINKEDIN, WHATSAPP OR WHAT-HAVE-YOU!
It is important that we ORGANIZE ourselves and that we get a good picture of who still uses Blaze, what is still being done out there with Blaze! and how we move forwards TOGETHER!!!
Thank you guys!
Slow claps from the only two chaps in the audience