Introduction
Hello,
Iām Daniel and our company has built a big Blaze - Everything-and-the-kitchen-sink - App in the last years.
This post is meant to become a āclearinghouseā of the current state of Blaze & Iāll likely update it going forward as new information arrives.
Please feel free to add any kind of information or constructive feedback! Itād be nice if youād include to which part of the post youāre responding to in your replies to keep the discussion focused on the different issues.
Our Stack
Think āclassic meteorā + kitchen sink.
Think Meteor & MongoDb. Blaze. Flow-Router. Pub/Sub and validated methods.
Think Cordova & Desktop. Android. iOS. Push Notifications. Video Calls. Fotos & Audio recording & sending. Custom / patched Cordova packages abandoned by others many years ago
Think multiple Admin-, Back- and Frontend-Interfaces for users in different roles. Think lots of different views. Custom & Handcrafted UI & Styling, all built on a fork of a CSS / UI Framework abandoned years ago (Meteor-Ionic, Based on Ionic Framework 1 I think ). SCSS, obviously.
All of this with full-stack-reactivity, and End-to-End - testing to go.
We have 50+ atmosphere packages in our packages
file.
Current Situation with migrating to Async & preparing for Meteor 3
Since the beginning of the year weāve been working on trying to figure out a way to navigate the maze from Meteor 2.0 to Meteor 3.0
Weāve been in touch with a handful of Projects & Teams whoāre also basing their Stack on Blaze.
Weāve also been supported by @grubba and @radekmie to get the first versions of a workable version of Blaze running.
I wouldnāt say weāre there yet, but
![:fire: :fire:](https://forums.meteor.com/images/emoji/twitter/fire.png?v=12)
I hope weāre now finally approaching something useful - as a stop-gap measure! ![:fire: :fire:](https://forums.meteor.com/images/emoji/twitter/fire.png?v=12)
![:fire: :fire:](https://forums.meteor.com/images/emoji/twitter/fire.png?v=12)
The sentence above is designed to induce terror in anybody having built anything substantial on Blaze, to get in gear to get more eyeballs to look at this & finally start converting their apps to ASYNC!
The state of Blaze, in general
Blaze, in itās entirety, conceptually foots strongly on a) Tracker for reactivity and b) synchronous code to control the entire process of rendering the entire page & data.
Also its data-available-everywhere model when using the minimongo liberally leads to really big issues with data not being fetched centrally, but things being re-run granularly in all different kinds of places in more complex projects / templates.
Which makes it unfortunately really difficult to just wrap access to reactive data sources in a little Tracker.withComputation
block and be done with itā¦ Access to the Tracker.currentComputation
- objects is lost as soon as one async function is being await
ed.
Making it fully Async-Aware would be a major undertaking, bordering on a re-write and possibly a re-integration with all areas it touches within meteor. I donāt think itās going to happen, there seems to be too little interest an resources available to take a real stab at it.
The state of Tracker
Tracker and ASYNC / Await: Cat, meet Dogā¦
Tracker & Async: theyāre unfortunately conceptually basically incompatible. The one (Tracker) builds on a global state being accessible (Tracker.currentComputation
) to enable reactivity.
ASYNC / Await builds on the foundation of āit doesnāt matter when this code gets executed, as long as it gets executed in a plan-able wayā, unfortunately not having a mechanism for storing & restoring global state.
This means, our Tracker loses control of the global variable and canāt control it anymore for ASYNC code as the code execution becomes asynchronous.
TL;DR: ASYNC & TRACKER: TRACKER FUNDAMENTALLY BROKEN.
There are measures to work around this, but they all orbit around the idea of trying to store the current computation somewhere, locally, and passing it into and around all reactive code, around the async/await callbacks, mostly using lexical scoping.
This means we now have to become very conscious of where reactivity comes into play, which autoruns are essential, and which code they call.
Howās tracker / async going for the non-blaze projects going by the way?
It looks like all the integrations into the major frontend frameworks are based on Tracker / autorun, right?
Are there maybe negative effects which might come from helper code suddenly becoming ASYNC there too (and thus breaking the reactivity)?
Or are the patterns there in general unproblematic? The recommended / default react-meteor - package, react-meteor-data seems to rely on Tracker to synchronize & fetch its information & keep it reactive - howās the state of the union over there, guys?
(The same is true for vue & svelte too I think?)
The problem with Blaze - thereās nobody having real āownershipā I guess?
I mean, there are maybe people still using it, and itās getting patched and prodded from multiple sides, but really, thereās no plan here, no central communication & no leadership.
If Iām wrong here and you or someone you know OWNS Blaze and WANTS to lead it into the next years, Iām all ears!
Anywaysā¦ hereās our current āsolutionā stack: ![:syringe: :syringe:](https://forums.meteor.com/images/emoji/twitter/syringe.png?v=12)
![:hocho: :hocho:](https://forums.meteor.com/images/emoji/twitter/hocho.png?v=12)
![:pirate_flag: :pirate_flag:](https://forums.meteor.com/images/emoji/twitter/pirate_flag.png?v=12)
First and foremost: These are basically all bandages around a very sick patient who is bleeding from multiple major woundsā¦ Blaze isnāt looking good and probably isnāt feeling well either!
WITH THESE MEASURES we mean to be able to convert out app to ASYNC to become ready for Meteor 3.0.
But Blaze itself isnāt really ASYNC aware and probably will never be.
So for many this might be the stopgap measure to convert your existing codebase, and start saying goodnight, sweet princeā¦ You were great while you lasted.
and Iāll miss you.
So, on to the meat - the current state of ASYNC Blaze, to the best of my knowledge:
I. Spacebars & Async Data
What would a frontend framework be without itās friendly templating language?
Here a lot of progress has been made, although not everything has been resolved. @radekmie did most of the heavy lifting, though he did make it look easy Cool stuff @radekmie !
Is there any documentation about what works (and what still doesnāt) in Blaze? Well of course not!
Let me give you the low-down, as far as I have it:
a) Implemented async bindings in #let
Thank you @radekmie !
(This is already in Blaze if youāre on the current 2.x branch.)
Thanks to this PR, you can fetch whatever code you want if it returns a promise & stuff it in a variable for the current block:
<template name="example">
{{#let name=getName}}
Hi, {{name}}!
{{/let}}
</template>
Works with both synchronous and asynchronous getName
:
Template.example.helpers({
// Synchronous value.
getName: 'John',
// Asynchronous value.
getName: Promise.resolve('John'),
// Synchronous helper.
getName: () => 'John',
// Asynchronous helper.
getName: async () => 'John',
});
Also provides additional Async state information in the new global helpers @pending
, @rejected
and @resolved
, if youāre into that kind of thing, and it can also accept multiple parameters, eg. {{#let name=getName shoeSize=getShoeSize}}
.
BAM! one thing done, this is a good as a catchall for other things which might not work in the template.
Again, The docs are here: Implemented async bindings in #let. #412
b) Added support for Promise
s in Spacebars.call
and Spacebars.dot
. #413
Again, thank you @radekmie !
(This is also already in Blaze if youāre on the current 2.x branch.)
In his words:
āIn this pull request, I made Spacebars.call
and Spacebars.dot
helpers aware of Promise
s. The former resolves all of the arguments before calling the function, while the latter wraps property accesses in Promise
s.ā
That means that this will now work whenever some part of your hpefully soon-converted-to-ASYNC API (any day now!) starts handing out promises and not result in flames and errors:
From this:
{{#if someFunc ((await calcParamA) (await someOtherparamB)) (await calcParamB (someOtherparamC) (someOtherparamD)) }}
(...)
{{/if}}
This will have to unwrap the Promise
, but the entire computation will be simple:
{{#let result=(someFunc (calcParamA someOtherparamB) (calcParamB someOtherparamC someOtherparamD))}}
{{#if result}}
(...)
{{/if}}
{{/let}}
Of course, itāll be possible to use the @pending
, @rejected
, and @resolved
helpers to check the progress of calculating result
as well.
Ah, isnāt that just beautiful?
Response: No.
But at least itās something!
c) Added support for Promise
s in #if
and #each
. #424
NOTE: Not in Blaze / your Meteor 2.13 or what have you yet (see below for how to put together my / our current stack):
Third cheers to @radekmie here! Heās the real hero Blaze needed, he took the time to make this work to help out us poor Blazers.
Again, see the PR for more details: Added support for Promise
s in #if
and #each
. #424
Basically, Radek is I following up on #412 and #413 above by supporting Promise
s in #if
/#unless
and #each
. The idea is basically the same as how it works in #let
except for where we actually store the ReactiveVar
s.
So that means, you can use your helper / data / whatever in {{#if}}
's and {{#unless}}
and {{#each}}
's again - I mean it would have been a sad thing if we couldnāt have used those going forward anymore, right?
TODO: This should get checked and merged soon-ish for the transition to continue.
d) What doesnāt work (without a little help from your new friend, the await
template)?
Aside: Thank you again, @radekmie! for this idea
!
So what doesnāt work here?
Hahahaā¦ outputting async values / promise results in templates, for example!
Like this:
{{myAsyncHelperResult}}
Sorry, canāt do that!
But fortunately, our new friends, the await
and the awaitRaw
- templates can help us here, by creating a view which can then react once the data arrives:
{{> await myAsyncHelperResult}}
(You could also use a {{#let }}
of course, but I like the simplicity of not having to wrap too many things.)
Where you can find these awesome helpers? Just put them in a .html file in your project & import it, here they are (- Gist here, same thing):
<!--
Display potentially async data source results in the template. Unsafe Content / HTML will be escaped like with regular {{ someValue }} blaze syntax.
Use like this:
{{> await someValue}}
-->
<template name="await">{{#let value=.}}{{value}}{{/let}}</template>
<!--
Display potentially async data source results in the template. Unsafe Content / HTML will NOT be escaped, similar to using the triple bracket {{{ someValue }}} blaze syntax.
Use like this:
{{> awaitRaw someValue}}
-->
<template name="awaitRaw">{{#let value=.}}{{{value}}}{{/let}}</template>
Ta-Daa!
e) What doesnāt work Pt II: Mostly things in attributes, basically:
None of these will work as soon as thereās some async going on:
<div class="{{#if getHighlight}}highlight{{/if}}"></div>
<div class={{getAwesomeClassList}}></div>
etc jadda jadda. Which is a bit sad.
f) - Bonus why is all / some of this so difficult?
If a spacebars language expression doesnāt create its own view, then itās difficult to handle. Thatās the main point why we canāt just create new await
helpers and such, because thereād be no point in the rendering process where the template could actually be rerun once the async data has been resolved / rejected.
So the fixes & hacks from above all work because thereās some place in the template where thereās a view which can be re-rendered when its state changes.
Everything else and you start to have to touch the spacebars compiler, and the whole framework, basically, and itās all really complicated. See the red/blue problem: What Color is Your Function? ā journal.stuffwithstuff.com ā¦
g) what are further workarounds to keep in mind?
ā Stuff stuff into reactive vars & fetch from there
So, basically everything you want to do in templates you can still do - if you go the detour & use reactive vars & share those with the template & use those.
For example, Iāve enjoyed using the template-controller
- package a lot in the last years, which allows you to add stuff to a reactive dictionary called state and access that in the template, Iām sure there many other ways too to do this without adding extra helpers for everything.
For example for the attribute
helpers above you could do something like this:
<template name="coolStuff"
<div class="{{#if getHighlight}}highlight{{/if}}"></div>
<div class={{getAwesomeClassList}}></div>
</template>
Add this to the template.js:
TemplateController('coolStuff', {
state: {
hasHighlight,
awesomeClassList
},
async onCreated() {
this.state.hasHighlight = await myCollection.findOne().highlight
this.state.awesomeClassList = await awesomeClassesCollection.findOne().awesomeClasses
}
})
state
is an automatic helper in all TemplateController templates, so we can use spacebars
in our attributes again now:
<template name="coolStuff"
<div class="{{#if state.hasHighlight}}highlight{{/if}}"></div>
<div class={{state.awesomeClassList}}></div>
</template>
āBut, but, that wonāt work in my {{#each}}'es and what have you out of the box!?!ā
I know, Iām crying tooā¦
Weāll have to deal with it. There are probably many ways to slice this, we just canāt have the promises end up in the attributes of our HTML elements.
II. Babel Plugin to enable autorun across async/await yields
Basically this: Babel Plugin to enable autorun across async/await yields
Hereās a link to a gist of the plugin: Meteor Async Transition: Babel Plugin to keep Tracker Computation alive after "await"s
Itās a Babel plugin we wrote to allow for rudimentary automatic passing of Tracker.currentComputation
in computation contexts across async function calls and awaits contained therein.
You can see more details & examples in the link above.
The good:
- works for simple code, async calls and simple assignments from async calls, mostly
- doesnāt break complex code (I think!)
- is only executed for client code / skips server code
- seems to be working in our relatively big & complex Blaze app
- Can be used as a stopgap measure to transition codebases into the ASYNC world
Caveats:
- could become more optimized
- could be extended for slightly more complex cases
- adds build & runtime overhead
Whether this is a good idea & should be used in wider circulation remains to be seen But itās there in case you want to give it a go!
By the way: This could be used outside of Blaze for any other Tracker code as well, Hint Hint to the react / vue / svelte gang ![:slight_smile: :slight_smile:](https://forums.meteor.com/images/emoji/twitter/slight_smile.png?v=12)
III. - Allow await-ing autoruns by adding firstRunPromise to computation objects
This PR: # Allow await-ing autoruns by adding firstRunPromise to computation objects adds a new, await
-able field, Computation.firstRunPromise
to newly created computations to allow autorun blocks with async autorun functions to be run in a predictable manner.
Note: This hasnāt been merged yet into Meteor, but you could still try it out anyhowā¦ Iāll get to finishing the open tasks on the PR as soon as I can. I think functionality-wise itās pretty final.
This allows you to await
autorun functions until theyāve been finalized:
Template.onCreated(async function() {
// autorun 1
await this.autorun(async(c) => {
// 1 - 1
await Tracker.withComputation(c, () => { ... })
// 1 - 2
await Tracker.withComputation(c, () => { ... })
// 1 - 3
// (...)
}).firstRunPromise
// autorun 2
await this.autorun(async(c) => {
// 2 - 1
await Tracker.withComputation(c, () => { ... })
// 2 - 2
await Tracker.withComputation(c, () => { ... })
// 2 - 3
// (...)
}).firstRunPromise
// autorun 3
await this.autorun(async(c) => {
// 3 - 1
await Tracker.withComputation(c, () => { ... })
// 3 - 2
await Tracker.withComputation(c, () => { ... })
// 3 - 3
// (...)
}).firstRunPromise
})
Without awaiting the firstRunPromise for all autoruns, each autorun function would āyieldā / leave the current thread on the first await
& the thread would zig-zag / ping-pong between autoruns as the individual.
Without the await this.autorun().firstRunPromise
, the order of the comments being reached up top would be:
1-1; 2-1; 3-1; (Now all bets are off, but maybe): 1-2, 2-2, 3-2, 1-3, 2-3, 3-3
etc etc.
Which could lead to a) lots of unnecessary re-runs and b) to actual race conditions where autoruns would run and complete in different orders, thus leading to unpredictable and random behaviours.
But - await
the firstRunPromise
s - and at least you can force your autoruns to run in a repeatable and predictable fashion again.
Some final points: Blaze was never made for this - async vs. lifecycle callbacks; Blaze rendering in general; What next.
Have a look at chapter III aboveā¦
I said ānow you can await
your autorunsā.
But that means you have to turn your lifecycle function into an async
function.
Blaze doesnāt await
its lifecycle functions. And itās also not easy to make it wait for it because it continues to render all children as soon as one call returnsā¦
So turning you onCreated
into an async function means that now your onRendered will be called before the async onCreated function has been completed.
**I actually donāt know how many issues still slumber below the surface ** besides the ones we have patched & tried to mitigate, see above.
Where do we go from here?
As I mentioned in the beginning,
I think there are basically two ways to go here -
a) a rewrite / total reevaluation of blaze - which probably will not happen.
b) we can continue to try to mitigate issues the best we can as they pop up and try to convert our existing codebases to ASYNC - for the short term. But Blaze will probably never become a fully supported citizen of the ASYNC world of Meteor 3.0 .
The latter is what Iāll continue to try to be doing. Itās a bit sad because it means accepting that Blaze will never be āfixedā for Async and will live in a continuous state of āknown brokennessā until the end of its days.
Our little demo / template project
Here is a little sample project which contains all the code mentioned above in one package - plus some sandbox code which isnāt really guaranteed to do anything for you
But you can use it as a grab-bag to get all the mentioned code, merged or not, in one handy package right now, here: - Blaze Async Sandbox
To use this āstackā in your own project, the easiest thing to do would be to
- clone the project above
- copy the following packages from the
packages
directory to your own projectsāpackages
directory in theconvert-to-async
- branch of your repository:blaze
tracker
Copy the babel plugin plugin-trackerAsyncMitigation.js
from the babelPlugin
folder into a cozy place in your project and update your .babelrc
like eg. this:
{
"presets": ["@babel/preset-env"],
"plugins": ["./babelPlugin/plugin-trackerAsyncMitigation.js"]
}
NOTE: you have to delete your meteor build cache once (eg. run meteor reset
once) after adding the babel plugin in order to get its effects applied to your codebase.
Calling all Canaries / Blaze users
These are all the people I know who have shown or expressed an interest in Blaze recently. Iāll mention them here to GET YOU IN HERE!!! HELLOOO!!! NICE TO SEE YOU GUYS!
Hi @fredmaiaarantes @TatiBarros @grubba @radekmie!
Hi @sebastianspiller and @thebarty !
Hi @alimgafar, @jkuester, @storyteller and @dr.dimitru !
This one has been a long time in the making, thank you all for your help so far.
But now Iād need your help again: Please reply, be heard, tell us about your current status re: Blaze & Async! Your news, your updates, your interests going forward, and all feedback in general! Anything. Please! (Just do it!!!
)
And, very impotant: WHOEVER YOU MIGHT KNOW, WHO WORKS WITH BLAZE, FOR WHOM IT MIGHT BE IMPORTANT TO GET INTO THE KNOW HERE, PLEASE PLEASE PLEASE MENTION THEM IN THE COMMENTS BELOW!
OR SHARE THIS POST WITH THEM, VIA EMAIL, VIA CHAT, TWITTER, X, Y, THREADS, LINKEDIN, WHATSAPP OR WHAT-HAVE-YOU!
It is important that we ORGANIZE ourselves and that we get a good picture of who still uses Blaze, what is still being done out there with Blaze! and how we move forwards TOGETHER!!!
Thank you guys!
Slow claps from the only two chaps in the audience