@faceyspacey here is an in-depth explanation on how Mobservable finds the minimum amount of computed values that needs to re-evaluate, without ever introducing glitches or staleness: https://medium.com/@mweststrate/becoming-fully-reactive-an-in-depth-explanation-of-mobservable-55995262a254#.xpxqzvkur
1 Like
Excellent! I’ll check it out.
Left u a response here:
And on ur medium article. I think basically Mobservable should replace Tracker. @sashko MDG should really take a look at this and let us know the pitfalls they see. There clearly will be some walls we run up against since we have a huge number of tools built on top of tracker. For its behavior to change in any significant way may cause things to break. But I think if we could prove Mobservable and the fixes it provides is a major enhancement we could get buy-in to upgrade tools that depend on it. Eg session, collection cursors, blaze helpers, etc.
1 Like
I PM’ed faceyspacey some upgrades on tracker that I have been using at my company and he said that I should post them here. Here is the original message:
I read your Medium post about Tracker 2.0 and I thought that it was very insightful. I’ve got a couple helper functions monkey patched onto Tracker that I think you might find interesting. Here is a Gist: https://gist.github.com/veered/30ae6f79ec48506af80f
I think these helpers are a step in the right direction. And I think the code at the bottom (which I haven’t spent any time on improving after I had the original idea), could be used to build out way better debugging tools for Tracker.
If he would accept this challenge and would say how much he needs to start working, I would donate (in BTC) him for sure 
@faceyspacey, @mweststrate this is my opinion and experience trying to make Tracker sync via ReactiveState modifiers added to the A neat diagram comparing Meteor, Flux, and Relay post.
I’d love your comments about this stuff, regarding a possible Tracker 2.0 implementation.
I have not read the @mweststrate article about the minimum amount of computed values yet but I will do as soon as I can and I will post my thoughts here as well.
This is the post:
Mobservable is a great package. It’s transparent reactive programming (yeah, Tracker) for the React world and it shows the advantages of TRP in terms of code which is simple to reason about and less prone to edge case bugs and even performance gains when used right (check this https://www.mendix.com/tech-blog/making-react-reactive-pursuit-high-performing-easily-maintainable-react-apps/ out).
The main differences I see between Mobservable and Tracker are two:
-
Mobservable has all state inside and Tracker has state outside.
-
Mobservable is sync and Tracker is async.
Both are related. Sync stuff is easier to debug than async stuff.
I tried to make my ReactiveState package sync because when you own all state, you can execute stuff synchronously:
State.modify('tasks.areEmpty', (state = true) => {
if (State.get('tasks.items').length === 0)
return true;
else
return false;
});
State.modify('tasks.items', (state = []) => {
return Tasks.find().fetch();
});
When you are evaluating tasks.areEmpty you can stop at the State.get('tasks.items') and execute its modify first, make sure it is up-to-date and then continue with tasks.areEmpty. Functions are executed synchronously and only once. Perfect!
The problem with that is, once again: Meteor reactivity comes from many places, not only a single tree. For example, add another reactive source to the mix: Meteor.userId().
State.modify('user.hasPendingTasks', (state = false) => {
if (Meteor.userId() && State.get('user.tasks').length === 0)
return true;
else
return false;
});
State.modify('user.tasks', (state = []) => {
let userId = Meteor.userId();
return Tasks.find({ userId }).fetch();
});
If you want to make this work synchronously, you need to know if Meteor.userId() has already changed or is going to change before executing any of the State modifiers.
The sad truth is: you cannot know. Tracker reactivity has no priorities and is spread in many places along Meteor. So the solution is what Meteor people already did: make Tracker async and wait for every reactive source to invalidate its computations. Then start executing computations.
For that reason, I don’t think Mobservable is useful or has any advantage over Tracker in Meteor if you don’t want to throw a lot of stuff which is already working with Tracker.
For Tracker, this means sometimes functions are going to be executed twice or even more times until they reach an stable state. That’s the reason MeteorFlux actions have three phases:
- Register callbacks (sync non-reactive).
- State.modifies or Tracker.autoruns (async reactive, it keeps executing functions until everything is stable).
- AfterAction callbacks (sync after all reactivity has finished and is stable, non-reactive).
Explained like this, this looks like really complex stuff, but in the end coding with TRP is simpler and less prone to bugs. Most of the time you don’t have to know about how it works internally and that’s what makes Tracker and Meteor so simple to start with.
2 Likes
Ok, I have read your article @mweststrate 
I really like your implementation. It’s similar to what I tried to achieve with my ReactiveState package. And the problem I wrote in the last post could be solved by your transaction pattern. Actually what Tracker does with its async approach is to make the current call stack a transaction and finishes it when everything has been executed and the CPU is idle again.
I have a question: why do think remaining sync is so important? Debugging, testing?
The other problem I had with my ReactiveState implementation was that I could only control what was inside ReactiveState, so reactive stuff outside the system broke the execution chain. But if instead of using a package on top of Tracker we re-write it (Tracker 2.0?) or find a way to use Mobservable instead, we could control the whole dependency tree and optimise the number of computations.
I think Tracker 2.0 could use two important performance improvements from Mobservable:
- Make a proper execution chain derived from dependencies so computations are executed in a optimal order, not randomly like now.
- Check for value changes in that chain so it can stop unnecessary executions.
I tried the first one and I am already doing the second one with ReactiveState:
// This retriggers when ANYTHING inside the post object change, not only title.
Tracker.autorun(() => {
let postTitle = Posts.findOne().title;
});
// But using ReactiveState to store the state...
State.modify('post', (state = {}) => {
return Posts.findOne();
});
// ...it retriggers only when the post title has changed.
Tracker.autorun(() => {
let postTitle = State.get('post.title');
});
Food for thought, really. Thanks for sharing all this amazing stuff!
4 Likes
Maybe community drive Tracker 2.0 can be implemented on top of mobservable.
Backwards compatibility relay in the fact that rerun behavior of actual implementation is a little “undetermined”.
Mobservable project go back us to good times of reactive data. I don´t really like the imperative aproach of ‘Flux actions’ and explicit ‘tree of truth’.
1 Like
@luisherranz Could you elaborate a bit on what you mean by internal vs. external state? Mobservable doesn’t rely on stores, single state tree or other dedicated places to stores state like in flux architectures. Anything that is decorated using observable can be used a source of reactive data, no matter where the data originates from. So I don’t think your state / userid example above wouldn’t be an issue for Mobservable.
Sync behavior is quite fundamental to Mobservable (it is quite similar to Javelin in this regard); asynchronicity shouldn’t be used as means for scheduling; the only reliable source for correct execution order is the dependency graph. Once you have that, running stuff asynchronous only complicates stuff unecessarily; one can observe glitches, or introduce really nasty bugs like:
// pseudo code:
x = 1;
y <= x * 2 // computed value
x = 3;
if (y > 4) {
// will this code run? impossible to answer in an asynchronous world...
}
Note that running asynchronous doesn’t prevent glitches from happening. You still have to make sure that everything in the right order. So it doesn’t solve the fundamental problem of running the computations in the right order and minimizing the amount of computations. The only thing it solves is processing multiple state changes at once (which can be achieved by transaction indeed in Mobservable). Note that transaction could be applied quite generically, you could by default wrap event handlers in transaction for example.
Running all derivations synchronous makes it a lot easier to comprehend what is happening, is easier to debug and is very optimizable.
My bad, I am sorry. I used those terms wrong.
What I tried to explain is that when somebody is using Mobservable, it controls all the reactive/observable sources, so all of them can be controlled, executed, scheduled, whatever.
In my case, I was trying the same approach with a Meteor package only (ReactiveState), where I store the state of the app. The approach worked well for all the reactive sources of the package itself (internal), but not with the other Meteor reactive sources (external) because I couldn’t control them.
So what I tried didn’t work not for Mobservable itself, but because I wasn’t modifying Tracker, just enhancing it.
The solution to improve Tracker needs to be Tracker 2.0 and not only a Meteor package and after reading your inspiring article about the Mobservable implementation I think that might be the way to go.
What do you think? And you, @faceyspacey?
Ok, I see. I like the transaction pattern but I can see a problem with everything-is-a-package or everythign-is-a-module architectures, where code runs from very different places. So you can’t just do a single transaction() function with the changes, they are spread over different modules. Default wrappers can be a solution but you have to implement them. That’s the good point of async execution, you don’t need to implement anything, it works in any case, although yes, it’s harder to debug, optimise and so on.
Note that it is harmless to not use transaction. Yes, it is a bit inefficient, but as long as your derivations are idempotent as they should be, you wouldn’t be able to tell the difference. So transactions can always be applied as an afterthought optimization, while the sync / async decision itself is quite fundamental. I think easiness / predicatability should prevail here over premature optimization 
Edit: btw, note there is actually an autorunAsync, but that is a mere exception
I do not think Tracker has really any real issues. The issues are defaults which are not clearly communicated to the users. Namely, that in Blaze, default equality used in ReactiveVars more or less means that everything is rerun every time any object changes, because if a value is an object, then equality is always false. This goes against the common recommendation to use objects for data contexts.
The solution is one we use in Blaze Components: do not even depend on the data context, if not needed (the issue in Blaze is that any helper you run automatically registers a dependency on the whole data context). The other is to use computed field to compartmentalize changes. The result is marvelous reduction in computation reruns.
1 Like
In the Meteor Guide we suggest not using each or with, in which case the number of data context invalidations is reduced, but you still get a re-render when the data context of the template inclusion changes.
Probably providing the data context as this inside helpers is the biggest problem with Blaze, one that was unfortunately inherited from Handlebars. But I guess Handlebars was one of the things that made Meteor easy to adopt in the first place. So who knows really.
2 Likes
I’d say if you’re ok with the unnecessary re-runs (even though it often does lead to unexplainable Tracker re-rendering errors), the real main “issue” is re-rendering based on unused fields in your queries. Perhaps this can be seen more of as a “feature request” rather than an “issue.” Either way, it’s the biggest thing that’s wrong with the current system. It’s exactly the problem React solves really well by requiring you only to pass the props a component needs down. Blaze on the other hand easily allows developers to pass every single prop/field of an object down, thereby triggering tons of unnecessary reactivity. And i’m talkin: reactivity all the way down the stack to the server! Now the server is sending you data you don’t even use!!
I personally have thrown in the towel. The “transparent” stuff more and more each day is looking like a “spreadsheet”–i.e. what I as a programmer perceive as a tool/toy for people who can’t really program. “Real” developers need purity, a predictability of inputs and outputs, tests that don’t require lots of fixtures and spys because of impurity.
That said a TRP platform would make a great way for developers to learn programming, and that’s why i always recommended Meteor as the best way to learn programming. I don’t wanna see it go away. Perhaps there is still a path forward for TRP–I’ve discussed much of what’s gotta be done to save it, to grow it, with @luisherranz and @mweststrate: we need time traveling + first class data-flow dependency visualizations to debug and always be aware of what’s going on; and Session + ReactiveDict etc should be relegated to a single state atom (leaving you with only 2 client side data sources: Minimongo Domain state + for example: @luisherranz’s Reactive State package).
1 Like
I don’t think this is actually the case - I know tons of react developers that just pass down a huge object as one prop. So React doesn’t really “solve” this. Besides, what you’re talking about is a problem with Blaze, not Tracker.
React is better at providing this ability to constrain. It’s at least better at promoting this as the way. But even then–you get more bang for your buck in react than you do specifying fields in Meteor: you get pure components and all its benefits (discussed ad nauseamin in a million articles you can read else where). You’re way more likely to put the time into performing this constraint in React–pure components thanks to props is what React is all about, the same can’t be said for Meteor/Tracker/Blaze. I can’t tell you how many apps I’ve seen where the fields option is not provided but hardly any of an object’s fields are actually utilized.
That’s not true. The suggestion is that there should be more intelligence about what fields are actually used without having to explicitly specify them–like Mobservable facilitates. Tracker will re-run your autoruns if you do collection.find() without specifying fields. This perhaps is even more of a problem with plain Tracker (i.e. as opposed to Blaze where the autoruns are created behind the scenes for you) since when you manually create autoruns it’s 100% of the time for the purpose of a side effect. For example if you use Meteor.user() here instead of Meteor.userId():
Tracker.autorun(function() {
if(Meteor.user()) doSideEffect(); //any updates to user fields trigger re-runs
});
then whatever it is that you’re doing better be coded in a way that it doesn’t matter if it’s executed multiple times. Sure you will likely do just a Meteor.userId() check, but this is just an example–I can’t tell you how many times I’ve seen code that just checks for the existence of a model object without specifying fields; I can’t tell you how many times I’ve been guilty of it; most developers don’t even know you have fine grained control of reactivity thanks to the fields option! It’s so easy to get caught with an autorun that keeps re-running because individual unrelated fields are updating, which in turn means you must take extra precautions to make sure doSideEffect() is insulated from any problems that come from running when it shouldn’t.
Do this all the time, thought most did outside of demos.
1 Like
So, we should educate better. I think this is done in the guide.
Also, for Meteor.user, you can use this package to limit which fields you are interested in.
It is interesting that you are complaining about the fact that you have to provide fields to the query, but on the other hand you like props. Think a bout fields being like props. You explicitly define what you are interested in. The difference is only what is the default. For queries, if you do not define fields, then everything is returned. If you do not like that, you can extend find in your app in a way that by default only _id is in fields, and you have to specify extra fields to get more fields. Being explicit is great, no?
I prefer to specify fields where it is reasonable, but otherwise I just wrap computations inside computed field instead of autorun and forget about premature optimizations.
3 Likes
Listen, I was being charitable in this discussion by letting it be ok that tracker runs when it doesn’t need to. The fact is it does cause problems and there is often tracker stack trace exceptions that aren’t caused by the developer! I get these all the time and have for years. It happens whenever your app becomes really interactive. In addition it can too easily block and make the whole interface stop responding. All while a million autoruns re-run themselves unnecessarily calculating nothing new! In addition, debugging tracker stack traces is rediculous–it reflects nothing to do with your actual program. The list goes on, and I’m not even talking about what everyone else is talking about which is it easily allows u to use spaghetti patterns–that isn’t even my gripe.
Tracker runs when it shouldn’t. Mobservable doesn’t. I’m not complaining. There are things that should be done. I personally have other fish to fry (apparently, like MDG) and I’m over this. This was like 2 months ago. I simply shed some light on some real problems with meteor, and if it matters to someone in the Meteor community they will address it. Tracker is not nearly as professional and stable as it should be. It’s been pitched for years about how small its library is, but what should be pitched is how battle-trodden it is, how many heavy duty use cases it is armed to combat for you. MDG should have made ways to visualize your runtime reactive dependencies long ago if we were ever able to take transparent reactive programming seriously. It’s already been done in RxJS–this in an instant would solve all observability issues with Tracker. What I’m talking about is: an in-browser visualization of all your reactive dependencies, the data sources involved, the functions that mutate them, and the chain of dependent autorun functions and blaze helpers. See, at the end of the day, the main problem here is: trust. React’s unidirectional data-flow is something developers can trust, Tracker/Blaze’s ad hoc near-random frames of reactivity isn’t trustworthy. That’s why observability and debuggability are the most important issues here–it’s our only recourse to TRP’s ad hoc nature. But the pre-requisite to debuggability/observability is making sure your autorun functions aren’t re-running more than they have to. That way you have a strict view into what’s happening; that way, for example, what fields triggering reactivity is easily and automatically known–the ones that are used. And here’s the real point regarding the fields debate:
- first i dont even like
propsmyself. i agree with you–they have the same problem asfields: - both force you to change in many places fields/props used when your needs change
- this is why automatically inferring them based on usage is by far the least amount of maintenance/work
- it’s especially important in Meteor, where if you don’t, actual data is pushed over the wire that you don’t even use!
- nobody uses
fields. Documenting it better, sure, is a good step, but the real problem is the culture: A) MDG doesn’t care about the view layer enough, and B) MDG seems to have no real application experience. They only build Meteor, they don’t use it. Can you name one production app they built? I talked to a core MDG developer–who will go unnamed–the other day who didn’t even know you could constrain reactivity by usingfields! So the cultural problem is that MDG isn’t focusing on client side reactivity. Maybe “culture” is the wrong word, but it’s not–there isn’t a concern, deep focus and high regard for client side reactivity within MDG like there is within teams devoted solely to the view layer. So we could add more documentation aboutfields, but that’s really just a symptom of a bigger problem. These guys don’t care about professional grade reactivity like the React team for example, or otherwise usage of thefieldsoption would be as highly promoted aspropsand and strict usage ofstateare thoroughly explained in the React docs! I’m not blaming MDG; I understand how challenging it is to build and maintain large software projects; it’s just a reality; their focus is elsewhere and I don’t recommend depending on them for reactivity related and view related tools anymore.
As far as Tracker, here’s my recommendation: everything possible should be done to insure it doesn’t recalculate when it doesn’t have to. That’s not currently the case. And yes there are always solutions and round about ways to get things done, but the system should be able to figure out as much as possible. Automatically figuring out fields would be a great thing, not sure why you are fighting that one. Yes, we love Meteor and wanna defend it, but we have to get real about its problems–tracker won’t even matter in a few more months unless we enhance it, because everyone will have moved to React; because of that unless your goal is to enhance Tracker rather than leave it the same, this conversation basically doesn’t even matter. Enhancing fields is just the tip of the ice berg. We need a first class transparent reactive programming development environment, as I’ve been discussing with @luisherranz and the creator of Mobservable, @mweststrate. If you have read my article or any of my other posts, you know some other recommendations. Right now tracker is basically a black box. We need tools to shed some light on what’s going on within it. TRP is a fantastic very special and unique productive model for reactivity, regardless of what all the people who have jumped ship to react say. But until its core problems are fixed and its observability/debugability is provided, I do not recommend using tracker or blaze anymore. Its just as unfortunate for u (creator of Blaze Components) as me–I spent an entire year building an entire OOP framework on top of Meteor, heavily based on tracker and blaze! It’s now useless since it’s not the way forward!!
I don’t recommend using blaze or tracker anymore and you shouldn’t either. Not until these issues are addressed. I wish I had the time, but my perception is it’s better to focus on what I perceive to be the future and solutions that have longer shelf lives than something its creators don’t even seem to wanna take seriously. If others like @luisherranz take this on their back to upgrade (and possibly I’ll contribute), we can revisit it. Until then, you won’t hear me in these forums promoting to new developers to use anything but React. And really my stance is this: if you can, avoid building anything new with Meteor until MDG releases their answer to the GraphQL/Relay concept. It’s a great concept, and if done correctly (like Om Next within the ClojureScript community has done) will also replace Redux, plus bring Time Traveling on both the client and server. At that almost inevitable point, Tracker will not even be a package within your application. The only way I see for that not to happen is if we make Tracker a first class priority again and upgrade it like I’ve thoroughly explained elsewhere. But hey @mitar you wanna defend Tracker, as it already is, to the death–well, that will be the death of it. At the end of the day @mitar I think you want these things I’m saying too. How could you not as someone that has invested so heavily in Blaze. It’s unfortunate, programming takes so much damn fucking time. But that’s where we are, and we all gotta make the pragmatic decisions we deem best based on realities. Tracker is not looking too good my friend. …@luisherranz and @mweststrate really should be getting lots of help by the Meteor community to make the various upgrades discussed. @mitar do you wanna be a part of that? Do you wanna upgrade Tracker with us?? @sashko what about you, can we get some resources to devote to Tracker? At the end of the day, the visualizations would be great and useable not just with TRP, but with Observable-based reactivity, and React+Redux. But they are especially important for us since our reactivity model is so unpredictable.
3 Likes