Also, are there any cases of Non-Profits maintaining open source software? It’s not just about paying for the tools, it’s managing the direction of the tools, it’s formal communication, it’s cooperation.
1 Like
That’s my precise goal in discussing it so much in the forum as of late. I’m about to release #2 on your list there. It’s called Meteor.Component. It’s not very big–so sometimes the best way to converse is by throwing an actual concrete example out there, i.e. something you can immediately use rather than just theoreticals you can just talk about.
…Go here:
and then go here:
I think MDG has done themselves a big disservice by keeping all their “packages” under the main meteor/meteor roof. What simply needs to happen–well, in implementation its not so simple at this point, but its easy to describe–is: everything needs to be unbundled by default, so anyone from the entire NPM community can work on it any time they want. Currently nobody outside of MDG does much for their core packages. There’s only 16 developers with more than 50 commits:
community developer and Blaze Components author we all know, @mitar comes in at 48 commits. Right near him is founder @debergalis with 113. That’s compared to @glasser at the top spot with 3,386 commits and @dgreensp with @dgreensp in second with 1,880 commits.
In addition to being “unbundled” more, they need to be promoted. Their tools/packages need a life of their own outside of Meteor.
I’m suggesting the first step is their own repos, rather than live in the main meteor repo. It’s pretty obvious stuff. I wonder if it’s just “technical debt” that they never got around to undoing. There’s likely some technical reason having to do with automated builds and whatnot, but I doubt it’s an unsurmountable challenge. My opinion is that Meteor hasn’t been concerned with MARKETING anywhere near as much as they should be. In the developer world, marketing isn’t about big billboards on the side of the highway–it’s about nuanced things like repos that are easily accessible, i.e. easy to contribute to. Meteor is anything but easy to contribute to. Cuz its tools are so coupled and bundled!
They’ve taken great strides at the technical aspects of unbundling their packages, hence all the packages we have pre-added in our packages file which never used to be there. We just need to complete that unbundling.
The most important thing toward that aim is being able to live on “bare metal” NPM. To achieve that goal, we have to unbundle everything to begin with. In my Meteor 1.3 wishlist I discuss generating a main.js file which actually has the code which loads and instantiates all the stuff Meteor does for us behind the scenes on startup. This seemingly small thing is a big deal in terms of showing developers how all the pieces of Meteor come together. Combine that with easy access to the individual repos, and you now have a recipe for more contribution.
…So to answer your question, basically what I’m saying is that if you converse in public (which MDG doesn’t really do, which I do), and if you develop in unbundled way within githbub repos (like MDG doesn’t do and most us developers do), then we already have the “democratic” checks and balances you’re asking for. After all anyone can contribute to a github repository. And if your pull request isn’t accepted, but it was something the community wanted, everyone’s gonna hear about it in the ISSUES or associated forum. The most important keyword I think of when it comes to the future is “decentralization.” That’s what bitcoin is about. That’s what NPM is about. That’s where everything is headed. When you take on big chunks like Meteor has, decentralization is harmed. But when you take on just Tracker 2.0, well, you’re basically voting by doing. Others can vote “with their feet” by using it, or make bigger “votes” by discussing or contributing code.
In short, the “democracy” question is barely a worry when it comes to one guy. We just have this issue because MDG is so non-decentralized. But you’re right, I need to make sure I consult with them if any of this stands a chance of gaining adoption. As far as the rest of the community, that’s exactly why I’m writing all this. A lot of people seem to like it, and a few people have a hard time taking off their fresh functional blinders. The reality is I’m less worried about them since they already are getting their hands dirty in preparation to move forward. If those same people–who are generally outspoken early adopters and experts–can see the merit in things like Tracker 2.0 it will go a long way in terms of executing the practical strategies I’m proposing to improve meteor.
2 Likes
I really like the idea of making tracker and minimongo “reduxable”. Mutable state is really something we got shot by a few times.
So that would not only help react but all v-layer libraries. Though I believe it could be hard to fully erase the magic happening at the edge of tracker<->minimongo
PS: Love the new forum post-style!
1 Like
@faceyspacey here is an in-depth explanation on how Mobservable finds the minimum amount of computed values that needs to re-evaluate, without ever introducing glitches or staleness: https://medium.com/@mweststrate/becoming-fully-reactive-an-in-depth-explanation-of-mobservable-55995262a254#.xpxqzvkur
1 Like
Excellent! I’ll check it out.
Left u a response here:
And on ur medium article. I think basically Mobservable should replace Tracker. @sashko MDG should really take a look at this and let us know the pitfalls they see. There clearly will be some walls we run up against since we have a huge number of tools built on top of tracker. For its behavior to change in any significant way may cause things to break. But I think if we could prove Mobservable and the fixes it provides is a major enhancement we could get buy-in to upgrade tools that depend on it. Eg session, collection cursors, blaze helpers, etc.
1 Like
I PM’ed faceyspacey some upgrades on tracker that I have been using at my company and he said that I should post them here. Here is the original message:
I read your Medium post about Tracker 2.0 and I thought that it was very insightful. I’ve got a couple helper functions monkey patched onto Tracker that I think you might find interesting. Here is a Gist: https://gist.github.com/veered/30ae6f79ec48506af80f
I think these helpers are a step in the right direction. And I think the code at the bottom (which I haven’t spent any time on improving after I had the original idea), could be used to build out way better debugging tools for Tracker.
If he would accept this challenge and would say how much he needs to start working, I would donate (in BTC) him for sure 
@faceyspacey, @mweststrate this is my opinion and experience trying to make Tracker sync via ReactiveState modifiers added to the A neat diagram comparing Meteor, Flux, and Relay post.
I’d love your comments about this stuff, regarding a possible Tracker 2.0 implementation.
I have not read the @mweststrate article about the minimum amount of computed values yet but I will do as soon as I can and I will post my thoughts here as well.
This is the post:
Mobservable is a great package. It’s transparent reactive programming (yeah, Tracker) for the React world and it shows the advantages of TRP in terms of code which is simple to reason about and less prone to edge case bugs and even performance gains when used right (check this https://www.mendix.com/tech-blog/making-react-reactive-pursuit-high-performing-easily-maintainable-react-apps/ out).
The main differences I see between Mobservable and Tracker are two:
-
Mobservable has all state inside and Tracker has state outside.
-
Mobservable is sync and Tracker is async.
Both are related. Sync stuff is easier to debug than async stuff.
I tried to make my ReactiveState package sync because when you own all state, you can execute stuff synchronously:
State.modify('tasks.areEmpty', (state = true) => {
if (State.get('tasks.items').length === 0)
return true;
else
return false;
});
State.modify('tasks.items', (state = []) => {
return Tasks.find().fetch();
});
When you are evaluating tasks.areEmpty you can stop at the State.get('tasks.items') and execute its modify first, make sure it is up-to-date and then continue with tasks.areEmpty. Functions are executed synchronously and only once. Perfect!
The problem with that is, once again: Meteor reactivity comes from many places, not only a single tree. For example, add another reactive source to the mix: Meteor.userId().
State.modify('user.hasPendingTasks', (state = false) => {
if (Meteor.userId() && State.get('user.tasks').length === 0)
return true;
else
return false;
});
State.modify('user.tasks', (state = []) => {
let userId = Meteor.userId();
return Tasks.find({ userId }).fetch();
});
If you want to make this work synchronously, you need to know if Meteor.userId() has already changed or is going to change before executing any of the State modifiers.
The sad truth is: you cannot know. Tracker reactivity has no priorities and is spread in many places along Meteor. So the solution is what Meteor people already did: make Tracker async and wait for every reactive source to invalidate its computations. Then start executing computations.
For that reason, I don’t think Mobservable is useful or has any advantage over Tracker in Meteor if you don’t want to throw a lot of stuff which is already working with Tracker.
For Tracker, this means sometimes functions are going to be executed twice or even more times until they reach an stable state. That’s the reason MeteorFlux actions have three phases:
- Register callbacks (sync non-reactive).
- State.modifies or Tracker.autoruns (async reactive, it keeps executing functions until everything is stable).
- AfterAction callbacks (sync after all reactivity has finished and is stable, non-reactive).
Explained like this, this looks like really complex stuff, but in the end coding with TRP is simpler and less prone to bugs. Most of the time you don’t have to know about how it works internally and that’s what makes Tracker and Meteor so simple to start with.
2 Likes
Ok, I have read your article @mweststrate 
I really like your implementation. It’s similar to what I tried to achieve with my ReactiveState package. And the problem I wrote in the last post could be solved by your transaction pattern. Actually what Tracker does with its async approach is to make the current call stack a transaction and finishes it when everything has been executed and the CPU is idle again.
I have a question: why do think remaining sync is so important? Debugging, testing?
The other problem I had with my ReactiveState implementation was that I could only control what was inside ReactiveState, so reactive stuff outside the system broke the execution chain. But if instead of using a package on top of Tracker we re-write it (Tracker 2.0?) or find a way to use Mobservable instead, we could control the whole dependency tree and optimise the number of computations.
I think Tracker 2.0 could use two important performance improvements from Mobservable:
- Make a proper execution chain derived from dependencies so computations are executed in a optimal order, not randomly like now.
- Check for value changes in that chain so it can stop unnecessary executions.
I tried the first one and I am already doing the second one with ReactiveState:
// This retriggers when ANYTHING inside the post object change, not only title.
Tracker.autorun(() => {
let postTitle = Posts.findOne().title;
});
// But using ReactiveState to store the state...
State.modify('post', (state = {}) => {
return Posts.findOne();
});
// ...it retriggers only when the post title has changed.
Tracker.autorun(() => {
let postTitle = State.get('post.title');
});
Food for thought, really. Thanks for sharing all this amazing stuff!
4 Likes
Maybe community drive Tracker 2.0 can be implemented on top of mobservable.
Backwards compatibility relay in the fact that rerun behavior of actual implementation is a little “undetermined”.
Mobservable project go back us to good times of reactive data. I don´t really like the imperative aproach of ‘Flux actions’ and explicit ‘tree of truth’.
1 Like
@luisherranz Could you elaborate a bit on what you mean by internal vs. external state? Mobservable doesn’t rely on stores, single state tree or other dedicated places to stores state like in flux architectures. Anything that is decorated using observable can be used a source of reactive data, no matter where the data originates from. So I don’t think your state / userid example above wouldn’t be an issue for Mobservable.
Sync behavior is quite fundamental to Mobservable (it is quite similar to Javelin in this regard); asynchronicity shouldn’t be used as means for scheduling; the only reliable source for correct execution order is the dependency graph. Once you have that, running stuff asynchronous only complicates stuff unecessarily; one can observe glitches, or introduce really nasty bugs like:
// pseudo code:
x = 1;
y <= x * 2 // computed value
x = 3;
if (y > 4) {
// will this code run? impossible to answer in an asynchronous world...
}
Note that running asynchronous doesn’t prevent glitches from happening. You still have to make sure that everything in the right order. So it doesn’t solve the fundamental problem of running the computations in the right order and minimizing the amount of computations. The only thing it solves is processing multiple state changes at once (which can be achieved by transaction indeed in Mobservable). Note that transaction could be applied quite generically, you could by default wrap event handlers in transaction for example.
Running all derivations synchronous makes it a lot easier to comprehend what is happening, is easier to debug and is very optimizable.
My bad, I am sorry. I used those terms wrong.
What I tried to explain is that when somebody is using Mobservable, it controls all the reactive/observable sources, so all of them can be controlled, executed, scheduled, whatever.
In my case, I was trying the same approach with a Meteor package only (ReactiveState), where I store the state of the app. The approach worked well for all the reactive sources of the package itself (internal), but not with the other Meteor reactive sources (external) because I couldn’t control them.
So what I tried didn’t work not for Mobservable itself, but because I wasn’t modifying Tracker, just enhancing it.
The solution to improve Tracker needs to be Tracker 2.0 and not only a Meteor package and after reading your inspiring article about the Mobservable implementation I think that might be the way to go.
What do you think? And you, @faceyspacey?
Ok, I see. I like the transaction pattern but I can see a problem with everything-is-a-package or everythign-is-a-module architectures, where code runs from very different places. So you can’t just do a single transaction() function with the changes, they are spread over different modules. Default wrappers can be a solution but you have to implement them. That’s the good point of async execution, you don’t need to implement anything, it works in any case, although yes, it’s harder to debug, optimise and so on.
Note that it is harmless to not use transaction. Yes, it is a bit inefficient, but as long as your derivations are idempotent as they should be, you wouldn’t be able to tell the difference. So transactions can always be applied as an afterthought optimization, while the sync / async decision itself is quite fundamental. I think easiness / predicatability should prevail here over premature optimization 
Edit: btw, note there is actually an autorunAsync, but that is a mere exception
I do not think Tracker has really any real issues. The issues are defaults which are not clearly communicated to the users. Namely, that in Blaze, default equality used in ReactiveVars more or less means that everything is rerun every time any object changes, because if a value is an object, then equality is always false. This goes against the common recommendation to use objects for data contexts.
The solution is one we use in Blaze Components: do not even depend on the data context, if not needed (the issue in Blaze is that any helper you run automatically registers a dependency on the whole data context). The other is to use computed field to compartmentalize changes. The result is marvelous reduction in computation reruns.
1 Like
In the Meteor Guide we suggest not using each or with, in which case the number of data context invalidations is reduced, but you still get a re-render when the data context of the template inclusion changes.
Probably providing the data context as this inside helpers is the biggest problem with Blaze, one that was unfortunately inherited from Handlebars. But I guess Handlebars was one of the things that made Meteor easy to adopt in the first place. So who knows really.
2 Likes
I’d say if you’re ok with the unnecessary re-runs (even though it often does lead to unexplainable Tracker re-rendering errors), the real main “issue” is re-rendering based on unused fields in your queries. Perhaps this can be seen more of as a “feature request” rather than an “issue.” Either way, it’s the biggest thing that’s wrong with the current system. It’s exactly the problem React solves really well by requiring you only to pass the props a component needs down. Blaze on the other hand easily allows developers to pass every single prop/field of an object down, thereby triggering tons of unnecessary reactivity. And i’m talkin: reactivity all the way down the stack to the server! Now the server is sending you data you don’t even use!!
I personally have thrown in the towel. The “transparent” stuff more and more each day is looking like a “spreadsheet”–i.e. what I as a programmer perceive as a tool/toy for people who can’t really program. “Real” developers need purity, a predictability of inputs and outputs, tests that don’t require lots of fixtures and spys because of impurity.
That said a TRP platform would make a great way for developers to learn programming, and that’s why i always recommended Meteor as the best way to learn programming. I don’t wanna see it go away. Perhaps there is still a path forward for TRP–I’ve discussed much of what’s gotta be done to save it, to grow it, with @luisherranz and @mweststrate: we need time traveling + first class data-flow dependency visualizations to debug and always be aware of what’s going on; and Session + ReactiveDict etc should be relegated to a single state atom (leaving you with only 2 client side data sources: Minimongo Domain state + for example: @luisherranz’s Reactive State package).
1 Like
I don’t think this is actually the case - I know tons of react developers that just pass down a huge object as one prop. So React doesn’t really “solve” this. Besides, what you’re talking about is a problem with Blaze, not Tracker.
React is better at providing this ability to constrain. It’s at least better at promoting this as the way. But even then–you get more bang for your buck in react than you do specifying fields in Meteor: you get pure components and all its benefits (discussed ad nauseamin in a million articles you can read else where). You’re way more likely to put the time into performing this constraint in React–pure components thanks to props is what React is all about, the same can’t be said for Meteor/Tracker/Blaze. I can’t tell you how many apps I’ve seen where the fields option is not provided but hardly any of an object’s fields are actually utilized.
That’s not true. The suggestion is that there should be more intelligence about what fields are actually used without having to explicitly specify them–like Mobservable facilitates. Tracker will re-run your autoruns if you do collection.find() without specifying fields. This perhaps is even more of a problem with plain Tracker (i.e. as opposed to Blaze where the autoruns are created behind the scenes for you) since when you manually create autoruns it’s 100% of the time for the purpose of a side effect. For example if you use Meteor.user() here instead of Meteor.userId():
Tracker.autorun(function() {
if(Meteor.user()) doSideEffect(); //any updates to user fields trigger re-runs
});
then whatever it is that you’re doing better be coded in a way that it doesn’t matter if it’s executed multiple times. Sure you will likely do just a Meteor.userId() check, but this is just an example–I can’t tell you how many times I’ve seen code that just checks for the existence of a model object without specifying fields; I can’t tell you how many times I’ve been guilty of it; most developers don’t even know you have fine grained control of reactivity thanks to the fields option! It’s so easy to get caught with an autorun that keeps re-running because individual unrelated fields are updating, which in turn means you must take extra precautions to make sure doSideEffect() is insulated from any problems that come from running when it shouldn’t.