I do not think Tracker has really any real issues. The issues are defaults which are not clearly communicated to the users. Namely, that in Blaze, default equality used in ReactiveVars more or less means that everything is rerun every time any object changes, because if a value is an object, then equality is always false. This goes against the common recommendation to use objects for data contexts.
The solution is one we use in Blaze Components: do not even depend on the data context, if not needed (the issue in Blaze is that any helper you run automatically registers a dependency on the whole data context). The other is to use computed field to compartmentalize changes. The result is marvelous reduction in computation reruns.
In the Meteor Guide we suggest not using each or with, in which case the number of data context invalidations is reduced, but you still get a re-render when the data context of the template inclusion changes.
Probably providing the data context as this inside helpers is the biggest problem with Blaze, one that was unfortunately inherited from Handlebars. But I guess Handlebars was one of the things that made Meteor easy to adopt in the first place. So who knows really.
Iād say if youāre ok with the unnecessary re-runs (even though it often does lead to unexplainable Tracker re-rendering errors), the real main āissueā is re-rendering based on unused fields in your queries. Perhaps this can be seen more of as a āfeature requestā rather than an āissue.ā Either way, itās the biggest thing thatās wrong with the current system. Itās exactly the problem React solves really well by requiring you only to pass the props a component needs down. Blaze on the other hand easily allows developers to pass every single prop/field of an object down, thereby triggering tons of unnecessary reactivity. And iām talkin: reactivity all the way down the stack to the server! Now the server is sending you data you donāt even use!!
I personally have thrown in the towel. The ātransparentā stuff more and more each day is looking like a āspreadsheetāāi.e. what I as a programmer perceive as a tool/toy for people who canāt really program. āRealā developers need purity, a predictability of inputs and outputs, tests that donāt require lots of fixtures and spys because of impurity.
That said a TRP platform would make a great way for developers to learn programming, and thatās why i always recommended Meteor as the best way to learn programming. I donāt wanna see it go away. Perhaps there is still a path forward for TRPāIāve discussed much of whatās gotta be done to save it, to grow it, with @luisherranz and @mweststrate: we need time traveling + first class data-flow dependency visualizations to debug and always be aware of whatās going on; and Session + ReactiveDict etc should be relegated to a single state atom (leaving you with only 2 client side data sources: Minimongo Domain state + for example: @luisherranzās Reactive State package).
I donāt think this is actually the case - I know tons of react developers that just pass down a huge object as one prop. So React doesnāt really āsolveā this. Besides, what youāre talking about is a problem with Blaze, not Tracker.
React is better at providing this ability to constrain. Itās at least better at promoting this as the way. But even thenāyou get more bang for your buck in react than you do specifying fields in Meteor: you get pure components and all its benefits (discussed ad nauseamin in a million articles you can read else where). Youāre way more likely to put the time into performing this constraint in Reactāpure components thanks to props is what React is all about, the same canāt be said for Meteor/Tracker/Blaze. I canāt tell you how many apps Iāve seen where the fields option is not provided but hardly any of an objectās fields are actually utilized.
Thatās not true. The suggestion is that there should be more intelligence about what fields are actually used without having to explicitly specify themālike Mobservable facilitates. Tracker will re-run your autoruns if you do collection.find() without specifying fields. This perhaps is even more of a problem with plain Tracker (i.e. as opposed to Blaze where the autoruns are created behind the scenes for you) since when you manually create autoruns itās 100% of the time for the purpose of a side effect. For example if you use Meteor.user() here instead of Meteor.userId():
Tracker.autorun(function() {
if(Meteor.user()) doSideEffect(); //any updates to user fields trigger re-runs
});
then whatever it is that youāre doing better be coded in a way that it doesnāt matter if itās executed multiple times. Sure you will likely do just a Meteor.userId() check, but this is just an exampleāI canāt tell you how many times Iāve seen code that just checks for the existence of a model object without specifying fields; I canāt tell you how many times Iāve been guilty of it; most developers donāt even know you have fine grained control of reactivity thanks to the fields option! Itās so easy to get caught with an autorun that keeps re-running because individual unrelated fields are updating, which in turn means you must take extra precautions to make sure doSideEffect() is insulated from any problems that come from running when it shouldnāt.
So, we should educate better. I think this is done in the guide.
Also, for Meteor.user, you can use this package to limit which fields you are interested in.
It is interesting that you are complaining about the fact that you have to provide fields to the query, but on the other hand you like props. Think a bout fields being like props. You explicitly define what you are interested in. The difference is only what is the default. For queries, if you do not define fields, then everything is returned. If you do not like that, you can extend find in your app in a way that by default only _id is in fields, and you have to specify extra fields to get more fields. Being explicit is great, no?
I prefer to specify fields where it is reasonable, but otherwise I just wrap computations inside computed field instead of autorun and forget about premature optimizations.
Listen, I was being charitable in this discussion by letting it be ok that tracker runs when it doesnāt need to. The fact is it does cause problems and there is often tracker stack trace exceptions that arenāt caused by the developer! I get these all the time and have for years. It happens whenever your app becomes really interactive. In addition it can too easily block and make the whole interface stop responding. All while a million autoruns re-run themselves unnecessarily calculating nothing new! In addition, debugging tracker stack traces is rediculousāit reflects nothing to do with your actual program. The list goes on, and Iām not even talking about what everyone else is talking about which is it easily allows u to use spaghetti patternsāthat isnāt even my gripe.
Tracker runs when it shouldnāt. Mobservable doesnāt. Iām not complaining. There are things that should be done. I personally have other fish to fry (apparently, like MDG) and Iām over this. This was like 2 months ago. I simply shed some light on some real problems with meteor, and if it matters to someone in the Meteor community they will address it. Tracker is not nearly as professional and stable as it should be. Itās been pitched for years about how small its library is, but what should be pitched is how battle-trodden it is, how many heavy duty use cases it is armed to combat for you. MDG should have made ways to visualize your runtime reactive dependencies long ago if we were ever able to take transparent reactive programming seriously. Itās already been done in RxJSāthis in an instant would solve all observability issues with Tracker. What Iām talking about is: an in-browser visualization of all your reactive dependencies, the data sources involved, the functions that mutate them, and the chain of dependent autorun functions and blaze helpers. See, at the end of the day, the main problem here is: trust. Reactās unidirectional data-flow is something developers can trust, Tracker/Blazeās ad hoc near-random frames of reactivity isnāt trustworthy. Thatās why observability and debuggability are the most important issues hereāitās our only recourse to TRPās ad hoc nature. But the pre-requisite to debuggability/observability is making sure your autorun functions arenāt re-running more than they have to. That way you have a strict view into whatās happening; that way, for example, what fields triggering reactivity is easily and automatically knownāthe ones that are used. And hereās the real point regarding the fields debate:
first i dont even like props myself. i agree with youāthey have the same problem as fields :
both force you to change in many places fields/props used when your needs change
this is why automatically inferring them based on usage is by far the least amount of maintenance/work
itās especially important in Meteor, where if you donāt, actual data is pushed over the wire that you donāt even use!
nobody uses fields. Documenting it better, sure, is a good step, but the real problem is the culture: A) MDG doesnāt care about the view layer enough, and B) MDG seems to have no real application experience. They only build Meteor, they donāt use it. Can you name one production app they built? I talked to a core MDG developerāwho will go unnamedāthe other day who didnāt even know you could constrain reactivity by using fields! So the cultural problem is that MDG isnāt focusing on client side reactivity. Maybe ācultureā is the wrong word, but itās notāthere isnāt a concern, deep focus and high regard for client side reactivity within MDG like there is within teams devoted solely to the view layer. So we could add more documentation about fields, but thatās really just a symptom of a bigger problem. These guys donāt care about professional grade reactivity like the React team for example, or otherwise usage of the fields option would be as highly promoted as props and and strict usage of state are thoroughly explained in the React docs! Iām not blaming MDG; I understand how challenging it is to build and maintain large software projects; itās just a reality; their focus is elsewhere and I donāt recommend depending on them for reactivity related and view related tools anymore.
As far as Tracker, hereās my recommendation: everything possible should be done to insure it doesnāt recalculate when it doesnāt have to. Thatās not currently the case. And yes there are always solutions and round about ways to get things done, but the system should be able to figure out as much as possible. Automatically figuring out fields would be a great thing, not sure why you are fighting that one. Yes, we love Meteor and wanna defend it, but we have to get real about its problemsātracker wonāt even matter in a few more months unless we enhance it, because everyone will have moved to React; because of that unless your goal is to enhance Tracker rather than leave it the same, this conversation basically doesnāt even matter. Enhancing fields is just the tip of the ice berg. We need a first class transparent reactive programming development environment, as Iāve been discussing with @luisherranz and the creator of Mobservable, @mweststrate. If you have read my article or any of my other posts, you know some other recommendations. Right now tracker is basically a black box. We need tools to shed some light on whatās going on within it. TRP is a fantastic very special and unique productive model for reactivity, regardless of what all the people who have jumped ship to react say. But until its core problems are fixed and its observability/debugability is provided, I do not recommend using tracker or blaze anymore. Its just as unfortunate for u (creator of Blaze Components) as meāI spent an entire year building an entire OOP framework on top of Meteor, heavily based on tracker and blaze! Itās now useless since itās not the way forward!!
I donāt recommend using blaze or tracker anymore and you shouldnāt either. Not until these issues are addressed. I wish I had the time, but my perception is itās better to focus on what I perceive to be the future and solutions that have longer shelf lives than something its creators donāt even seem to wanna take seriously. If others like @luisherranz take this on their back to upgrade (and possibly Iāll contribute), we can revisit it. Until then, you wonāt hear me in these forums promoting to new developers to use anything but React. And really my stance is this: if you can, avoid building anything new with Meteor until MDG releases their answer to the GraphQL/Relay concept. Itās a great concept, and if done correctly (like Om Next within the ClojureScript community has done) will also replace Redux, plus bring Time Traveling on both the client and server. At that almost inevitable point, Tracker will not even be a package within your application. The only way I see for that not to happen is if we make Tracker a first class priority again and upgrade it like Iāve thoroughly explained elsewhere. But hey @mitar you wanna defend Tracker, as it already is, to the deathāwell, that will be the death of it. At the end of the day @mitar I think you want these things Iām saying too. How could you not as someone that has invested so heavily in Blaze. Itās unfortunate, programming takes so much damn fucking time. But thatās where we are, and we all gotta make the pragmatic decisions we deem best based on realities. Tracker is not looking too good my friend. ā¦@luisherranz and @mweststrate really should be getting lots of help by the Meteor community to make the various upgrades discussed. @mitar do you wanna be a part of that? Do you wanna upgrade Tracker with us?? @sashko what about you, can we get some resources to devote to Tracker? At the end of the day, the visualizations would be great and useable not just with TRP, but with Observable-based reactivity, and React+Redux. But they are especially important for us since our reactivity model is so unpredictable.
i mean why is programming so time consuming? the devilās in the details, and details take time to explain. Itās the nature of programming. itās the nature of our industry. if you care, youāll take the time to read it, analyze it, address every detail. if you donāt care about the topic at hand, simply donāt read it.
That said, Iāve been trying to limit post lengthāI agree, this is somewhat of a relapse. In general, donāt expect to hear much more from me; Iām over a lot of this, and just trying to leave a sense of gravity behind certain issues that I think they are due, especially when community leaders like @mitar downplay their importance. I think he hasnāt had the time to focus on these things like I have, and my hope is when heās given time for some of this to sink in, heāll come aroundāor likely, end up in React land like everyone else (myself included) because itās just easier and less time consuming than fixing all the aforementioned problems.
People jumped ship to React, but didnāt even really understand the problems with Blaze + Tracker. Transparent Reactive Programming is an amazing thing, and could even stay as a pillar of Meteor, but it absolutely will notāthis is my prediction at leastāunless people become aware of the concepts shared above and elsewhere ;).
Basically, itās the squeaky wheel argument. But since nobody cares, and since React has such a great ecosystem emerging around it, fuck it. But just know there was a lot of blind āfollow the leaderā behavior exhibited within the Meteor community in the grand switch to Reactāpeople barely understood what was wrong with Blaze/React. And people still fighting React, barely have a clue and need any help they can get to understand current seismic changes. The truth is, it worked beautifully and for all but the most professional of apps was a blessing, and I have essentially been nit-picking which is why @mitar disagreed.
But that said, as developers, our job is to nit-pick! If we can squeeze a small percentage of performance gain from another solution, we should. Eventually the trade-offs in all such decisions made add up. Surely, if care isnāt taken to make many microscopic decisions, your house of cards (aka your āapplicationā) falls. Iād say the Tracker/Blaze stuff is a slightly larger problem worth truly understanding, rather than sweeping under the rug and pretending like the core of Meteorās reactivity that has kept us hooked to Meteor for so many years is a step child we can kick out of the house without taking the time to truly understand.
At the end of the day, if youāre paying close attention, you will realize the points Iām making arenāt: choose React, or choose Blaze. The real points Iām making are shedding light on the details of everything going on, so you can make decisions for yourself. And very few are taking the time to do that, so thatās why my posts are so long. And Iām not guilting anyone about not doing all this research and sharing the results (many people do). Itās totally understandable to want to focus on building applications instead. Thatās the whole pointāthis information serves the purpose to help those people. I happen to be in between applications, and specifically as a result of all these changes. The application I was building was my OOP Meteor framework. I determined I should not pursue it anymoreāat least for the time being. Iāve done lots of long hard soul searching (and researching) to determine this. The results however, i.e. the fruits of my labor, are the insight Iām providing for free. I look at it as my role to share this info. In addition, sharing it has helped me collate and organize it for myself. So thatās just what it is. Thereās lots of hard-earned insights in a lot of the information Iāve been sharing lately. Iāve both analyzed whatās wrong with Meteor and all the emerging technologies in depth (Elm, ClojureScript, Haskell, Cycle and of course everything having to do with React). If you donāt wanna take the time to read them, thatās understandable. Letās get back to workā¦
thatās what this thread was based on. In short, thereās a lot of hard work to do, but there are some low hanging fruit which definitely should be addressed.
Lol, I have developed a cloud data-flow programming language on top of Meteor, Tracker, and observeChanges (sadly not open source). And server-side autorun. I have been deep into Tracker.
What I learned is that people do not understand that what Meteor provides are still just very basic low level primitives. You should not be using them directly. They were meant to build abstractions on top. But people use them directly and then they get burned. You should wrap collections with your models, rendering with your views, and publish functions and methods with things which provides extra features. Sadly, people do not see what are those needed higher-level things. Or, they see, implement, but packages providing them are not discovered by other people.
Anyway, I think that Meteor provides really nice building blocks and people should build stuff on top of that. Tracker is also an example of that. If you need some magic which detects which fields you are accessing in templates automatically, do that. If you want to limit reactivity, do that.
If you are saying that some technologies are too low-level, maybe. Would need more features? Maybe. But also it is nice that they are clean and simple and you can reason about them easy.
I agree that the criticism is posed too informally.
I have noticed a lot of technical writing ignores the idea of an abstract, thesis, or opening paragraph in the well established essay style. The use of āTL;DRā indicates less crafty writing in that the well established styles address this. The first line in a long document says most of what is being discussed. Thesis. The first paragraph expounds on it and even posits the conclusion which the rest of the body supports.
Paragraphs break up defined points. The last line of each paragraph dovetails the next point.
Essays are not necessary of course. This is a discussion forum. Essays however do help consume large complicated points.
Exactly! Thatās exactly why itās so hard for you, as it is me, to grapple with the fact that Tracker may not be the way forward. If youāre both as deep into Tracker as we all know you are, and as deep into the pushes for purity within the React community coming from Haskell, Elm, and ClojureScript, then you know Tracker has some serious weaknesses in comparison. Itās far from pure. That means testing is a lot more complex, and perhaps more importantly it means your code isnāt easily composeable and decomposeableāyou canāt nearly as easily extract a portion of your code and make another library out of it; most OOP code is application specific because itās not easy to extract, because you have to account for all the side effects and mutations, and make sure all such related code is properly extracted. One of the biggest benefits of functional program is the ability to extract and libraritize any pure code! Thereās many more reasons, all stemming from purity. TRP will never be able to be pure. Purity, limiting side-effects, is a big deal.
Iām saying something like this. Iād say Tracker is āsimpleā but āreasoningā about it isnāt as easy as it should be. In addition it has performance problems.
Overall, I perceive what you are saying as: āMeteor is a platform, not a framework, get over it!ā Just as you, I have a million abstractions Iāve built. Iām all about building and using such abstractions. I try out every cool Meteor package on atmosphere. Iāve tried and love many of your packages. I have at least one or 2 of yours permanently embedded in one of my production apps. Itās great that people like you and @arunoda (and ultimately myself) take the route of just solve it ourselves. I however donāt think itās healthy for that to be the defacto solution to certain core problems. Iām proposing that a certain core behavior be as Iāve described, and ultimately be officially sanctioned by MDG. I think that would best suit the Meteor community.
Letās take another example: people often talk about how Router should be in the core. I happen to agree, but for different reasons. Iāve been very happy to use Iron Router and Flow Router etc as libraries. I have had no complaints there, and I perceive most peopleās complaints there as baseless with no reasoning to back it. My guess has been that there can be a psychological sense of trust or lack of it for those that donāt understand the NPM decentralized way. The lack of trust by these developers has been the main reason they want it in the Core, even though @arunodaās Flow Router etc works so well. That said, thereās a great opportunity here: code splitting. Since likely we arenāt gonna incorporate Webpack itself (because of other challenges), MDG might have to build code splittingāthatās what @benjamn has hinted at a few times. Code Splitting is a ācross cutting concernā that concerns both routing and building. If code splitting was built in as a feature to a router, it would make for a simple streamlined interface for application developers. Therefore, since building is and will continue to be a core Meteor feature, if we were to add code splitting, only then would it make a lot of sense to bring a Router into the core.
What does this have to do with Tracker and its core problems? Some things make sense to be addressed in the core, and others as 3rd party community libraries. I get it, @mitar, youāve done great work in library form to solve a lot of these issues. Like any additional required libraries, it can be noise; the overall work/code produce could also be a lot less. In addition, the net API surface area is a lot larger and therefore more complex. A simpler interface can be provided to the application developer. The solution is to go to the root. That way you or me or anyone doesnāt have to make these libraries. That way there isnāt a million options. That way the net sum of lines of code written is less. The canonical example is The Great Pyramid vs The Arch: it took tons and tons of time and blocks to build it, but there is virtually no available space inside; the arch on the other hand changed that, and through one simple but innovative concept better efficiency can be achieved; you can actually make use of all the space inside the building. So Iād say Tracker is definitely an example of where going to the root would produce wins for allā¦
ā¦Well, all except for basically people like you and I who have built libraries to patch holes. So, yea, I understand your pain man. Quick fix patches we built wonāt continue to garner usage. In terms of Blaze, our Blaze libraries will fade away as Reactāa solution that better went to the rootātakes over.
One concept that rings true in both what we are saying is the idea of āprimitives.ā You mention āMeteor provides really nice building blocks and people should build stuff on top of that.ā Iāve felt the same way for years. What you are talking about is quality āprimitives.ā Without starting a whole other discussion, it seems to me that finding the best primitives is what all/many groundbreaking technologies do so well; itās where they innovate. The reason primitives are so importantāaka the ārootāāis because it exponentially limits the amount of code birthed from those primitives. Itās not like making a mistake in the design of code that lives on the outer edges. Visualize the outer edges as like the planet Pluto and the root primitives, the Sun. So when you get your core primitives correct, it means a lot less work for application developers. Basically this is the case for Elm. This is what Elm has shown us all so well. And of course, this isnāt the first time. Itās the history of software. My point is just that there is a lot of work going into the concept of āprimitivesā right now. A lot of thought is being put into it; being put into rethinking what the best primitives are, and exploring new possibilities of what they can be. The best current example is Signals and Observables, aka ātime varying variables.ā And what youāre presenting right now I perceive as being āmeh, itās good enough.ā So was jQuery.
Itās hard to let go, man. I understand. Either solve Tracker or be willing to put it to rest. over n outā¦
Okay bro I read your post on medium and some of your long comments here and I felt your deception about Tracker and the time you spent building your app which is now corrupted because of Tracker!
But as I saw in your article you got the exact issues in Tracker:
Developers donāt use the fields option to their find() calls.
so just appending the options would work as you said
developers combine multiple, but unrelated, data sources in a single autorun
autoruns re-run when changed dependencies have no effect on the main output
autoruns re-run when data dependent on doesnāt even change!!!
Why do you want to store the state?
why not just use state based hash ?
Simple example
State 1: hashcode1 Initial State
State 2: some Inserts,updateā¦ hashcode2 now when the Tracker finds that the hashcode stored and the State hashcode are different it updates what need to be updated! only once because after updating it now it has the same hash code as the State of the current Data!
State 3: same thing and so on
So these two things can be easily fixed I think? since simple to implement
First one is appending the option to .find() to control the reactivity as you said
The Second is before run is called check if the hashcode stored is the same of the Data State? if yes re-run and save the hash code if no return 0
What do you think?
Donāt lose hope bro and drop all your code of your amazing app, you know how many people right now have their apps built in blaze?
Some big projects like : Telescope, RocketChat and if they had to move to react they will need lots and lots of work to refactor to React!
So better Fix Tracker?
MDG Should just have fixed the performance issues of blaze which were caused as you found by Tracker! and Blaze will rock! (on mobile too)
BTW, I made a data flow framework in Haskell as well.
So yea, of course there are always trade-offs. Pure functions vs. side-effects. Strict typing vs. dynamic typing. The point is that it is hard to say that one is the best than another. The are suitable for different things and all of them require you to learn best practices and patterns. In Tracker those are different than in React. What I am saying that once you know Tracker good patterns then it is often good enough.
So this is why I would rather see development focus in other things Meteor needs, not Tracker. But of course, there is always room for improvement.
