Agreed. I do believe Tracker and its diff-ing engine need work. For instance, it was built for Mongo, with a focus on _id. It also does not go deep enough in objects to clearly identify what changed. Likely it was enough for observe and observeChanges but not detailed enough for a templating engine.
Maybe we need a completely new diff-ing for Blaze - This repo is simply amazing and would do the job great.
Nice. Is it a fair observation to say React may perform better than Blaze in Meteor because with React, you are passing data down from a top component, thus, tracking far less dependencies?
I think @mitar had some really great points in that issue in response to your PR.
So does this along with your comment above mean you’re stopping work on ‘actual’ Incremental DOM?
Yes, we are stopping as the purpose of achieving improvements has been fulfilled. You won’t get improvement from Incremental DOM. Just like @msavin commented, the reason React was faster than Blaze lies elsewhere (in the diff-ing), which we worked around.
I don’t think so (can’t say for sure as I don’t know how React works internally).
With Blaze, you use reactive variables, the changes of which are injected directly into the DOM. Blaze knows exactly where that variable is located in the DOM, so the swapping is blazing (pardon the pun) super fast.
I think with React, it should be the same thing. Likely that React remembers where each function’s JSX returns fits in the Virtual DOM and can diff locally (in the Virtual DOM, so faster than actual DOM at the expense of memory).
In IDOM you can chose which element to affect, so it can diff locally too. However, the diff-ing is slower as it a DOM-diff (which is what Blaze did before my change, it would render a DOM element and compare) and is computationally expensive.
Technically, I expect Blaze to be faster, as we are diffing variables in JS, not DOM (like IDOM) or JSX (like React). But I am shooting from the hip as I don’t know how React diffs (maybe it diffs variables too within the Virtual DOM).
Lol, you stopped half-way through. Just because you observed optimization you think you are done? Of course is faster if you improve reactivity. But it is just partial faster from what it could be.
Again, every code has much more data than DOM. Data >>> DOM. Because humans have to process DOM. And we are limited in how much information we can take in.
So diffing at DOM level instead of data level means that you have to diff much much less. React observed this fact and discovered that it is faster to just compute everything and diff at the end.
Blaze is great because it has also low-level data management. So we can have the best of both words and leave to developers to tweak things as they need. But diffing only at data level will never be as fast as React. But combining those two things, that could be the most powerful thing ever. Especially if you can tweak them to optimize for a particular use case.
I talked with Pete Hunt (creator of React) at length about this when we organized his talk at Berkeley.
(We also had Matt from Meteor over. It is really interesting to compare those two views, especially now in retrospective.)
That’s why I am suggesting we should be doing diffing at the HTMLJS level. Remember old HTMLJS, diff, apply changes. That could even lead to long wanted feature: callbacks when Blaze changes something. From diff you could exactly know what changed and call callbacks for that.
Of course the downside is that you cannot use 3rd party things then. And this was a design consideration with original Blaze, when jQuery was very strong. I think it is a powerful thing to have, so we should create that you can switch this HTMLJS diffing on or off as needed. Some templates you would use diffing for, and for example you would not be able to use template.$ for it.
But we are repeating again things from GitHub issues and this is bad because arguments are being lost. So I will not comment here anymore on this and let’s move discussion back to GitHub again.
So we agree then, HTMLJS diff-ing is JS diff-ing.
Diff-ing at doc level will always be faster (unless the developer is providing more fields than needed). It’s a gift that Blaze gave us to be so low-level. And if we wanted to, we can diff only right at the end before a field is about to change (so we diff String, Number and Boolean vs Object). My PR does both checks (at the doc level and at the field level).
The challenge of diffing at the end is that you ran so many lines of code before getting to a field, diff-ing the source doc could be faster for unchanged objects – but that part will be flushed once Tracker is improved.
Even if I am wrong, we are 80% there. We have more important things to fix before we get to that last 20% (if it indeed exists) and start looking at semantics (and whether we can dispense with jQuery and the like).
Which is always? Because it is so painful to have to specify fields at Minimongo query, which you can then use in the template somewhere completely else and you have to keep those in sync.
We should really stop using of Minimongo fields and make use of JavaScript getters to register dynamic dependencies on fields as you access them, if you do so, in a template.
So yes, you can try to limit reactivity with limited fields, but it is really painful and is destroying logic/template separation.
So all those points you are mentioning are true. Those are points on how to optimize Blaze today. But we are discussing Blaze 2.0. The future. And in the future developer would not have to be doing those things.
Blaze 2.0 should be fast even if developer is doing bad things and are passing all data around. And it should have features to tweak things for those who want more control.
This is why Blaze is neat. Because it is easy for beginners. And sorry, limiting reactivity with Minimongo fields is not easy for beginners.
If we were in the world before React existed, I would maybe believe you. Now it is clear that this “intuition” you are having is plain wrong because React is doing exactly what you are saying that will not work, and they are doing it very well. So if you want to convince anybody that you are right you will have to show some numbers. For that you will have to implement HTMLJS diffing or incremental DOM and then show that you are right.
Maybe, but that doc-level diff-ing is only done because Tracker doesn’t do a good job, it shouldn’t even be there. So let’s skip it.
If you don’t like it / remove the first diff and keep the second (at field-level). That second one was not optimal its prior implementation.
And btw, if only relevant data is fed to template, that diff is blazing fast.
Also - what tells we can’t make blaze faster than react 
Ok, let’s fix Blaze 1 so we don’t lose people and then we can implement things right.
People want improvement NOW. If we lose users there will be no one left to work on Blaze 2. Many who switched to React are not returning because they have a working toolkit, we need to have a working toolkit today.
@ramez, it must be really hard to do ‘true’ Incremental DOM, huh?
I kinda agree with this as well. @mitar, I thought Incremental DOM was something on the Blaze 1 Roadmap?
To use incremental Dom with blaze we have to throw out quite a bit. Blaze is very low level - it implements its own incremental Dom. Switching to Google’s idom implementation is essentially removing meteor’s own ‘idom’ in favor of someone else’s.
Why would we do that? It’s working now. We barely have enough community resources, why waste them. Not to mention that we have to convince people that there is value in doing that so they dedicate their time to it.
Oh, I use “incremental DOM” and “virtual DOM” as a similar thing. It is the question of optimizing the materializer layer. And we should probably investigate which one is more suitable for Blaze. Incremental DOM seemed more suitable because it looked like it would be easier to adapt, but @ramez is reporting that this is not so. So, we can try then virtual DOM, which already has some precedent with Blaze.
Again, I am not saying that we should not be optimizing the reactivity, but that we should also investigate improvements to DOM layer rendering.
And for reactivity, it would be great to join the discussion there, because that was discussed already before and I think there even better solutions were suggested than this patch which changes defaults. So, if people want to join the discussion about reactivity, they should read this and this first
I looked at the repo you linked to. Doesn’t seem to be a proper / complete implementation. The whole of Blaze is still there. What I think the developer did was render from Blaze to vDOM then vDOM to actual DOM. So he didn’t solve the same issue I faced, is that the Blaze implementation is already low-level.
Incremental DOM was meant for (originally) non-reactive templating engines (like Mustache and Handlebars). So it’s easy to patch them up; instead of making direct DOM calls (document.createElement) they use the Incremental DOM API (elementOpen and elementClose). The change to the code base is minimal for such templates. But for us, it’s big. You have to flush out so much to get back the same functionality you had before with someone else’s library …
[Edit: Removed my non-funny comment at end]
@sashko, after reading this thread, what’s your opinion (or will you pull someone in that can provide new insights)?
I think people who use Blaze daily in their production apps are the best equipped to have opinions on this! I don’t.
Personally I never had performance issues in Blaze rendering - they were all related to Tracker stuff rerunning too often, or unexpectedly.
Can anyone comment on the state of incremental DOM? Is it “complete” and is Google committed to maintaining it?
I think implementing it would give credibility to Blaze, as it has a fancy-DOM utility underneath and one that is Google-backed.
Another, less attractive path, is to re-brand Blaze’s rendering engine. Maybe “Incremental DOM” would apply, or “Reactive DOM”, or something