useTracker React hook lessons learned

A few of us have been attempting to migrate withTracker away from the old class Component based implementation to a hooks based implementation. This attempt creates a new hook based alternative to the old HOC, while also porting the old withTracker to use the new hook, solving a specific problem introduced by React’s deprecation of the very practically minded componentWillMount.

There have been some interesting challenges implementing that new hook, and some things I’ve learned about hooks in general that I thought I’d share. I’ve also been thinking about alternative ways to solve some of the problems in a more integrated way than useTracker and withTracker current afford.

The most important thing to understand with functional components in React, is that they really should not create any “side effects” in the render method (which is the only method you have for functional components). This means you shouldn’t mutate or set any data outside of the function scope, or create asynchronous code (callbacks, or subscribe to events) which will be called later. Instead, all that type of stuff should go in a useEffect hook, which will execute after the first render IF the render is committed. Now this is the complex part - renders are sometimes thrown away! There are 3 scenarios that might cause that - Concurrent Mode (multiple renders happen at once), Suspense (if a promise is thrown before the render commits), or Error Boundaries with handled errors. If you are using Concurrent Mode, renders are thrown away MORE OFTEN than they are committed.

This makes things tricky for Meteor. We’ve gotten used to doing everything in a single tracked function in withTracker - calling subscriptions, which set up an obvious side effect, and data queries inside Tracker.autorun all create a side effect, because Tracker.autorun creates a Meteor computation. This computation watches for accesses to reactive sources using the observer pattern, and makes sure to re-call the reactive function whenever any of those APIs notify the computation of a change.

The direct port path for updating withTracker seemed easy enough - convert the single tracked function paradigm from a HOC to a hook. The algorithm is simple then - run the tracked function once during render without side-effects (either monkey patch Meteor.subscribe to avoid side-effects, or set up the computation, then immediately destroy it after one run), and then set up the actual computation in useEffect (and then trigger at least one re-render). The problem with this approach is that it means 2 renders for every mount. The React team says that’s fine - but in my testing (and others) this proved to be quite janky.

There were also a lot of subtle bugs that would make that implementation challenging - for example, if you use a hook in a component like Layout, which is used for multiple routes in a React Router Switch block, RR will re-use an existing instance when the route changes, and only change the contained elements. This means there could be a gap between the route change, and the re-creation of the computation (on mount it still uses the old computation, and doesn’t update until useEffect).

One workaround for that is to invalidate the computation when an update occurs instead of running the computation immediately, then force a re-render, and in the next render create a new computation (this is actually how withTracker has always worked, so it’s not a big problem). Of course, we are back to a sort of circular problem - creating side effects in render is a no-no!

But we wanted something more - we wanted a way to optimize the hook a bit better, to at the very least provide a way to keep the computation around, and not rebuild it every time - and to not require 2 renders for every mount (and other case) by default. We did that by accepting a deps parameter, which works almost exactly like React’s deps in useEffect or useMemo (it uses useMemo behind the scenes). We also wanted a backward compatible solution for the withTracker HOC, which could not accept deps, so we needed to also support a path that doesn’t require them.

Because there is no hooks replacement for componentWillMount, and there is no way to clean up “tossed” renders, and because doing things later in useEffect simply isn’t good enough, we ended up creating what might become a common pattern - a poor man’s garbage collector, based on setTimeout. Basically, we create the side effect in render, then make sure to clean it up in cases when the render is not committed. If the render is committed, we clear the timeout in useEffect and avoid re-render.

This had its own set of challenges and edge cases to worry about. What if we are in concurrent mode, and 4 renders of this component run at the same time - do we keep 4 computations around waiting for mount - which can take up to 1 full second! What do we do with 4 reactive updates that happen before commit? I decided to ignore those updates, and invalidate the computation, and re-create it in useEffect for the one committed render - the alternative would have been to run the users reactive function 4 times, which seemed wasteful. What happens if it takes longer than 1 second for the component to mount? The timeout based GC will have destroyed the computation - we’ll have to recreate it in useEffect and eat a second render. These were all pretty interesting and challenging to get right - though I think it’s pretty solid now.

What’s interesting is that this is a common problem for observer pattern APIs like that used in Meteor.

I believe the hook is pretty solid now. We’ve been testing the heck out of it for weeks. We are now at the phase of development where we’re starting to think about new advanced features to add to it, such as a way to optimize the compare function which would design when to force update (or not), which the very excellent @menelike (I don’t know your forum handle!) has been trying to get in, and maybe a transform method, a computation lifecycle handler (which is already in, but undocumented/might change), etc.

We’re also thinking about some other possibly more integrated hook options, which throw away some backward compatibility. This is actually the main point of writing this whole thing (way to bury the lede!!). The React team seems adamant about not giving us a super easy way to clean up side-effects from discarded renders, but that’s not even the main idea here. In Blaze, reactive API points were fairly well integrated with Blaze. There was no type of HOC, or special function or anything like that that you needed to stuff all your Meteor APIs in to. With React, Meteor’s APIs never really felt integrated. So what I’m wondering is, should we be focusing on creating more integrated APIs that work with Meteor, instead of just trying to isolate it?

For example, instead of something like this:

const PageView = ({ pageId }) => {
  const [page, isLoading] = useTracker(() => {
    const subscription = Meteor.subscribe('page', { pageId })
    const page = Page.findOne({ _id: pageId })
    return [page, !subscription.ready()]
  }, [pageId])
  return (
    <div>{isLoading ? 'loading' : page.content}</div>
  )
}

What if things were a bit more compartmentalized, and integrated feeling:

const PageView = ({ pageId }) => {
  const [isReady] = useSubscription('page', { pageId }, [pageId])
  const [page] = Pages.useFindOne({ _id: pageId }, [pageId])
  return (
    <div>{!isReady ? 'loading' : page.content}</div>
  )
}

With a set of APIs like this, we can drastically simplify the implementation and make it completely compatible with React’s rules of hooks, and side-step the need for creating side effects in render altogether. The useSubscription hook above could just wrap useEffect and the query hook could do a number of optimizations to make sure it’s not always re-rendering, and do the initial query without any kind of side-effects on render. This would have to include using some kind of more robust comparative function to avoid updates, or we could place an immutable guarantee between the old query (which always returns a new object) and the hook. With a system like that we might even be able to infer the deps, instead of requiring them be explicitly defined.

I’m also wondering (and this is truly pie in the sky) if for list queries (Collections.find) we can create some way to iterate over a cursor in render, and have it set up an individual computation for each document, instead of always blasting the entire list at the renderer using .fetch. This one is maybe a bit out there, but it seems it should be doable in some optimal way.

It would be a bit of work, but by constraining the API, we can make better assumptions about the shape of the queries, and the results, and optimize better, while staying consistent with the React way of doing things. What do you think?

28 Likes

Amazing work @captainn! I hope we can get this merged soon @benjamn.

1 Like

@captainn

Very well written! I agree, IMHO every reactive data source should have it’s own hook.

4 Likes

Wow, very great explanation! So, what is the current status then? Is there a hook version of react-meteor-data available?

I know there’s an NPM package available, react-meteor-hooks, but it didn’t take all these details into consideration and I couldn’t really make it work in my project, after several attempts…

A more fine grained API that would split the different requests in small chunks would be ideal, I think. Thanks again!

There’s a PR for it to get it integrated into the react-meteor-data package, but it hasn’t been reviewed or accepted yet. You can grab a copy from my branch if you like, and put it in your packages directory to try it out.

2 Likes

hah! I started trying to do this the other day and rapidly managed to confuse myself. Thanks for the super-detailed writeup!

1 Like

One hook per reactive source definitely makes sense to me. However, as reactive sources are extensible (e.g. ReactiveDict), there should be a more low-level API to build hooks for them on top of it.

The thing about writing hooks is that they are relatively easy to write, except in the one case where you want to either set things up early, or you want to set things up at the same time as the first render. This is more common than the React teams wants to admit. Meteor’s Tracker performs well when you do it this way, but React team went and removed (or deprecated) all the ways to do that cleanly, in pursuit of functional purity (which they won’t admit to) and more esoteric future optimizations like concurrent mode. They say it’s necessary for concurrent mode optimizations to work well, but I suspect it’s more about controlling the kind of code people write on the platform (it won’t work - folks will write worse code, not better. You can already see it if you look around at various code postings and attempts to code around the lack of a componentWillMount hook).

There is a way forward in many cases, using contexts to hide the double renders, etc. But it’s all a lot more work, and much harder to reason about than just having a componentWillMount or at least a side-effects cleanup hook.

BTW, I wrote a package called npdev:meteor-react-state on the current useTracker hook for managing ReactiveDict, and built a Context driven Session pattern on top of it as well.

1 Like

Actually, I don’t like the way the React team deprecated all of these life cycle methods. They worked pretty well, and now they tell you they were unsafe. Maybe I’m just to dumb, but the traditional life cycle methods were pretty easy to grasp whereas the hooks stuff took a while to digest.

I’ve been looking pretty hard at Svelte over it. This is normal - typically, front end tech turns over every 5 years. React is 6 years old.

I even do like the hook patterns (not happy about he closure confusion, but the patterns are nice). They could even have simply provided a componentWillMount or similar hook, or a cleanup hook so you can clean up side-effects created during rendering, they just have chosen not to. But really, the hooks pattern is so different from previous versions of react, they should have just called it something else - maybe “hoops”.

On the other hand, I find myself using refs a lot, which would be much much easier to think about if they were just a class (refs are basically a psuedo-functional alternative to “this”).

2 Likes