A few of us have been attempting to migrate withTracker away from the old class Component based implementation to a hooks based implementation. This attempt creates a new hook based alternative to the old HOC, while also porting the old withTracker to use the new hook, solving a specific problem introduced by React’s deprecation of the very practically minded componentWillMount.
There have been some interesting challenges implementing that new hook, and some things I’ve learned about hooks in general that I thought I’d share. I’ve also been thinking about alternative ways to solve some of the problems in a more integrated way than useTracker and withTracker current afford.
The most important thing to understand with functional components in React, is that they really should not create any “side effects” in the render method (which is the only method you have for functional components). This means you shouldn’t mutate or set any data outside of the function scope, or create asynchronous code (callbacks, or subscribe to events) which will be called later. Instead, all that type of stuff should go in a useEffect hook, which will execute after the first render IF the render is committed. Now this is the complex part - renders are sometimes thrown away! There are 3 scenarios that might cause that - Concurrent Mode (multiple renders happen at once), Suspense (if a promise is thrown before the render commits), or Error Boundaries with handled errors. If you are using Concurrent Mode, renders are thrown away MORE OFTEN than they are committed.
This makes things tricky for Meteor. We’ve gotten used to doing everything in a single tracked function in withTracker - calling subscriptions, which set up an obvious side effect, and data queries inside Tracker.autorun all create a side effect, because Tracker.autorun creates a Meteor computation. This computation watches for accesses to reactive sources using the observer pattern, and makes sure to re-call the reactive function whenever any of those APIs notify the computation of a change.
The direct port path for updating withTracker seemed easy enough - convert the single tracked function paradigm from a HOC to a hook. The algorithm is simple then - run the tracked function once during render without side-effects (either monkey patch Meteor.subscribe to avoid side-effects, or set up the computation, then immediately destroy it after one run), and then set up the actual computation in useEffect (and then trigger at least one re-render). The problem with this approach is that it means 2 renders for every mount. The React team says that’s fine - but in my testing (and others) this proved to be quite janky.
There were also a lot of subtle bugs that would make that implementation challenging - for example, if you use a hook in a component like Layout, which is used for multiple routes in a React Router Switch block, RR will re-use an existing instance when the route changes, and only change the contained elements. This means there could be a gap between the route change, and the re-creation of the computation (on mount it still uses the old computation, and doesn’t update until useEffect).
One workaround for that is to invalidate the computation when an update occurs instead of running the computation immediately, then force a re-render, and in the next render create a new computation (this is actually how withTracker has always worked, so it’s not a big problem). Of course, we are back to a sort of circular problem - creating side effects in render is a no-no!
But we wanted something more - we wanted a way to optimize the hook a bit better, to at the very least provide a way to keep the computation around, and not rebuild it every time - and to not require 2 renders for every mount (and other case) by default. We did that by accepting a deps parameter, which works almost exactly like React’s deps in useEffect or useMemo (it uses useMemo behind the scenes). We also wanted a backward compatible solution for the withTracker HOC, which could not accept deps, so we needed to also support a path that doesn’t require them.
Because there is no hooks replacement for componentWillMount, and there is no way to clean up “tossed” renders, and because doing things later in useEffect simply isn’t good enough, we ended up creating what might become a common pattern - a poor man’s garbage collector, based on setTimeout. Basically, we create the side effect in render, then make sure to clean it up in cases when the render is not committed. If the render is committed, we clear the timeout in useEffect and avoid re-render.
This had its own set of challenges and edge cases to worry about. What if we are in concurrent mode, and 4 renders of this component run at the same time - do we keep 4 computations around waiting for mount - which can take up to 1 full second! What do we do with 4 reactive updates that happen before commit? I decided to ignore those updates, and invalidate the computation, and re-create it in useEffect for the one committed render - the alternative would have been to run the users reactive function 4 times, which seemed wasteful. What happens if it takes longer than 1 second for the component to mount? The timeout based GC will have destroyed the computation - we’ll have to recreate it in useEffect and eat a second render. These were all pretty interesting and challenging to get right - though I think it’s pretty solid now.
What’s interesting is that this is a common problem for observer pattern APIs like that used in Meteor.
I believe the hook is pretty solid now. We’ve been testing the heck out of it for weeks. We are now at the phase of development where we’re starting to think about new advanced features to add to it, such as a way to optimize the compare function which would design when to force update (or not), which the very excellent @menelike (I don’t know your forum handle!) has been trying to get in, and maybe a transform method, a computation lifecycle handler (which is already in, but undocumented/might change), etc.
We’re also thinking about some other possibly more integrated hook options, which throw away some backward compatibility. This is actually the main point of writing this whole thing (way to bury the lede!!). The React team seems adamant about not giving us a super easy way to clean up side-effects from discarded renders, but that’s not even the main idea here. In Blaze, reactive API points were fairly well integrated with Blaze. There was no type of HOC, or special function or anything like that that you needed to stuff all your Meteor APIs in to. With React, Meteor’s APIs never really felt integrated. So what I’m wondering is, should we be focusing on creating more integrated APIs that work with Meteor, instead of just trying to isolate it?
For example, instead of something like this:
const PageView = ({ pageId }) => {
const [page, isLoading] = useTracker(() => {
const subscription = Meteor.subscribe('page', { pageId })
const page = Page.findOne({ _id: pageId })
return [page, !subscription.ready()]
}, [pageId])
return (
<div>{isLoading ? 'loading' : page.content}</div>
)
}
What if things were a bit more compartmentalized, and integrated feeling:
const PageView = ({ pageId }) => {
const [isReady] = useSubscription('page', { pageId }, [pageId])
const [page] = Pages.useFindOne({ _id: pageId }, [pageId])
return (
<div>{!isReady ? 'loading' : page.content}</div>
)
}
With a set of APIs like this, we can drastically simplify the implementation and make it completely compatible with React’s rules of hooks, and side-step the need for creating side effects in render altogether. The useSubscription hook above could just wrap useEffect and the query hook could do a number of optimizations to make sure it’s not always re-rendering, and do the initial query without any kind of side-effects on render. This would have to include using some kind of more robust comparative function to avoid updates, or we could place an immutable guarantee between the old query (which always returns a new object) and the hook. With a system like that we might even be able to infer the deps, instead of requiring them be explicitly defined.
I’m also wondering (and this is truly pie in the sky) if for list queries (Collections.find) we can create some way to iterate over a cursor in render, and have it set up an individual computation for each document, instead of always blasting the entire list at the renderer using .fetch. This one is maybe a bit out there, but it seems it should be doable in some optimal way.
It would be a bit of work, but by constraining the API, we can make better assumptions about the shape of the queries, and the results, and optimize better, while staying consistent with the React way of doing things. What do you think?