Synchronous Collection methods removed from Meteor, now we need async signals

1

Removing synchronous collection methods from the backend is Meteor’s biggest hurdle (I wonder how many apps have yet to migrate?).

Without the old Fibers-based “synchronous” APIs, we are missing one very grand thing in Meteor: Isomorphic reactive code

In lieu of this, it would be great if Meteor would gain new signals-and-effects patterns (reactivevar-and-tracker.autorun patterns) for handling async values.

The issue:

Promises are not repeatable, whereas signals are. This makes a world of difference in writing clean robust code: signals and effects allow for logic trees (including async state) to be easily cancelable and repeatable, but the removal of synchronous-like collection methods has caused Meteor developers to loose the ability to easily create robust (as in clean and error free) code that can easily be canceled and re-executed.

With single-valued Promises, we now have a landscape of new and inflexible code that is difficult to refactor to make cancelable and repeatable, risking race conditions and errors, which can only be solved with unweidly and error-prone try-catch blocks all over the place.

TODO give code examples.

Towards a solution:

If you’ve been keeping track of the frontend JS ecosystem, you may have noticed a thing called “signals and effects” have proliferated across frameworks (Preact, Vue, Svelte, Angular, Lit, etc, and even React with its fake version of it called Hooks (let’s not get into that)).

Meteor was king of this pattern, with ReactiveVar/Deps (signals) and Tracker.autorun (effects) existing before the signald and effects of all those other frameworks.

Signals and effects have gotten a lot of use lately, and people have come up with a lot of new patterns.

For example, “async signals” could mean different ways of implementing signals and effects, or implementing async handling on top of signals and effects.

I don’t have much time now, so I’ll leave two things here for now:

1) a contrived example of async code running two things in sequence, without using async/await (here using Solid.js API naming):

createEffect(() => { // Solid's "Tracker.autorun"
  const [valueOne, oneDone] = processOne(input())
  const [valueTwo, twoDone] = when(twoDone, () => processTwo(valueOne()))

  const allDone = createMemo(() => oneDone() && twoDone())

  createEffect(() => allDone() && console.log('done'))
})

which is equivalent to the following Promise-based (non-robust) example:

async function (input) {
  const valueOne = await processOne(input)
  const valueTwo = await processTwo(valueOne)
  
  console.log('done')
})

The signals+effects code can sometimes be just a little bit more verbose than async/await, but the benefits are much more worth it.

I would like to write more on this topic (so far, it’s just been chats with people in the signals+effects community, namely Solid).

TODO more code examples.

2) A video that describes the robustness:

This video mentions “this/they/them” in reference to signals and effects for async management, it was a video I was giving someone in the context of a chat on the topic.

The video show an example written in a way similar to the contrived example above: sequences of async processes (in this case, sequences of animations) which can easily be executed in sequence, robustly canceled at any point in time, and repeated, without having to write unweildy error-prone try-catch/throw code as would be required with Promise/async/await, and without all the manual wiring and de-wiring of patterns like events, observables, pubsub, etc.

Now, I may need to give more code examples to fully explain this, but so far signals and effects allow writing the closest thing to declarative code for async processes (f.e. similar to CSS transitions and animations) that can easily be interrupted to let other process take over (in a similar way as how CSS transitions can be canceled/repeated/reversed simply by toggling a class name for example). Basically if one were to write the actual engine for something like CSS, this way of managing async code would be the best way to do it. Other patterns like Events, Observables, pubsub, and Promises, cannot achieve the same robustness in as clean of a manner in which all the pieces are composable while maintaining robustness.

If you know about how React Hooks greatly improved composition in React, well, same thing with signals and effects in Solid/etc (execept that signals+effects in Solid/etc are leaps ahead of React in at least a couple major ways: for one React Hooks are coupled to a component system and unusable on their own, and they do not have nesting hence no reactive robust cancelable/repeatable logic trees).

TODO more explanations and examples of these concepts, and how they are better than other patterns.

4 Likes
2

Can you elaborate on that?

3

in client-side, we might use abortController / signal to control:

async function (input, { signal }) {
  const valueOne = await processOne(input)
  if (signal.aborted) {
    return;
  }
  const valueTwo = await processTwo(valueOne)
  
  console.log('done')
}

But I wonder, can we send a signal to a server to cancel a method call? It’s rare, but if we have a heavy method, processing video for example, we might want to cancel the job running on the server at some cases.

4

Sure! Signals and Effects these days (including Meteor’s Tracker (effects in today’s common terminology) and ReactiveVar (signals in today’s common terminology)) typically allow effect nesting.

In Meteor that looks like this:

// outer effect
Tracker.autorun(() => {
  someVar.get() // tracks 'someVar' as a dependency

  // inner effect
  Tracker.autorun(() => {
    otherVar.get() // tracks 'otherVar' as a dependency
  })
})

In Solid.js that looks like this:

// outer effect
createEffect(() => {
  someVar() // tracks 'someVar' as a dependency

  // inner effect
  createEffect(() => {
    otherVar() // tracks 'otherVar' as a dependency
  })
})

And similar with Vue, Svelte, Angular, Preact, etc.

In React, useEffect cannot be nested inside another useEffect, which means that in React there are a variety of incredibly powerful compositional patterns that cannot be done.

5

Note that, in this example, processOne is not being canceled, but only the outer function is returning after processOne completes.

To make Promises cancelable requires a unweildy try/catch code, and the messiness of it will scale up the more promises are involved. This is where Signals and Effects (Tracker and ReactiveVar in Meteor) shine.

(Just in case to avoid confusion for anyone reading, “signals” in Signals and Effects, is not the same as the “signal” from an AbortController.)

There are two base network web APIs: XMLHttpRequest, and the more modern fetch API. If Meteor methods happen to use fetch, then it would be possible to make them cancelable (with Tracker.onInvalidate (the equivalent of onCleanup in Solid.js)) by having them abort an AbortController controller whose abort signal was passed into the fetch call.

In lieu of this, then the next best thing is to just early-out like in your example, without actually canceling the network request.

As far as actually sending a signal to a server to cancel a method call, I bet that’s possible, but the question is, would the value of it beat the complexity? I have a feeling it might not. As alternative to that, perhaps the system needs an undo mechanism so that partial updates to the database can be undone (also complex). Has anyone implemented this in an app-specific way we can look at? That would be interesting to see.

6

It sounds like using database transaction.