I’ve been trying to figure out how to support async method stubs in zodern:relay without the issues the current implementation in Meteor has. I think this is the most promising solution I’ve found.
Method stubs is another name for Methods implemented on the client (by
calling Meteor.methods on the client, importing validated methods on the client, etc.). These simulate what the server will do, allowing the user to immediately see the result. After the stub finishes, Meteor runs the method on the server, and updates the client state if the server result is different from what the stub simulated.
Many methods are not implemented on the client (some apps don’t have any stubs).
Before Meteor 2.8, all stubs were sync (they couldn’t use promises, async/await, or other async api’s).
Meteor 2.8 added support for async method stubs. The current implementation has a few limitations that will hopefully be fixed. There’s been a number of discussions about it over the last year, but there isn’t a good solution to it.
When running a stub, Meteor creates a simulation to track what the stub does and simulate what the server will do. This simulation is global - any code that runs is inside the simulation. With sync stubs this works fine - no other code is able to run in the simulation. However, with async stubs it is very easy for outside code to run during a simulation.
During a simulation, some of Meteor’s api’s behave differently, which can cause unexpected results when code outside of the method stub uses them:
Meteor.applyAsynconly run the stub when called in a simulation (they expect the server implementation of the first method will also call them)
- Due to a bug,
Meteor.callAsyncerrors if called during a simulation
- Modifications to a collection (
Collection.update, etc.) in a simulation are tracked. When the server result from the method is received, the modifications are reverted or updated to match what the server did. For apps that modify collections outside of stubs (using the allow-deny package) this can cause changes to be lost if they happen to be made while a simulation is running.
- Some Meteor apis do not allow themselves to be used while a simulation is running.
Currently, the recommendation is to use async/await or
.then callbacks to ensure the app only calls one method at a time and wait until it receives the server result before calling another method. For some apps this is difficult to check for - there are many ways multiple methods could be called at the same time in different files or in different functions that can require a good understanding of the browser and js libraries used to identify.
I created a package that modifies Meteor to implement this proposal. Add it with:
meteor add zodern:fix-async-stubs
Then you can call methods whenever and however you want without worrying about the problems mentioned above.
For it to completely remove those problems, async method stubs can not use some web api’s:
- setTimeout or setInterval
- web workers
- any other async web api that schedules and waits for macrotasks
If an async method stub does use one of these, a warning is logged to the console with a link to the docs.
Browsers have two queues - a macrotask queue and a microtask queue (mostly for promises) that it runs during the event loop. It runs one macrotask at a time. When the task finishes, it then runs all microtasks until the microtask queue is empty. While a macrotask and the microtasks scheduled by it is running, unrelated code does not run.
This proposal is to schedule a separate macrotask to run each async stub in. As long as the method stub only schedules and waits for microtasks (no macrotasks), it will finish running before any other app or package code can run.
The api’s that only support sync stubs (Meteor.call and Meteor.apply) run the stub immediately. This is still possible (my package does), as long as the methods are still run on the server in the original call order. This means that the stubs could run in a different order than the methods run on the server increasing the chance of the stubs producing different results than the server. Depending on how large of a breaking change it is, it might be possible to only run a stub out of order when
Meteor.apply is used with the
- This proposal is reliant on some assumptions about how browsers and the event loop work. It is possible my understanding of these is incorrect, or future changes to the specifications could break these assumptions.
- If this happens, it most likely would be specific api’s that break it. We could probably wrap those api’s to ensure they don’t run app/package code while an async stub is running.
- This requires browsers to be adequately spec compliant for the event loop and how various api’s interact with it. For example, this would not work in old versions of Firefox (I think it was fixed in Firefox 58).
- Can we clearly explain how to write async method stubs, and show a warning message that new developers can understand?
- It does restrict what api’s method stubs can use. Are these restrictions reasonable or easy to work around if necessary?
- Personally, I don’t think method stubs should be using these api’s anyway. These make a simulation take several macrotasks. This allows unrelated macrotasks to run unrelated app/package code for event listeners, setTimeout callbacks, or other code while the simulation is running. Even if you use async/.then to run one method at a time, that isn’t enough when method stubs use these api’s.