Here’s a link that was featured in the Syntax newsletter today:
Most apps should use a sync engine instead of doing RPC.
It’s more clear to me now than ever before: This will become the new default for future app development.
Gets rid of massive amounts of complexity and empowers frontend developers
The OP defines sync engines like this later in the thread:
An end-to-end system that syncs data directly into clients (often into a client-side database) instead of the client having to manually fetch/post data.
Correct me if I’m wrong, but doesn’t that sound like Meteor live data? If so, maybe we should add some posts to this thread.
You are partly correct. However, Meteor live data doesn’t cache or sync things yet. There needs to be more work so it could be considered a true sync engine. Right now, we only fetch the data (initial add) and then react to changes from the server.
Given that Meteor caches data in client-side MongoDB, what are one or two of the differences between Meteor live subscriptions, and a true sync engine?
Yup, never used Meteor without adding persistence either.
I’d say with few packages/lines of codes, Meteor is exactly what they’re advocating for.
The only thing I’d really loved improved is subscription performance and decoupling of update queue, it was always pretty standard part of many boilerplate codebases to introduce offline-first/ssr/webpack to Meteor.
Here’s a way to test different headlines on the Meteor home page - e.g. test headines that feature “the leading sync-engine framework” vs. 5-20 other headlines.
Javascript weekly is talking about sync engines today - featuring a library to help build them – but notes that this library can be hard to ‘get’ – which supports the case for Meteor as an easy-to-use full-stack sync-engine framework.
Yjs is a bit different than what Meteor does. Yjs is for when you are working on the same document with multiple people. Think Google Docs when working at the same time with multiple people. You see their cursors and their edit live. Where Yjs stands is the resolution of the changes, this allows multiple people to work on one sentence for example without it becoming totally crazy. It creates a queue of all the changes and then resolves it one change at a time. It also does conflict resolution.
Meteor saves the data as they come and updates everyone who is subscribed to that data. It is not really suited to editing documents where the latest change gets saved because you might be missing changes from somebody else who is editing the document.
While Sync Engine can have different implementations, the main challenge with heavily relying on raw sync is that you’re exposing your database structure to your client, and creating a very big risk of accidentally exposing sensitive values if any one developer forgets to properly filter the fields (I’ve seen this happen countless times with my team over the 10 years we used Meteor).
To be more scalable, I would prefer a Sync Engine that lets you explicitly define all reactive dependencies (similar to React useEffect) and recalculate the output on the server when any of these change. This way it will always send the transformed version to the client without exposing all direct data - sometimes you just need to show a high level realtime stat based on large data and it doesn’t make sense to ship all of it to the client just to calculate a few fields.
I’d be curious to hear from everyone what you would want in your perfect version of Sync Engine - I’m actually building live data engine for full-stack JS (inspired by Meteor) and happy to take suggestions/preferences on what you would change compared to Meteor.
