Just saw it recently. It adds subscription support to PostgreSQL, which is the SQL implementation Meteor originally attempted. I reviewed its code once quickly and they seem to be doing a Triggers + Stored Procedure approach to sync with Meteor webservers (and the subscriptions they are currently monitoring). Perhaps PipelineDB provides a more “native” and performant solution for subscriptions, as well as hopefully a simpler interface.
Personally, I think any database support that relies on someone having a special database is doomed to fail. Many people have an existing database with a lot built on top of it, and aren’t excited about changing that.
1 Like
I think the right solution is going to be having a database connector which is not realtime by default, letting anyone write a basic connector for any database, and then have a way to trigger reactive changes on that as an added feature.
it claims to be “Postgre Compatible.” From the homepage:
PostgreSQL Compatible
PipelineDB is built into the PostgreSQL core and can be used as a drop in replacement for PostgreSQL without making any application code changes. It comes pre-installed with PostGIS and supports all other extensions too.
so basically build non-subscription-based support for, say, MySQL using GraphQL, and then provide an interface (or perhaps raw GraphQL) to make certain queries reactive? It seems everyone is determining that having an entire DB be reactive is just not scalable (to the level we would like, unless maybe it’s Subscribable by default like Datomic)–so it’s best to provide a mechanism to intercept certain queries and make them reactive on the webserver layer, rather than coordinate any subscriptions directly within the DB. In addition, rather than the whole DB be reactive, leave it up to the developer (and provide some helpful tools) to make some queries reactive on a query by query basis. That seems to be why Facebook avoided the LiveQuery approach. @sashko is that the sort of conclusions you’ve come to?
Are you not seeing it as possible to make, say, MySQL reactive/observable only using the webserver layer? I’ve been wondering for a while what the challenges are there–cuz obviously if someone subscribes to say the most recent 10 comments on an article, then as new comments come in, you need to deem they are part of that subscription. I imagine there are use cases here that just make it impossible to track this properly or have challenges with webservers now maintaining too much state again. I’d love to get a better understanding of the challenges with building subscriptions 100% completely on the webserver layer with no fancy DB support like oplog. How do you see it @sashko?
Yeah, something similar to that. The situation is made much clearer if you realize that a lot of what used to be in “databases” is now in REST APIs or similar, which are never going to get a realtime driver.
@justinsb and I are currently doing some experiments and aren’t ready to discuss details of an approach before validating it a little, but it’s basically aligned with what you’re saying. Expect more details sometime next week probably.
The real trick is going to be doing that while preserving the buttery-smooth data loading experience Meteor had when hard-wired to a specific database, and that’s our top priority.
2 Likes