RethinkDB could “rethink” filling that void somehow. Perhaps by implementing methods similar to Meteor’s methods from within the database that automatically scale with the database. Sort of like the concept of stored procedures only over Sockets.
Meteor being tied so strongly to Mongo is practically acting like a dedicated product for it–what RethinkDB could implement for itself one day. Meteor adds value to Mongo, possibly more value than itself as an entity (I’ve read MongoDB is valued at over $1.6 Billion, Meteor as a real-time platform dedicated to Mongo could feasibly be worth more to MongoDB than Meteor’s current valuation).
What would be powerful is if Meteor, as a Javascript platform, would provide an API to package authors. The Accounts package written with the API could automatically “Just Work” with whatever the database is in the supported stack or at least degrade nicely.
It sounds cool. If I’m correct, on the front-end, we’d put up the query for the data we need, and on the back-end, we would define where that query needs to go to get the data. I imagine it would be like a standardized way of writing a method and calling it.
Since April this year, my business has been using Ben Green’s numtel:mysql package in several production systems handling real-time data, performing reactive updates. My business has contributed fixes and enhancements to this package and we are about to announce a few more.
For example, it will now be possible to subscribe to a published MySQL query using Meteor.subscribe(), just like you would with a published Mongo Collection. The user can choose to use one or more fields in the SQL result set to be used as the substitute for the Mongo unique document id. Alternatively, the numerical index of the row can also be used as the unique document id.
MySQL 5.7 was recently released and one of its most exciting additions is support for a JSON datatype. This will make it much easier to give Meteor MySQL support for popular packages like Accounts (accounts-base).
We see a bright future for enterprise apps that leverage the strengths of Meteor and MySQL.
We need to refactor the Meteor data system to let you fetch data from any source (database or API) with minimal engineering effort. This should support optimistic UI, manual invalidations, and polling out of the box.
This post is going to look like a shameless plug- but I made it nonetheless in the hope that it can spark a bit of cross-pollination. What you describe is exactly something we’ve been working on for 4 years, and it would be criminal to not at least share our ideas.
Our project is called Corto, and is an MIT licensed, backend data management API designed for building microservices. Corto combines data from multiple sources while providing a uniform (typed) API, whether data is realtime or “at rest”. Adding new connectors is very easy, by implementing a so called “replicator”.
The Corto API supports:
Data replication using CRUD
Poll & merge data across different data sources
RMI that “hops” automatically to the process where the object “lives”
Object hierarchies
Object graphs (references)
Object invalidation
Realtime notifications
Automatic plugin-based translation between representations (JSON <-> XML <-> binary <-> ???)
Caching remote data (for example, store result of remote REST queries)
Persisting local data
A shell for exploring the Corto datastore (& stores connected to it)
Noteworthy roadmap items:
Request object history across different data sources
Perform queries at the source across different data sources
Data units (miles, km, Celsius etc)
The ‘directed state’ pattern ( a “richer” latency compensation w/request vs. actual values)
We’ve recently developed a DDP connector that makes it very easy to communicate with Meteor front ends, demonstrating that the two frameworks can be quite complimentary.
The API is written in C and low-footprint (~300kb). We’re finalizing v1, after which the focus will be on growing a connector ecosystem. High on the wishlist is integration with time-series databases & mongodb, as well as popular REST interfaces like fitbit & the likes.
Since it’s in the interest of both Corto / Meteor’s users to have a rich (data-centric) connector ecosystem ASAP, it would be great to have our communities join forces.
Finally, to give you an idea, this repository shows a simple Corto replicator. This repository shows a replicator that seamlessly ‘mounts’ a filesystem into the Corto store. This repository shows how Corto handles realtime updates. For a deep dive, see the Corto core specification.
