App architecture for distributed database

Hi,
We are thinking of using Meteor for an application monitoring multiple remote industrial sites.
We have several remote ‘field’ sites (A, B, C, …) with poor internet connectivity (dropping often).
On each site, we thought of a meteor app (on server side : data logging on a couchdb database, on client side : webapp for local monitoring). Each meteor app should be fully usable on each site even without internet connection.
For the global webapp, we thought of a meteor central app built on top of a couchdb instance getting its data thru filtered replication from remote sites when they are connected. The client webapp will be used to monitor several remote sites. Any data modified from this global app and the global couchdb database will then be synced (filtered) for each site.
We want to use for each meteor app (local and global) : meteor, couchdb, apollo/graphql and react for the front end. Monitoring of sensitive data should be realtime.

We failed to interface meteor, couchdb and graphql subscriptions.
Do you have any advice ? Should we use mongodb instead (but we don’t figured how to have the filtered distributed replication offered by couchdb) ?
Thanks

Anybody there to help us out ?
:slight_smile:

I don’t have any experience with couchdb but did you check out mongodb replication?

I checked it out. Mongodb has a master which takes all the writes and replicate to slaves which you can read.

We have a lot of masters (the field) in remote area with poor or limited connectivity which should sync with the cloud both ways when a connection is available.
The opposite of mongodb !

As far as I understood, I think one of the easiest solution would be:

Every site has a meteor app running.
Every database operation is replicated to the master database:

  • After every insert, update and remove add the change to collection ‘queue’.
  • Cron: Check if secondary MongoDB is connected, execute every operation in the ‘queue’. Add operation to a log and remove from ‘queue’.
  • If not, check again in X minutes.

I think this solution may be very easy to implement. You don’t need to interface with other Meteor apps, and it can be very abstract, supporting multiple databases to sync.

You could use the packages:

  • matb33:collection-hooks: to get the collection hooks (insert, update, remove).
  • percolate:synced-cron: to get the cron running.

To connect to other instances of MongoDB:

Use: MongoInternals.RemoteCollectionDriver();
Init it at startup time, or at cron time.

With this solution, you only need one ‘Master’ MongoDB, that every site syncs to. Your ‘Master’ Meteor app will get the updates automatically reading the oplog.

PS: You can filter what they are going to sync.