Meteor suitable for realtime collaboration of heavier data?

help
1

Hi,
I’m looking for a Google Docs style realtime collaboration solution and trying to assess whether Meteor is a suitable solution for my use-case. Any advice appreciated.

  • The app itself is a 3D content management system. Primarily the data being synced is object properties with simple types (string, numbers), but also includes heavier data types like arrays of floats such as vertices, pixels, etc.
  • Collaborators can use both native mobile apps and web apps running WebGL in a desktop browser.
  • Underlying data can be encoded into JSON or key/value sets for sending on the wire, merging, etc.
  • Both client and server utilize C# heavily, currently no JS being used anywhere (using web assemblies for browser stuff). But C# has support for web sockets, HTTP, etc.
  • Not using React, Vue or other JS UI frameworks: Using native UI and custom WebGL to render all GUI
  • The number of collaborators with be small (2-10 people) and generally people won’t edit the exact same thing at the same time.

Any thoughts on whether I can leverage Meteor for my needs?

Thank you,
Brett

2

Hi. That’s a rather wide question to answer. But a good starting point is to think about how the out-of-the-box Meteor realtime data layer works. Although Meteor apps are node apps and you can essentially build anything on top of node, considering using Meteor only makes sense if you plan to use its bult-in pub/sub system. In that system you don’t have to manage individual websocket messages. Instead you put data into mongodb and connected clients make live queries into it. Is it acceptable for you to add mongodb to the backend? I don’t see any reason why it would be a problem in this case, but you probably have more information than I do.

If you determine that the default pub/sub system works for you, then Meteor can provide you some sigificant benefits. You can have a realtime system up and running really fast and you’ll be able to iterate quickly.

Maintaining publications on server side is expensive, especially if these hold a large amount of data. But if the expected user count is low, then that should not matter much.

If you want to use Meteor in native mobile applications, then you might run into trouble. In a JS environment (i.e. Rwact Native) you could use SimpleDDP.

5 Likes
3

Thanks for reply and info. Will check it out.

4

Some considerations:

  • MongoDB is ok for storing the type of data you have and one of the fastest to retrieve this data. However, you’d rather go with a self maintained Mongo in your local VPN for lowest latency and transfer costs.
  • I am thinking of SVGs which might be the case for some of the data type you have. In SVG, you cannot save some part of the SVG strings, you need to do a complete overwrite. Similarly, you might not be able to update a vertice with every geometry change and you might need to update an entire object (or other data structure) of vertices. Doing this at every click/drag is costly and could be laggy and buggy if you keep sending large writes to the DB.
  • You would definitely have to use the in-memory DB provided by redis-oplog for large documents reactivity.
  • The way I would do it, I would use Node Streams for collaboration. Example:
    A) If I am alone on the document, no need for reactivity, edit the document in memory and Save (and autosave changes every … x seconds)
    B) Somebody joins the document: will pull the latests save from the server, will get the latest local updates from the other user via Streams and both continue to receive the updates from others via streams. With multiple users on a document, I would keep the first user as “master” to receive and send Streams from the other users and update to Mongo (instead of all writing to Mongo).
    Streams will help you keep data flowing between clients (reactivity) via the NodeJS Server and have control over how often you want to write to Mongo.

In general Meteor will give you out-of-the-box setup with DB, a server, an in-memory DB layer (redis-oplog) and reactivity via both Streams or Meteor’s redis-oplog.
I feel that in general you need that in-memory DB buffer between your clients or use one of the clients to keep track of things via Streams. Writing every little change to the DB I don’t see it as on option, regardless of what the DB may be.
I used to do 3D processing with Blender with the project hosted in Dropbox. Somehow Dropbox was extremely efficient in passing these work updates between peers.

5

Thanks for taking the time to reply and offer some insights. Very helpful.