WYSIWYG Editors and Collaborative Edit Safety

help
1

@mitar, @aadams, @seeekr, @camiel, @robfallows, @mizzao, @vioan and anyone else interested

Sorry to bother but in looking around I’ve seen your names pop up related to WYSIWYG editors and was hoping to have a discussion about collaborative editing.

Have any of you solved or have any ideas on how one would solve the issues that could come up when multiple users could potentially edit the same document simultaneously.

I know Operational Transform or some other type of CRDT implementation that interfaces to an editor would be the holy grail. Looking around though almost none of the common WYSIWYG editors could even support this. Solutions like Etherpad basically need to be implemented outside of Meteor and I’d imagine it would be a rough road to figure out how to get the two to interact seamlessly. Sharejs is good for plain text but there is a bunch of work to get it to support Rich Text. Prosemirror seems like it could have a promising future but is still very early in development.

Having said that I think in the short term just having something that is merely better than last write wins would be a step in the right direction. Maybe some type of automatic locking mechanism. Maybe new version exists notifications with the option to diff and merge. Maybe some combination.

Ideas?

1 Like
2

This has been discussed for a while, since the first issue opened on ShareJS: https://github.com/share/ShareJS/issues/1

I think there we concluded that some integration of ShareJS and QuillJS would work well, although no one has taken a stab at that yet, that I know of.

3

I feel your pain. I’ve been working on this exact problem for awhile now. As @mizzao mentioned sharejs with quill would be amazing, but it doesn’t sound like it’s ever coming (or anytime soon anyways) so you should take a look at sharedb. Sharejs was split into separate components and sharedb was the result of that. Sharejs is aimed at collaborative editing of plaintext (there’s a google group post somewhere in which the author explains the split), sharedb is for keeping any javascript object in sync with multiple users editing it simultaneously. There’s also the Derby framework’s Racer library, which users sharedb to create data models and keep them in sync over the network, you can use Racer or just user sharedb directly.

My solution right now is to have a WYSIWYG editor that uses a model similar to what’s described in this post by the medium.com team https://medium.com/medium-eng/why-contenteditable-is-terrible-122d8a40e480 (see the “The Medium Editor Model” section). So the editor uses an internal JS data structure to represent the state of the editor, with a React front end that can render that data structure into HTML. Then you can use sharedb to keep that JS data structure in sync and handle edit operations by end user. So as users change the data model, React re-renders the appropriate component with the new data.

If I can figure out how to integrate sharedb nicely with Meteor I’d like to publish a package for it, OT in Meteor is something the framework needs really badly. You’d think with the creator of Etherpad being on the Meteor team they’d have integrated something already. This is something I need to build for a project so eventually I will make it work regardless of the solution, and after looking around online it looks like I’m pretty much alone in this venture, so when it’s done I’ll be sharing as much as I can with the community.

1 Like
4

Yeah, I’ve been following those threads for a long time. As you allude to and @efrancis mentions, basically no progress has been made. But like I said, those type of solutions are the holy grail. I still actively wonder what is the next best alternative that could be implemented in Meteor?

I started by imagining a simple reactive locking mechanism. That alone could get you 60-70% of the way towards making multi-editor document editing safe. It does seem to quickly devolve from their into more complex solution to handle the remaining cases:

  • What if an editor who has obtained an automatic lock loses their internet connection?
    • Does the lock remain? (maybe other users now see the lock as being abandoned?)
    • Can others steal an abandoned lock?
    • Can others resume a draft in-progress draft that has been autosaved?
  • What if they regain the connection and someone else is actively in the process of resumed their draft has discarded it and is actively working on their own edit?
    • ???
  • What if they regain the connection and someone has saved a new version?
    • diff and merge?
5

yeah I’ve thought about other solutions but because of those issues you listed, I think OT is pretty much the only sensible and bulletproof approach

6

We tried a very simple test version of editable documents - using contentEditable textarea (so no formatting or anything clever) - trying to resolve a number of issues

  • locking a document for user
  • allowing private editing
  • try to avoid race conditions

we used a lock field on the doc which was set as null until a user chose to edit it, by selecting an edit button, the event would call a method to set the editing user’s id (we tested again in the method if the lock was still unlocked, to avoid possible race condition) and field privateEditing to true

if the loc === userId then a editor wrapper template was displayed, with checkbox for private editing (which was default to true).

then we’d set ready a document for editing (id in a reactive var on the wrapper)

if privateEditing === true , we’d clone the document - but still leaving the original document in a locked state (so visible, but uneditable) and put the clone ID into the reactive var

if privateEditing === false, we’d just use the original ID into the reactive var

and a subtemplate would be given the doc based on the id in the reactive var to edit…

when editing was complete, privateEditing === true, we’d merge the edited clone document back into the original (merge -> update)…

then we’d set the lock back to null

it was a quick hack but it did give us

  • app level lockable documents
  • private editing
  • editing with reactive views

IIRC we got in a bit of a tangle if the editor changed the privacy setting mid while in the editing state, and didn’t resolve it as we moved on

Further back I also made a [really basic meteorpad][1] which again was an attempt app level locking. The logic is not closed at all and there are lots of trapdoors, but it was to try and see where the terrain was

We did a thinking session on a WYSIWYG reactive editor too, but it got messy at the level of atomisation

  • should each <p> have it’s own doc?
  • manging changes to document flow (this paragraph move to before that para)
  • etc

We ended agreed it was a really interesting challenge but closed the session and went to the pub instead
[1]: http://meteorpad.com/pad/vxo4v6ac2MKMx87tc/App%20Level%20Locking

1 Like
7

There is a demo of a simple per-paragraph lock mechanism here.

8

Is the code open - repo link? ?

9

I am not ready for opening the code yet. I use a similar pattern than the one you have just described: there is a lock field in each subsection, to keep track of who is locking it:

Meteor.methods({
  'Subsecs.lock': function(subsecId) {
    check(subsecId, IdPattern);

    // It is useless to lock on client, as it would give user the false
    // impression that the lock operation has succeeded (through latency
    // compensation). The lock operation can fail, for example if 2 users try
    // to lock the same subsec at about the same time.
    // We could at least execute the initial perm check on client, but we don't.
    // The reason is that we call 'Subsecs.lock' in a focus event, in which
    // any heavy operation (such as database access) will result in making
    // the editable area less responsive (especially when selecting text)
    if (Meteor.isClient)
      return;

    // Get the subsec
    var subsec = Subsecs.findOne(subsecId, { fields: { ..., lock: 1 } });
    if (!subsec)
      throw new Error('subsec not found');

    //### Check perm ###
    ...

    // Check if subsec is already locked by someone else
    // This might happen, in case 2 users lock the same subsec at about the
    // same time
    // NOTE THAT WE ALLOW A USER TO RE-LOCK THE SAME SUBSEC, hence extending
    // the lock time
    if (subsec.lock && subsec.lock.userId !== this.userId)
      return false;

    // Get the user active profile
    var profile = ...

    // Lock the subsec
    var lock = {
      id     : Random.id(),
      userId : this.userId,
      profile: profile
    };
    Subsecs.update(subsecId, { $set: { lock: lock } });

    // Set unlock operation in the future
    Meteor.setTimeout(function() {
      Subsecs.update(
          { _id: subsecId, 'lock.id': lock.id },
          { $unset: { lock: '' } }
      );
    }, Subsecs.lockExpirationDelay);

    return true;
  }
});

The lock expiration delay is set to 10 minutes and is enforced client-side: after 10 minutes minus epsilon, client will save content, unlock, and force the contenteditable to blur.

10

Fair enough. Thanks for the response.

11

I just looked into Quill and ShareJS, and it looks like there’s support for ottype so the deltas should be usable by sharejs: http://quilljs.com/blog/a-new-delta/

Not tried it yet, but that’s the road I would go down

1 Like
12

@garrilla did you make a judgement on how the system should react if the editing user

  • drops his/her connection to the application (loses internet connection, laptop goes dead, or simply closes the browser tab)
  • edits a page but then fails to return the lock (walks away for the day, forget about the tab, etc…)

@sylvain looks like you did make a call on the latter, but how about the former?

13

@funkyeah no, we didn’t get that far… I’d probably drop it into a onClose() callback on the server and combine with some ssome kind of timer as per @sylvain’s solution - you don’t want a user who drops a connection momentarily to be closed out

nb - but this makes me aware that the public view of editing, which we were doing reactively, could drop into a hole if the user lost their conncetion for longer than the timeaway an dthey hadn’t finisehd editing and we’re midway through a senta

[delib but weak joke there]

14

If the client doesn’t unlock or re-lock the content within 10 minutes, the server will force the unlock. It is the responsibility of the client to behave accordingly in order not to lose any edited content.

Consequently:

  • If the user walks away for the day, the client can wait 9 minutes then save the edited content and unlock (this is what I implemented).
  • If the client looses the connection, there is not much it can do, except maybe saving the edited text locally and try to merge it later when the connection is back (didn’t implement this).
15

I started using Trix and I really like it. I have not yet used in a collaborative fashion yet, though. You can see the package here.

16

Trix looks really great :+1: thanks for sharing

17

Quill, Trix, Squire, Prosemirror all branch away from contenteditable and towards having their own internal document structures. Of these quill and prosemirror seem most suitable for getting to true real-time collaboration. Without the real-time component they and pretty much all other editor options are faced with the same hurdles when trying to allow for collaboration safety.

I think @garrilla and @sylvain have articulated some of the best options for ensuring this safety.

The list of things to address as I see it:

  • locking with server-side arbitration
    • needs to be done on a per connection basis and not a per-user basis since multiple connections from the same user would still effectively require real-time collaborative editing
  • client no-activity timeout - if the client walks away for too long treat the lock as abandoned - this could be handled in a few different ways
    • if no edits have been made just force the release of the lock and exit the client from editing mode
    • if edits have been made then you could
      • automatically save the client changes and release the lock
      • treat the lock as abandoned - wait until someone else requests the lock to force save and/or allow new editor to decide if they want to steal the lock and/or continue the draft
  • implement drafts and micro-saving
  • when editing a document a draft for that document is created
  • every X amount of time the draft is saved to the server
  • if editor loses connection draft can be resumed:
    • by themselves on re-connection
    • by themselves on another PC
    • by another editor
  • implement conflict resolution
  • full saving of the document can only be done if the draft version is one in front of the main document otherwise need to conflict resolve
    • automatic merge - most difficult
    • manual merge with visual diff - difficult
    • ask to overwrite - easy peasy

Other Items:
May need to be careful if document permissions can change on the fly

18

We’ve used the https://github.com/ottypes/rich-text lib (@sam mentioned) to compose and transform deltas against each other (including the document) successfully. To add liveness, subscribe to a collection of deltas and make sure you know which ones you’ve applied locally so you can transform and apply the new ones coming in before you apply them to the doc.

19

I’ve been thinking about this approach for awhile (subscribing to a collection of deltas). are you purely just listening for new deltas and applying them? it seems like things could get out of sync if another user had a version with one or two less deltas applied than anothers because of a slower connection and that user submits a delta when their local version is slightly behind. delta objets don’t seem to have a “version” or some sort of increment value so how do you know which delta’s you’ve applied locally and which you haven’t? I’d like to hear whatever insights you’ve found while playing around with this because it seems like it could be a solid approach and entirely avoid ShareJS’s OT

20

Was there any conclusion to this? What is currently the best way to do collaborative editing?