Saving and observing changes on an collection using another collection


I have following challenge: observing changes on a hierarchy (tree - like data) with preserving the order of the children.

What I need to do (initially) is: Get the data -> transform the data (create hierarchy) -> compute layout -> display.

The problem:

I can not transfer the complete data to the client whenever a change on any of the hierarchy node appears, because this would be very inefficient. I can work efficiently with atomic changes like: add, remove, change position …

Other problem: I could not find a way to implement the hierarchy in MongoDB so I can make changes (node addition, removal or position changes…) in one atomic operation. The order matters, so I either have to update the children array reference on the parent, or the “ordering index” on the siblings.

This results to multiple database operation on node creation/deletion, resulting in multiple events on the client. Which causes issues, eg. when you get the information, than a node reference was added to the parent, but the node is not in the database yet.

My solution:

Having a separate collection keeping the atomic changes like: node was added to parent X at position Y or node at position Y was removed from parent X

SoI would fetch the data fro the whole hierarchy once and observe changes on the collection of changes The changes would be defined object storing also the timestamp of the change.


How do I handle the situation, when there were changes on the collection between I started to fetch the complete data and started to observe the collection of changes ?

—cursor.fetch() ----------------------> all data fetched |-start-observing-changes------->