Using true CRDT (for non-document resources)

[joepio]

We've implemented Yjs some time ago for documents #720. But the rest of AtomicServer still kind of assumes that there is one server in control. Even though we now have a local-first setup (using did:ad #1148 #1145) our commits are pretty linear by design. At some point, we're gonna need true conflict resolution. And I would prefer to have a system where we don't have to replay all of history to reconstruct the current state.

Some of my goals:

• Multiplayer where conflicts are resolved automatically
• History, potentially for all transactions / changes
• Signatures for each change
• Option to do all messages encrypted.
• Keep JSON-AD for those who need it
• Sync efficiently, no need to replay all events to reconstruct state

I think we have a couple of paths to achieve this:

Roll our own CRDT, update Atomic Commits

• Takes a lot of time
• Hard to make efficient
• Seems like a bad idea

Use Loro

• Fast!
• Doesnt work with tiptap it actually does
• Rust / Wasm (native on server)

JSON-Joy

• Even faster!!!
• Less document-focused, lower quality interleaving
• TS based

Use Yjs / Yrs for everything

• We already use it for documents it works
• Not very fast, requires history playback

[joepio]

We're using loro now :) working great

[joepio]

Experimented quite a bit:

• Loro seems like a great way to deal with conflict resolution
• It replaces commit methods like .set and .push.
• Reading loro objects is about 150 times slower than reading plain json maps, so we can't just take loro maps and do .get all the time every time something re-renders. We can memoize / cache the json representation though
• loro can replace yjs's realtime text cursor features and it works with tiptap
• loro updates are VERY ligthweight, the history is insanely compressed compared to atomic commits