Real-Time Collaboration Under the Hood: CRDTs, OT, and Presence at Scale

Real-time collaboration feels simple to users: two people type, changes appear, cursors move. Under the hood, it is a distributed systems problem with concurrency, ordering, conflict resolution, presence, offline support, security, and scale. This article walks through the architecture of collaborative apps, compares operational transformation (OT) and conflict-free replicated data types (CRDTs), and shows how to combine transport, presence, durability, and access control into a production-ready design.

Why Real-Time Collaboration Is Hard

Every collaborative session is a small distributed system. Clients have different clocks, networks drop messages, users go offline, and multiple people edit the same object at the same time. The system must decide what the document looks like without blocking every keystroke on a round trip to a central server.

• Concurrency: Two users can edit the same paragraph, cell, or shape at the same moment. The system must merge those edits without losing intent or creating impossible states.
• Ordering: Messages can arrive out of order. If user A inserts a word and user B deletes the surrounding sentence, applying those operations in the wrong order can corrupt the document.
• Causality: An edit often depends on a previous edit. The system must preserve causal relationships so a reply does not appear before the message it replies to.
• Conflict resolution: Some conflicts are deterministic (last-writer-wins on a title), while others need intent preservation (keeping both concurrent inserts in a text stream).
• Presence: Cursors, selections, and typing indicators are high-frequency, ephemeral signals. They should not pollute the durable document history or force a full sync.
• Offline and recovery: Users expect to keep working on a plane, then reconnect and merge changes. The sync protocol must handle long disconnections, partial state, and duplicate updates.

The Consistency Spectrum

There is no single consistency model for collaboration. The right choice depends on the data, the user experience, and the deployment topology. Most systems sit somewhere between strong consistency and eventual consistency.

Strong Consistency

Strong consistency makes every client see the same order of operations, often through a central server or consensus protocol. This is simple to reason about and works well for authoritative workflows such as billing, permissions, and final document commits. The trade-off is latency: every edit may need a round trip before it is visible, which breaks the local-first feel users expect from real-time editors.

Eventual Consistency

Eventual consistency lets replicas diverge temporarily and converge later. This is the foundation of CRDTs and many offline-first systems. Users can edit immediately, and the system reconciles in the background. The challenge is designing merge rules that produce a predictable, useful result and keeping metadata from growing without bound.

Local-First and Causal Consistency

Local-first software treats the local device as the primary source of truth and syncs opportunistically. Causal consistency preserves the order of related operations without requiring a total order for everything. This is a sweet spot for collaboration: a user sees their own edits instantly, sees others’ edits in a causally consistent order, and can merge offline work when connectivity returns.

Operational Transformation (OT)

OT was popularized by Google Docs and remains a strong choice for text-heavy editors with a central authority. It represents each change as an operation, such as insert, delete, or format, and transforms concurrent operations so they can be applied in a consistent order.

For example, if two users insert a character at the same position, the server can order the operations. The second operation is transformed against the first: if the first insert added one character before the second insert position, the second insert index is shifted by one. The result is a document that reflects both edits without duplication or corruption.

• Strengths: Compact operations, low bandwidth for text, mature algorithms, and a central server that can enforce ordering and access control.
• Weaknesses: Transformation functions are notoriously complex, edge cases multiply with operation types, offline support is difficult, and the central server can become a bottleneck or single point of failure.

OT is a good fit when you have a central authority, text-heavy editing, and limited offline requirements. It is less attractive when you need peer-to-peer sync, rich data types, or long offline sessions.

Conflict-Free Replicated Data Types (CRDTs)

CRDTs make concurrent updates commutative, associative, and idempotent. Replicas can apply updates in any order and converge to the same state. This property enables local-first apps, offline editing, peer-to-peer collaboration, and multi-server sync without a single authoritative coordinator.

State-Based vs. Operation-Based CRDTs

• State-based (CvRDT): Replicas exchange full or delta state, and a merge function combines states. The merge must be commutative, associative, and idempotent. State-based CRDTs are simple to reason about, but full-state exchange can waste bandwidth. Delta-state CRDTs send only changes and are more practical.
• Operation-based (CmRDT): Replicas exchange operations and apply them locally. Operation-based CRDTs are compact, but the delivery layer must provide causal order or the operations themselves must be commutative. This often requires a reliable broadcast or a sync protocol with version vectors.

Common CRDT Building Blocks

• G-Counter: A grow-only counter. Each replica increments its own slot, and the value is the sum of all slots. Merge takes the maximum value per slot. It is useful for likes, view counts, and other monotonic metrics.
• PN-Counter: Two G-Counters, one for increments and one for decrements. The value is increments minus decrements. It supports both directions while remaining conflict-free.
• OR-Set: An observed-remove set. Each add carries a unique tag, and a remove only removes tags that the remover has observed. Concurrent adds and removes converge to a set that includes elements added after the remove. This is useful for tags, participants, and collections.
• Sequence CRDTs: RGA, Logoot, LSEQ, Yjs, and Automerge assign stable identifiers to items so concurrent inserts can be ordered deterministically. They power collaborative text, lists, and rich outlines. The trade-off is metadata overhead and the need for compaction.
• Map and JSON CRDTs: Nested structures with conflict resolution policies such as last-writer-wins, multi-value, or custom merge functions. They let you model documents, settings, and application state without building every type from scratch.

OT vs. CRDTs: How to Choose

In practice, many products use a hybrid: CRDTs for offline-capable documents and server transforms for high-frequency text streams, or a server-authoritative CRDT sync service. The important decision is where ordering and conflict resolution happen, and how much metadata you are willing to store and transmit.

Transport and Sync Architecture

The sync algorithm is only half the system. The transport layer must deliver updates reliably, handle reconnects, and scale to many rooms and users. It also determines latency, battery usage, and how well the app works on unreliable networks.

WebSocket, WebRTC, and Fallbacks

• WebSocket: The default for client-server sync. It offers low overhead, full-duplex communication, and broad compatibility. Use TLS (wss://) and handle reconnection with exponential backoff and jitter.
• WebRTC Data Channels: Peer-to-peer, low latency, and a good fit for small groups or local-first mesh topologies. NAT traversal, signaling, and peer discovery add complexity, and mobile networks can still be hostile.
• Server-Sent Events (SSE): One-way server push that pairs with HTTP POST for client updates. It is simpler than WebSocket and works through most proxies, but it lacks full-duplex communication.
• HTTP polling: A fallback for restrictive networks. Long polling or delta queries reduce load, but polling is still less efficient than a persistent connection.

Scaling the Fan-Out Layer

Once a room has hundreds or thousands of participants, the server that holds a connection is not necessarily the server that should process every update. A fan-out layer distributes messages between nodes and keeps rooms available across regions.

• Room sharding: Assign each document or room to a shard. A consistent hash ring or directory service maps room IDs to nodes. This limits the blast radius of hot rooms and simplifies capacity planning.
• Pub/sub backbone: Use Redis Streams, NATS, Kafka, or a cloud pub/sub service to broadcast updates between application nodes. The pub/sub layer should be durable enough for critical updates but not a bottleneck for ephemeral presence messages.
• Sticky sessions: Route a client to the same node when possible, but do not rely on stickiness for correctness. The sync protocol should tolerate reconnects to a different node.
• Backpressure: Clients can produce updates faster than the network can send them. Buffer at the edge, batch small updates, and drop or coalesce non-critical messages such as cursor moves.
• Message batching: Group multiple operations into a single frame to reduce overhead. For text editing, send deltas at a steady cadence rather than on every keystroke.
• Regional routing: Place sync nodes close to users and replicate durable state across regions. Use conflict-free merges or a consensus layer for metadata such as permissions.

Presence and Awareness

Presence is ephemeral state: who is online, cursor position, selection, typing indicator, and viewport. It should not be persisted in the document history or mixed with durable CRDT updates. Mixing the two forces unnecessary writes, bloats history, and makes privacy harder to control.

• Separate channel: Send presence over a different topic, room, or message type. Presence can use a last-write-wins map with short time-to-live values, while document updates use the CRDT or OT protocol.
• Heartbeats: Each client sends a heartbeat every few seconds. The server marks a participant offline after a timeout. Heartbeats are small, but they add up in large rooms, so use adaptive intervals and jitter.
• Throttling: Cursor moves can fire dozens of times per second. Throttle or debounce them to 10-20 Hz, interpolate on the client, and send only meaningful changes such as selection updates.
• Privacy: Presence can reveal who is viewing a document and when. Give users controls to hide activity, and avoid storing presence longer than necessary. If you use end-to-end encryption, presence metadata may still leak information unless you design it carefully.

A common pattern is a presence CRDT or a last-write-wins map with short TTLs. Each client publishes its state with a monotonically increasing timestamp and a session ID. The server broadcasts changes and expires stale entries. When a user reconnects, they receive the current presence state and merge it with their local view.

Durability, Storage, and History

Real-time collaboration must survive crashes, reconnects, and long offline periods. Durability is not just writing the final document to a database; it is preserving enough history and metadata to reconstruct state and merge future updates.

• Append-only update log: Store every operation or CRDT update in an ordered log. This log is the source of truth for incremental sync and audit. Use a durable write-ahead log or a database with append-only tables.
• Snapshots: Periodically materialize the full document state. Snapshots speed up cold starts and reduce the number of updates a new client must replay.
• Compaction: Remove tombstones, merge redundant operations, and discard history that is no longer needed. Compaction keeps metadata bounded and improves performance, but it must be coordinated so replicas do not lose necessary causal information.
• Version vectors: Track which updates each replica has seen. Version vectors let peers compute missing updates efficiently and avoid sending duplicates. They are essential for multi-device sync and server-to-server replication.
• Event sourcing: Treat all changes as events and derive current state from them. This fits naturally with collaboration logs and makes it easier to add features such as undo, replay, and analytics.

For CRDTs, store both updates and periodic snapshots. Updates allow incremental sync; snapshots speed up cold starts. Compaction removes tombstones and redundant operations. Version vectors or state vectors let peers compute missing updates efficiently. The exact retention policy depends on your product: some apps keep full history, while others keep only the latest snapshot plus recent updates.

Security and Access Control

Real-time collaboration expands the attack surface. Every update is a potential injection vector, and access control must be enforced at the document, room, and operation level. A sync server that trusts clients to send valid state is a security incident waiting to happen.

• Authentication: Verify user identity with tokens, mutual TLS, or session cookies. Short-lived tokens and refresh flows reduce the impact of leaked credentials.
• Authorization: Check permissions before joining a room and before applying updates. A user may be allowed to read a document but not write to certain fields. Enforce this on the server, even if the client UI hides controls.
• End-to-end encryption: If the server must not read document contents, encrypt updates on the client and share keys only with authorized participants. This complicates server-side validation, search, and moderation. Design permissions so that key distribution matches document access.
• Data validation: Treat client updates as untrusted input. Validate lengths, types, and positions to prevent denial-of-service and memory exhaustion. For CRDTs, enforce limits on metadata and tombstone growth.
• Audit logs: Record who changed what and when. Audit logs help with compliance, incident response, and user-facing version history. Store them separately from ephemeral presence data.
• Rate limiting: Limit connections, updates, and presence messages per user and per room. Use quotas and anomaly detection to stop abuse without punishing normal collaboration.

An Implementation Blueprint

1. Choose a consistency model: Decide whether the app needs strong consistency for some fields and eventual consistency for others. Most collaborative documents can be eventually consistent, while permissions and billing should be strongly consistent.
2. Pick a sync algorithm: Use OT for text-heavy, server-authoritative editing. Use CRDTs for offline-first, multi-device, and rich data. Consider a hybrid if you need both.
3. Design the data model: Separate durable document state from ephemeral presence. Model text, lists, maps, and counters with appropriate CRDT types or OT operations.
4. Build the transport layer: Start with WebSocket and a pub/sub backbone. Add WebRTC or SSE only if the use case demands it. Handle reconnects, batching, and backpressure from day one.
5. Separate presence from documents: Use a lightweight presence protocol with heartbeats and TTLs. Never write cursor positions to the durable log.
6. Add durability: Persist updates and snapshots. Implement version vectors and a compaction job. Test recovery from partial writes and corrupted snapshots.
7. Implement security: Authenticate, authorize, validate, and rate limit. Add end-to-end encryption if the threat model requires it, and accept the trade-offs.
8. Instrument and test: Add metrics, tracing, and replay. Test with simulated latency, packet loss, offline periods, and concurrent edits before you ship.

Minimal CRDT Sync Sketch

// Client-side sketch with a CRDT library such as Yjs
const doc = new Y.Doc()
const text = doc.getText('content')

text.observe(event => {
  render(text.toString())
})

const ws = new WebSocket('wss://collab.example.com/room/123')

ws.onmessage = event => {
  const update = new Uint8Array(event.data)
  Y.applyUpdate(doc, update)
}

doc.on('update', update => {
  ws.send(update)
})

On the server, authenticate the connection, join the room, persist updates, broadcast to other clients, and send the current state vector so reconnecting clients can request missing updates. The exact protocol depends on your CRDT library, but the shape is the same: authenticate, join, sync state, stream deltas, persist snapshots.

Testing and Observability

Collaboration bugs are often race conditions that only appear under specific network conditions and edit orders. Testing must go beyond unit tests for merge functions.

• Property-based testing: Generate random operations and apply them in different orders to different replicas. Assert that all replicas converge to the same state.
• Network simulation: Use tools such as Toxiproxy, tc, or a custom test harness to inject latency, packet loss, reordering, and partitions. Verify that the app recovers and merges correctly.
• Conflict fuzzing: Focus on adversarial edit patterns: concurrent inserts at the same position, delete-insert races, undo during sync, and reconnection after long offline periods.
• Metrics: Track sync latency, update size, conflict rate, merge time, reconnect rate, and metadata growth. Alert on spikes that indicate a hot room or a compaction failure.
• Distributed tracing: Trace an update from client to server to peers. Include room ID, user ID, operation ID, and version vector in spans so you can debug causality and ordering issues.
• Replay: Store update logs so you can replay a session and reproduce bugs. Replay is also useful for load testing and for validating new merge algorithms against real traffic.

Common Pitfalls

• Mixing presence with durable state: This bloats history, leaks privacy, and forces unnecessary sync. Keep presence ephemeral.
• Ignoring backpressure: A fast typist or a misbehaving client can overwhelm the server and other clients. Batch, throttle, and drop non-critical messages.
• Underestimating metadata: CRDTs can accumulate tombstones and identifiers. Plan for compaction before your document reaches millions of operations.
• Weak offline recovery: Reconnecting clients may need a large missing update set. Use version vectors and snapshots to make recovery fast and bounded.
• No access control at the sync layer: Hiding a button in the UI is not authorization. Enforce permissions on every update and every room join.
• Testing only happy paths: Real collaboration is messy. Simulate packet loss, duplicate messages, out-of-order delivery, and concurrent destructive edits.

Conclusion

Real-time collaboration is a stack, not a feature. The sync algorithm, transport, presence, durability, security, and observability all shape the user experience. OT is a proven choice for server-authoritative text editing, while CRDTs unlock local-first, offline-capable, multi-device apps. In many production systems, the best answer is a hybrid: use the right consistency model for each piece of data, separate ephemeral presence from durable documents, and design for failure, reconnection, and scale from the beginning. When you get those foundations right, real-time collaboration feels effortless to users, even though the machinery underneath is doing serious distributed systems work.