Local-First Software: CRDTs, Sync Engines, and the End of the Loading Spinner

For two decades, the default answer to the question of where data lives has been: someone else’s computer. Thin clients, thick clouds, every keystroke round-tripping to a region that might be 200 milliseconds away. It works, right up until it does not — a flaky train connection, a saturated API gateway, a regional outage, or simply a user who expects an app to respond instantly because every other app on their phone does.

Local-first software inverts that default. The primary copy of the user’s data lives on the user’s device, in a real database, and the network is used to synchronize replicas rather than to gate every interaction. The result is software that feels instant, works offline, keeps functioning when the backend is degraded, and — with the right design — still supports real-time multiplayer collaboration.

The catch: local-first is not a rendering strategy or a clever cache. It is a distributed systems problem wearing a friendly user interface. This article walks through the architecture — how replicas converge, where CRDTs fit, what a sync engine actually does, and where the sharp edges are hiding.

What Local-First Actually Means

The term was popularized in a 2019 essay by Martin Kleppmann and colleagues, who enumerated seven ideals for this class of application:

• No spinners. Reads and writes hit local storage, so interactions are bounded by disk and CPU latency, not by network round trips.
• Multi-device. The same user gets the same data on laptop, phone, and tablet, with changes propagating automatically.
• Offline-capable. The app is fully functional with no connectivity, for minutes or for weeks.
• Collaborative. Multiple users can edit the same content concurrently, as in a shared document.
• Longevity. The data outlives the vendor. Files on disk and open formats survive; an abandoned SaaS backend does not.
• Privacy by default. End-to-end encryption becomes practical when the server is not the authority on every mutation.
• User ownership. The user can inspect, export, back up, and delete their own data without asking permission.

Most products today satisfy none of these. A typical CRUD app is a thin view over a remote database: every list render is a network call, every form submit is a mutation request, and offline mode is a polite error page. Local-first moves the authority boundary to the device and treats the server as one replica among many — an important one, but not a privileged one for read latency.

The Hard Part Is Not Offline. It Is Convergence.

Making an app work offline is straightforward: write to a local store and queue the mutations. The difficulty appears the moment two replicas make conflicting changes and later have to agree on a single state.

Consider a shared note. Alice’s laptop and Bob’s phone both start from the same revision. While partitioned, Alice appends a paragraph and Bob inserts a sentence in the middle of the first section. When connectivity returns, what is the correct result?

Naive strategies fail in predictable ways:

• Last write wins at the record level. One entire edit is silently discarded. Users experience this as data loss, and they are right.
• Server-side merge with field-level LWW. Better, but it still loses concurrent edits to the same field and produces non-deterministic results if replicas apply merges in different orders.
• Manual conflict resolution. Acceptable for rarely edited documents, catastrophic for real-time collaboration with hundreds of edits per minute.

What you actually want is a merge function with three mathematical properties: commutative (order of merges does not matter), associative (grouping does not matter), and idempotent (merging the same update twice is a no-op). Any data structure with those properties forms a join-semilattice, and replicas that exchange updates eventually reach the same state regardless of message order, duplication, or delay. That is the entire promise of CRDTs.

CRDTs: Convergence Without Coordination

Conflict-free replicated data types come in two broad flavors. Operation-based CRDTs (commutative replicated data types) broadcast individual operations and require reliable, causally ordered delivery. State-based CRDTs ship the whole state and merge it; they tolerate lossy, unordered, duplicated delivery because the merge is idempotent. In practice, most production systems use a hybrid: delta-state CRDTs, which transmit compact deltas that can be merged into any replica’s state.

The Canonical Data Types

• G-Counter and PN-Counter. A grow-only counter stores a per-replica count and sums them on merge. A PN-Counter pairs two G-Counters to support decrements. Simple, small, and useful for likes, view counts, and quota tracking.
• LWW-Register. A single value with a timestamp; the highest timestamp wins. Convenient, but the clock is the conflict resolution policy, so clock skew and ties become correctness concerns. Hybrid logical clocks mitigate this by combining physical time with a logical counter.
• OR-Set (observed-remove set). Each element is tagged with a unique identifier on add; removal deletes only the tags the remover has observed. This is what makes add wins over concurrent remove the default and correct behavior for most collaborative collections.
• Sequence CRDTs. The hard problem. Text needs a stable position for every character. RGA, YATA (the algorithm behind Yjs), and Fugue each solve this with different trade-offs in metadata size, memory locality, and worst-case interleaving behavior. This is where naive implementations produce the infamous tangled-text bug where concurrent insertions interleave character by character.
• JSON and rich-text CRDTs. Automerge models documents as nested maps, lists, and text objects, and adds rich-text semantics on top with marks that carry their own conflict behavior. Peritext-style approaches handle overlapping and nested formatting spans more gracefully than treating formatting as plain attributes.

OT Versus CRDT: An Honest Comparison

OT is not obsolete. If you have a trusted central server, a bounded set of document types, and a strong reason to minimize metadata — such as a text editor at massive scale — a well-tested OT implementation is a legitimate choice. But for general application state that must survive partitions, CRDTs win on composability.

The Real Costs of CRDTs

Convergence is not free, and the costs are architectural, not incidental:

• Tombstones. Deletions leave markers so that a later-arriving concurrent insert is not resurrected. Uncollected, tombstones dominate storage in delete-heavy workloads.
• Identifier growth. Every list element and every set member carries a globally unique identifier, often 16 bytes or more. A document with a million characters can carry tens of megabytes of metadata before compression.
• Causal context. Version vectors and deleted-set summaries grow with the number of replicas. Systems with many short-lived replicas need compaction and state-vector summarization.
• Garbage collection is a protocol. Safe tombstone collection requires knowing that every replica has observed a deletion. That means exchanging stability information, which is one more distributed agreement problem.

Modern libraries attack these directly with columnar binary encodings, run-length compressed tombstones, delta compression over the wire, and periodic snapshotting. Still, sizing your CRDT metadata budget before launch is a design decision, not an optimization you defer.

Storage: SQLite Ate the Client

The second enabling shift is that a real relational database now runs everywhere a user is. SQLite is already on every phone and in every browser stack worth targeting, and the WebAssembly build plus the Origin Private File System gives browser apps a durable, transactional, indexed store with synchronous reads.

This matters because it decouples the query model from the sync model. Users get SQL — joins, indexes, aggregations, full-text search — against local data, while the sync layer moves CRDT deltas or change logs behind that abstraction. A common pattern is a three-layer stack:

1. Storage layer. SQLite (native or WASM) holding materialized tables for fast queries.
2. Log layer. An append-only operation log or CRDT document store that is the source of truth for synchronization.
3. Sync layer. The engine that pushes local operations, pulls remote ones, and applies merges.

Keeping the materialized view separate from the log is what lets you re-derive state after a buggy merge, replay history for debugging, and change sync semantics without rewriting every query in the app.

Anatomy of a Sync Engine

The Write Path

A local mutation follows a strict sequence, and the ordering is what guarantees durability:

1. Apply the change optimistically to the local store so the UI updates in the same frame.
2. Append the operation to a durable log, in the same transaction where possible. If the app crashes here, the change must survive.
3. Enqueue the operation for transmission in an outbox with a monotonically increasing sequence number.
4. Push to the server, with retry and exponential backoff.
5. On acknowledgement, advance the client’s replication cursor and release the outbox entry.

The crucial detail is that step 2 and step 3 must be atomic with respect to step 1. If a write is visible in the UI but not in the log, a crash produces a change that will never sync — the worst kind of bug, because it is silent and unreproducible.

The Read Path

Local reads are just queries, but the UI still needs to know when to re-render. Two approaches dominate:

• Observation and invalidation. Queries register the tables or documents they touch; when a sync delta mutates those, dependent queries are invalidated and re-run. Simple, predictable, and adequate for most apps.
• Incremental view maintenance. The engine computes the delta to query results rather than recomputing. Much faster for large result sets, considerably harder to implement correctly across joins and aggregates.

Either way, treat local reads as synchronous and fast. If your local query path involves an await on the network, you have not built a local-first app; you have built a cache with extra steps.

The Server’s Job

Even in a peer-to-peer model, a server earns its place by providing: durable backup, ordering for those who need it, authorization enforcement, fan-out to large user sets, history compaction, and initial backfill for new devices. The design question is not whether the server is authoritative for reads — it is not — but whether it is authoritative for authorization and durability. It should be both.

Pull Strategies and Clocks

Naive polling by timestamp breaks with clock skew and with writes arriving out of order. Robust engines use one of:

• Server-assigned monotonic sequence numbers per sync scope, with clients tracking the highest contiguous sequence applied.
• Hybrid logical clocks that combine physical time with a logical counter, giving a total order that is causally consistent and roughly wall-clock aligned.
• Version vectors per replica, which detect concurrent changes precisely but cost space proportional to replica count.

Whatever the choice, the client must be able to detect gaps. A client that applies sequence 41 without having applied 40 has a silent correctness bug that may not surface for weeks.

Authorization Is the Sharpest Edge

This is where local-first designs most often break, and it deserves more attention than it usually gets.

If the client holds a full replica, then the client holds data. Server-side filtering at read time solves nothing if the data already reached the device. Conversely, if the server filters what it replicates, then each client holds a partial replica, and partial replicas interact badly with CRDT merge semantics: merging a document you can only partially see can produce a result that violates the permission model, or that a later-authorized client interprets differently.

Practical approaches that hold up:

• Partition by sync scope. Define replication units (workspace, project, document) and grant access at that granularity. Within a scope, everyone sees everything. This keeps CRDT merge sound because all replicas of a unit converge to the same set of operations.
• Server-side authorization on every push. The server validates each operation against current permissions before accepting it, and rejects rather than silently drops — rejected operations go to a dead-letter queue the client can surface.
• Per-scope encryption keys. Encrypt each sync scope with a key distributed only to authorized members. Key rotation on revocation is the hard part; a removed member keeps everything they already decrypted, so revocation is inherently forward-only.
• Capability tokens with short lifetimes. Bound the window in which a revoked client can still push operations.

Two honest constraints: end-to-end encryption and server-side conflict resolution are in tension, because a server that cannot read operations cannot merge them on the client’s behalf. And permission changes are not CRDT operations — they must be serialized through an authority, which means an authorization change is not available offline in the general case.

Ephemeral State Deserves Its Own Channel

Cursors, presence indicators, typing notifications, and selection highlights are produced at high frequency, are meaningless after a few seconds, and must never be persisted. Putting them in the CRDT is a classic mistake: it bloats the operation log with tombstones no one needs and makes the document history unreadable.

Run ephemeral state over a separate, unreliable, unordered transport — typically a WebSocket with a heartbeat and a TTL. Each participant publishes a presence record that expires automatically; no explicit cleanup message is required, because absence of a heartbeat is the removal signal. Keep the durable CRDT channel for anything the user would be upset to lose, and the ephemeral channel for everything they would not notice going missing.

Schema Evolution in a Multi-Version World

In a client-server CRUD app, you deploy the server and the schema changes everywhere at once. In local-first, clients lag — sometimes by months. A user on a six-month-old app version is a permanent resident of your data model.

Rules that keep this tractable:

• Additive changes only, by default. New optional fields are safe. Removing or repurposing a field requires a migration period spanning at least the longest supported client lifetime.
• Never let an old client write a shape it does not understand. If a new field carries semantics that old clients would corrupt, gate the whole document behind a version marker and refuse sync from clients below the threshold.
• Keep migrations pure and total. A migration from version N to N+1 must be a deterministic function with no side effects and no network access, so it can run on any replica at any time and produce the same result.
• Store schema version in the document or sync scope, not only in the client binary. The data needs to describe itself.
• Treat rollback as a real scenario. If a deploy introduces a schema change and rolls back, data written by the newer version must still be readable. That usually means unknown fields are preserved, not dropped.

Testing Convergence Is Not Optional

Convergence bugs are the local-first equivalent of race conditions in threaded code: rarely reproduced, impossible to reason about from a stack trace, and utterly corrosive to user trust. The only reliable defense is a deterministic simulation harness.

A workable harness has four parts:

1. A virtual clock and virtual network. Latency, jitter, reordering, duplication, and partitions become parameters you control, not conditions you hope for.
2. N in-memory replicas running the exact sync code used in production, driven by a seeded PRNG.
3. A property-based operation generator that emits random mutation sequences with a bias toward concurrency: same element inserted by two replicas, simultaneous delete and update, rapid create-then-delete.
4. An invariant checker that asserts, after delivering all messages in a random order, that every replica’s state is byte-identical, and that the merge order was irrelevant by replaying the whole scenario with shuffled delivery.

for seed in range(0, 10000):
    net = VirtualNetwork(seed)
    replicas = [Replica(i) for i in range(3)]
    ops = generate_ops(seed, count=500)
    for op in ops:
        r = seed_choice(ops, replicas)
        r.apply_local(op)
        net.deliver_randomly(replicas)   # may reorder, duplicate, drop
    net.flush_all(replicas)
    assert_converged(replicas)           # all states identical
    assert_matches_reference(replicas[0])  # semantic correctness

Run this in CI with a fixed seed corpus plus a rotating seed budget. Every bug it finds becomes a permanent regression case with the offending seed pinned. This style of simulation testing — popularized for distributed databases and now standard in serious sync engines — finds failure modes that no amount of manual QA will surface.

Observability for a Sync Engine

You cannot operate what you cannot see, and sync problems manifest as user complaints about missing data long before they show up in server error rates. Instrument these:

• Replication lag per client: the delta between the server’s head sequence and each client’s applied sequence, tracked as a distribution, not an average.
• Pending outbox depth and age. A client with a persistent queue is a client whose writes are not reaching anyone.
• Rejected operation rate, broken down by reason — authorization, schema violation, quota. Each reason has a different fix.
• Local database size and tombstone ratio per active user, with alerting on the right tail.
• Bytes transferred per session and per operation, to catch encoding regressions after a library upgrade.
• Client-side error telemetry from the merge and apply paths, with the operation log ID attached so a support ticket can be replayed offline.

Also build the operator tooling before you need it: a way to inspect a specific user’s operation log, a way to force a client to re-bootstrap from a snapshot when its local state is suspect, and a way to replay a problematic history against a fixed code version.

Performance and Data Modeling

Local-first performance work concentrates in a few places:

• Subscription granularity. Subscribing at the workspace level means every change wakes every client. Subscribe at the smallest unit that matches how users actually work together.
• Partial replication. Ship only the recent window of history plus a compacted summary of the rest. Most users never open a document from four years ago, and those who do can afford a two-second fetch.
• Batch and compress deltas. One websocket frame carrying fifty merged deltas costs far less than fifty frames, both in bytes and in per-message overhead.
• Avoid hot documents. A single document edited by thousands of users concurrently is a throughput bottleneck regardless of algorithm. Shard the collaboration unit.
• Index the log, not just the tables. Cursor advancement, gap detection, and compaction all query the operation log. Leave it unindexed and you will find out during your first large backfill.

When Local-First Is the Wrong Answer

Local-first is a strong default for notes, design tools, project management, CRM, field data collection, and anything where users create content collaboratively. It is a poor fit when:

• You need global invariants enforced immediately. Unique usernames, inventory decrements, seat counts, and ledger balances require a serialization point. CRDTs converge; they do not enforce uniqueness across a partition.
• The data is inherently server-side and large. Analytics over billions of rows, search across a global corpus, and ML inference over a shared model do not want a device-resident replica.
• Regulatory constraints forbid it. Data residency, mandatory server-side audit logging, and lawful-intercept requirements can make a device-authoritative model non-viable regardless of engineering elegance.
• Nobody uses it offline. If you are building a back-office admin panel used only from office desktops on reliable networks, the local-first machinery is pure overhead.

A Practical Adoption Path

Do not rewrite the whole product. Stage it:

1. Pick one workflow that is read-heavy, latency-sensitive, and genuinely benefits from offline behavior. A single document type or a single mobile surface is usually enough.
2. Choose a data model that fits CRDTs. Prefer append-heavy, order-independent structures: sets, maps, logs, counters. If the domain needs uniqueness, model it as a claim with a deterministic winner rather than assuming mutual exclusion.
3. Adopt an existing sync engine or CRDT library rather than writing your own sequence CRDT. The algorithms are subtle and the failure modes are silent. Build on proven implementations.
4. Run local-first in shadow mode first. Sync in the background, compare the derived state against the current server-of-record, and log every divergence without affecting users. This is the cheapest possible correctness validation.
5. Cut over behind a flag per account, with the ability to force a re-bootstrap from the server if a client’s state is inconsistent.
6. Invest in the simulation harness and observability dashboards before the general rollout, not after the first data-loss report.

Bottom Line

Local-first software is not a trend or a framework choice. It is a commitment to a data model in which replicas are peers, conflicts are resolved by design rather than by luck, and the network is an optimization instead of a dependency. The price is real: CRDT metadata budgets, tombstone collection, partial-replica permission models, multi-version schema discipline, and a simulation-based testing culture.

What you get in return is software that feels instantaneous, that survives outages and airport Wi-Fi, that treats the user’s data as theirs, and that keeps working when your backend does not. For a growing class of applications — collaborative, mobile, latency-sensitive, privacy-conscious — that trade is not merely worth it. It is the correct architecture.