Dev Learning
Sync and offline

File sync across devices

Design a file sync service. A folder on several devices stays in step, edits made offline sync when the device reconnects, and history is recoverable. Files range from a few kilobytes to several gigabytes.

hardObject storage and presigned uploadsClocks, ordering and why timestamps lieTransactions, isolation and lockingChange data captureCDNsMessage queues and backpressureAuthentication, authorization and secrets

Where’s the slack?

Local edits: zero. Propagation: seconds. Full sync of a new device: hours.

The local device must feel instant even with no network at all. Everything else (reaching other devices, agreeing with the server, resolving conflicts) has real slack. The design is built on that asymmetry.

A user saving a file has zero tolerance. The save writes to local disk and returns, even offline. Sync happens afterwards, in the background. If the product ever makes someone wait for the network to save a file, it has failed at its core job.

Propagation has seconds of slack. If a change reaches another device in three seconds rather than three hundred milliseconds, the user will not notice, because they are rarely watching two devices at once. So propagation can go through a queue and be batched, which matters a lot when someone drops in a folder of 10,000 files.

A new device syncing 200 GB has hours of slack, and the user knows it. What it must not do is block anything else, or start from the beginning if it is interrupted at 90%.

The place with no slack and no way to buy any: not losing an edit. If two devices both edited a file offline and one version silently disappears, that is not a sync delay. It is data loss, and the user will not know it happened.

So: The client is the source of truth for local state and works fully offline. Sync is asynchronous, resumable and chunked. Conflicts are detected and preserved as both versions rather than resolved by picking a timestamp.

Work it through

Five stages, gated. You write your answer for each stage before its model answer unlocks. That is the core of the method.

Loading your progress…

Reference material for this problem. The diagrams below are here to check your own against once you have drawn it, not to read first.

The lifecycle

Edited locallyuser deviceSettled + hashedclient watcherMissing chunks uploadedclientVersion committedserverCursor moved, devices notifiedsystemApplied on other devicesclientsFile still being writtenwait for it to settle, or you upload a corrupt half-fileConnection drops at 90% of 5 GBresume from the last completed chunk — never restartParent version no longer matchesconflict: keep both, as a named conflicted copy. Never pick by timestampDevice offline for a monthpull the delta from its cursor — compact current state, not replayed historyLocal disk full mid-syncfail resumably; never leave a partial file that looks complete50,000 files dropped in at oncebatch hashes, bound parallelism, one cursor advance for the whole batch
A file's life across devices. The failed conditional commit is not an error. It is the mechanism that turns a silent overwrite into a visible conflict.

The numbers

Edit a paragraph in a 2 GB file — no chunking2 GB uploaded
Same edit, 4 MB content-addressed chunks4 MB uploaded — 500x less
File metadata rows~1 trillion — a partitioned database

So the hard part is conflicts and metadata, not bandwidth. Chunking plus object storage makes the bytes a solved problem; knowing which version is current on every device is the design.

Chunking is the number that changes the design. Everything else follows from it.

The architecture

have these hashes?presigned PUTconditional commitcursor movedDeviceslocal DB, offline-capableMetadata svcversions + cursorMetadata storepartition: accountChunk storagekeyed by content hashNotify svc"cursor moved"Chunk GCconservative sweepEvent queue
Bytes go straight to object storage by content hash. The metadata service owns versions and the per-account cursor, which is what makes sync a delta rather than a replay.

The decision this turns on

A
Pick a winner by timestamp
  • —One version survives, so the folder stays tidy and the user is never confused.
  • —Depends on device clocks, which on consumer machines are routinely wrong.
  • —The loser is deleted with no error, no notification and no way to recover it.
  • —Feels clean right up until someone loses an afternoon of work.
B
Detect and keep both
  • —The conditional commit fails, so the second device knows it is behind.
  • —Both versions exist; nothing is ever silently discarded.
  • —The user sees a conflicted copy they did not create and must resolve it.
  • —A long offline period across several devices can produce several such copies.
Which one: Keep both. The messy folder is a visible, recoverable annoyance; a silently deleted edit is invisible and permanent. The version check that produces the conflict is two lines of logic, and the product's correctness depends on it.
What to do when two devices committed edits from the same parent version.

Follow-ups you should expect

Many interviews are decided in the follow-ups, not the first diagram.

Two users click at the same millisecond. What happens?Now it must work offline and sync when the connection returns.A worker dies halfway through processing a batch. What is the state of the world?Your storage bill doubles every six months. What do you do?You grow 10x every year for three years. What do you rebuild?You write to the database and then publish an event. Name the two failure modes.Now it must be privacy compliant — users can demand deletion of everything about them.You need to move this data to a new store with no downtime. How?

Same shape, other products

The same data lives on several devices, all of which can be edited while disconnected.

Two people (or one person on two devices) changed the same thing offline. Everything hinges on whether you detect that or silently pick a winner, and on how little you can transfer to bring a device up to date.

Others with this shape: File sync · Offline-first notes · Collaborative editing · Mobile app caches

Before you leave
Try this problem again in a week without reading anything. Comparing what you write then with what you wrote today shows you what stuck.
Stuck, or new to this?Read the full worked solutionAll five stages end to end, with the diagrams and the costs, written out the way you would say it. Best read after you have had a go.Read it →

How it is really built

Write your own answer first. Then read how teams who run this at scale describe it. Their constraints were real, and the write-ups often say what their choices cost.