Chat and messaging — a full worked answer
The whole thing, start to finish, in the order you would actually say it. Read it to learn the shape of a strong answer — then go and produce one yourself, because reading is not the same skill as producing.
The question
Design a messaging service. One-to-one and group chats, messages delivered in order, delivery and read receipts, and history that survives a phone being lost. Assume mobile clients on unreliable networks.
Write-heavy / fan-out. One action by one person creates work for thousands of other people.
The question is always the same: do you do that work when they write, or when everyone else reads? And what do you do about the one user with 200 million followers?
Before anything: where is the slack?
How long the system has before a person notices. Ask this before you draw a single box — it decides more of the design than scale does.
Two people are looking at the same screen expecting the message to appear. There is almost no slack on delivery. There is plenty on everything around it, and the design depends on using it.
Delivery has near-zero slack when both people are in the conversation. A message that takes three seconds feels broken, because the sender is watching for the tick and the receiver is waiting to reply. So the delivery path has to be a push down an already-open connection, not a poll.
Everything else has slack, and the design spends it. Read receipts can be batched and sent every few seconds, because a delay of a few seconds before the tick turns blue is not something users notice. Push notifications to an offline device go through a third party with its own latency, so seconds are already baked in. History sync after reinstalling can take a minute, because the user knows they just reinstalled.
The one place with no slack at all, and no way to buy any, is ordering within a conversation. A reply appearing above the message it answers is a correctness problem, and making things faster does not fix it. That constraint, more than throughput, decides how you partition.
So: Persistent connections for delivery, a queue for everything after it, and all writes for one conversation routed through one partition so the sequence is unambiguous. Receipts and notifications are batched, because a few seconds of delay on them goes unnoticed.
Requirements
about 5 minWhy this stage exists: Because the question is deliberately vague, and every box you draw before this is a guess.
Assumptions, said quickly: mobile-first, global, 500 million users with about 50 million connected at any moment, messages are small text with occasional media, and history lives on the server so a new phone can restore it.
The question that changes a box: how big can a group get? Ten people and a hundred thousand people are different systems. A small group can fan out on write into every member's mailbox. A hundred-thousand-member broadcast group cannot, and needs pull at read time. I will assume groups up to about 1,000 members fan out, and anything larger takes a separate path. I will say that out loud rather than pretend one design covers both.
Second question: is this end-to-end encrypted? If it is, the server cannot search, cannot generate previews, cannot do content-based spam detection, and history restore depends on key backup rather than on my database. That one answer removes several features from the design. I will assume not encrypted end-to-end for this exercise, and flag it as the thing I would ask a real product owner first.
Scope I propose: send and receive one-to-one and group messages, ordering within a conversation, delivered and read receipts, offline delivery via push, and history sync to a new device. Out of scope: voice and video calls, end-to-end encryption, and search. Each is a separate design.
What an interviewer is ticking off in this stage
- Stated connected-user count, not just registered users — capacity here is connections
- Asked about maximum group size, and said why it changes the fan-out decision
- Asked about end-to-end encryption and named what it removes from the design
- Defined what "delivered" means before designing receipts
- Proposed a scope and named something you left out on purpose
Actors and lifecycle
about 6 minWhy this stage exists: Because the happy path is the easy half. The failure branches are where the design actually lives.
Actors: the sender, each recipient, each of their devices separately (one person with a phone and a laptop is two delivery targets), and the system itself, which assigns sequence numbers, decides when to give up on a device, and decides when to hand off to the push provider. The push provider is a fourth actor I do not control and cannot make reliable.
The chain: composed on device → accepted by server and assigned a sequence number → fanned out to recipient mailboxes → delivered to a connected device → read by a human → retained → eventually deleted. The ticks people see map onto three of those: accepted, delivered, read.
The failure branches are where the design is. At accepted: the client sent it twice because it lost the response. So the message needs a client-generated id and the server deduplicates on it. Otherwise a flaky network produces double messages, which users notice immediately. At fan-out: a recipient is offline, so the message sits in their mailbox and a push notification is fired. If the push provider is down, the message is still safe and is delivered on reconnect, because the mailbox is the truth and the push is only a hint. At delivered: the device receives it but crashes before storing it, so the acknowledgement has to come after the client has persisted it, not on receipt. At read: the receipt itself is lost. That is fine. Receipts are best-effort, and I would say so rather than build guaranteed delivery for a tick.
One case the system owns: a device that has been offline for two weeks reconnects and has 4,000 messages waiting. That cannot be one enormous push down a socket. It is a paginated sync from a sequence number, over normal HTTP, with the socket only carrying new traffic.
What an interviewer is ticking off in this stage
- Listed devices, not just users, as delivery targets
- Named the push provider as an actor outside your control
- Client-generated message id for deduplication on retry
- Mailbox is the truth; push notification is only a hint
- Acknowledgement after the client persists, not on receipt
- A reconnect after a long absence is a paginated sync, not a socket flood
The lifecycle, drawn
Numbers
about 6 minWhy this stage exists: Because one part of this is hard and the rest is easy, and they look identical until you do the arithmetic.
Assume 500 million users, 100 million active daily, sending 40 messages each. That is 4 billion messages a day. Divide by 100,000 seconds (a day is 86,400, rounded up) and you get roughly 40,000 messages per second on average; call it 120,000 at peak. Large, but not frightening.
Deliveries are the number that matters. If the average conversation has 3 participants, each send becomes about 3 deliveries, so 120,000 sends per second is around 360,000 deliveries per second at peak. That is the write amplification, and it is why fan-out is the decision to get right.
Connections: 50 million devices connected at once. If one tuned server holds 100,000 connections, that is 500 servers just to hold the sockets, before any messages flow. Capacity here is measured in connections, not requests per second. That changes how I think about deploys, because restarting the fleet makes 50 million clients reconnect at once.
Storage: a message row is roughly 300 bytes with metadata. 4 billion a day is about 1.2 TB a day, so 440 TB a year, which is significant but not exotic. Media is most of the bytes, and it does not belong in the database. It goes to object storage, and the message row holds a reference.
So the hard part here is fan-out and connection management, not raw message throughput. 120,000 writes a second is ordinary. Turning them into 360,000 ordered deliveries across 50 million live connections, without losing anything when a device is offline, is where the design effort goes.
What an interviewer is ticking off in this stage
- Produced messages per second with assumptions stated
- Multiplied sends into deliveries — named the write amplification
- Counted concurrent connections and turned it into a server count
- Separated media from message rows in the storage estimate
- Finished the sentence: fan-out and connections, not throughput
The numbers, drawn
So the hard part is fan-out and connection management. 120,000 writes a second is ordinary; turning them into 360,000 ordered deliveries across 50 million live sockets is not.
High-level design
about 12 minWhy this stage exists: Because now — and only now — you have a reason for every box you are about to draw.
Clients hold a WebSocket to a connection service, which does nothing but terminate sockets, authenticate, and hold a registry of which device is on which server. Those servers are sized by connection count, and they are kept simple on purpose, because they are the layer that has to be restarted carefully.
A send goes over the socket to the connection service, then to a message service, which does the work. It deduplicates on the client-generated id, assigns a per-conversation sequence number, and appends the message to the conversation's log. Everything is partitioned by conversation id, which gives ordering cheaply: all messages for one conversation are handled by one partition, so the sequence is arrival order at that partition, and no clocks are involved. That is why the partition key is conversation and not user.
Fan-out then writes a reference (recipient id, conversation id, sequence number) into each recipient's mailbox via a queue, because the sender must not wait for it. The mailbox is the durable truth of what a user has not yet seen. For groups above the threshold, no fan-out happens: members pull the conversation log when they open it.
Delivery: the message service asks the connection registry where each recipient's devices are and pushes to those servers. A device that is not connected gets nothing now, because its mailbox already has the entry. A push notification is sent through Apple or Google as a hint to open the app. That push is best-effort by design.
Storage: the conversation log is the source of truth, partitioned by conversation id and clustered by sequence number, so "last 50 messages" is one continuous read. Mailboxes are per user and capped. Media goes straight to object storage over a presigned URL, and the message row holds only the key, so my servers never carry the bytes.
Receipts flow the same way as messages but batched: a client sends "read up to sequence 812" rather than one receipt per message, which cuts that traffic sharply at no cost a user can see.
What an interviewer is ticking off in this stage
- Separated the connection layer from the message logic and said why
- Partitioned by conversation id, and justified it with ordering rather than load
- Per-conversation sequence numbers rather than timestamps
- Mailbox as the durable truth, push notification as a best-effort hint
- Media to object storage via presigned URL, never through the servers
- Receipts sent as a high-water mark, not one per message
The architecture, drawn
Deep dive and tradeoffs
about 14 minWhy this stage exists: Because naming what a decision costs is the single clearest signal of seniority there is.
Hard part one: the connection layer during a deploy. Restarting the fleet disconnects 50 million clients, and they all try to reconnect within seconds. That is a self-inflicted spike bigger than normal traffic, and each reconnect costs an authentication and a sync query. Defences, in order: roll the fleet slowly, a few percent at a time, so the reconnect load is spread over an hour. Clients reconnect with randomised, growing delays, so even a sudden mass disconnect arrives smoothly. And the reconnect handshake is cheap by design: the client sends the last sequence number it has per conversation and gets back only what is missing, rather than re-fetching state. Cost: deploys to that layer take an hour rather than five minutes. So I keep the layer small and change it rarely, which is why it is separate from the message service.
Hard part two: the very large group. A 100,000-member group posting actively would generate 100,000 mailbox writes per message, which floods the queue and delays everyone else. So above a threshold (a thousand members, tuned from queue lag) I stop fanning out. The message is written once to the conversation log, and members read it when they open the conversation. That log is one key with enormous read volume, which caches very well. Cost: two code paths for one feature forever, a threshold that needs an owner, and unread counts for those groups become approximate rather than exact, because there is no mailbox to count. I would show "99+" and accept it.
History sync to a new device. Not over the socket. The client requests conversations in pages over normal HTTP, newest first, and gets recent messages per conversation immediately so the app is usable within seconds, with older history filling in in the background. Cost: full restore takes minutes, which is acceptable because the user knows they just installed the app.
Duplicate and out-of-order handling on the client. The client can receive the same message twice (from the socket and again from a sync), so it deduplicates on message id and inserts by sequence number rather than by arrival. So the client is idempotent too, not only the server. That part is easy to forget.
What an interviewer is ticking off in this stage
- Named the reconnect storm on deploy and gave three specific defences
- Set a fan-out threshold and said how you would tune it
- Accepted approximate unread counts for large groups, explicitly
- History sync over HTTP, paged, not down the socket
- Client-side deduplication and ordering by sequence number
- Named the cost after every decision
The decision this turns on
- —Opening the app is one read of a precomputed list, which is fast where users notice it.
- —Exact unread counts, because the mailbox is countable.
- —A 100,000-member group is 100,000 writes per message, flooding the shared queue.
- —Mailbox writes and storage grow with messages times members.
- —One write per message regardless of group size, so enormous groups cost no more to write to.
- —Reading means querying every conversation you are in, then merging.
- —Unread counts become approximate unless you track a read pointer per conversation.
- —One cached log key serves everyone, which caches very well.
And then they push back
A complete answer is where the interview starts, not where it ends. These are the questions this design invites — the interview is decided here far more often than in the diagram.
One user has 200 million followers. What happens?
"The fan-out would be slow but it would eventually finish." It would, and by then the post is hours old and every other user's timeline was blocked behind it. Treating this as a latency inconvenience rather than a systemic failure misses the point.
"One post becomes 200 million writes. At my fan-out capacity that is minutes, and worse, it occupies the pipeline so everyone else's posts are delayed behind one person's. So above a follower threshold I stop fanning out entirely — their posts stay in place and readers pull them at read time, merged with their normal inbox. That works because a huge account's recent posts are the most cacheable data in the system: millions of people want the same few rows, so the hit rate is near perfect. Cost: two code paths for one feature forever, a threshold that needs tuning, and timelines assembled from two sources so pagination needs a cursor rather than an offset."
Two events for the same record arrive out of order. What happens?
"The queue guarantees ordering." Only within a partition, and only if you partitioned by the right key. Assuming global ordering is a common mistake with queues.
"If they went to different partitions, they can arrive out of order, and the second one applied could overwrite the newer state with older data. Two answers. Structural: partition by the record's id, so all events for one record land on one partition and are ordered relative to each other. I do not need global ordering and would not pay for it. I need per-key ordering, and that is cheap. Defensive: make the application tolerant anyway, by carrying a version or timestamp on each event and ignoring one that is older than what I have already applied. I would do both, because the partitioning can be right and a consumer restart or a retry can still deliver out of sequence. Where it really matters, pushing full current state with a version, rather than increments, makes ordering much less important: a duplicate re-applies a state that is already correct, and a late, older state is discarded by its version."
A third-party API you depend on starts taking 30 seconds to respond. What happens?
"We would retry." Retrying against something already overloaded makes it worse, and retrying a 30-second call means holding your resources for a minute or more. This answer usually turns one slow dependency into your own outage.
"Slow is more dangerous than down, because down fails fast and slow holds resources. Without a timeout, every request calling that API holds a thread and a connection for 30 seconds, my pools exhaust, and endpoints that never touch that API start failing. One slow dependency has become a total outage. So: a timeout set from their real latency distribution, not a round number; a circuit breaker that opens on failure rate so I stop waiting entirely once it is clearly bad; a bulkhead giving that dependency its own limited pool so it cannot consume capacity the checkout path needs; and a defined fallback — cached data, a degraded response, or dropping an optional feature. The fallback has to be decided before the incident, per dependency, or I fail fast into a blank page. And retries need backoff with jitter and a cap, or I contribute to keeping them down."
Why precompute this instead of computing it on read?
"Precomputing is faster." It is, at read time, and it costs you at write time and in storage and in freshness. Not naming what you traded means you have not made a decision.
"Because of the ratio and the slack. Reads outnumber writes here by a hundred to one, so paying once at write to save a hundred times at read is worth it. And the freshness budget is hours, so precomputing something that is a few minutes old costs nothing a user can perceive. What it costs: writes get more expensive and amplified, storage grows with the precomputed shape, and any change to the output format means a full rebuild. The place I would reverse this is where the write amplification becomes extreme — one account with millions of followers — so I compute those on read instead, and accept two code paths. The general rule I am applying: move work toward whichever side is rarer, and check the tail of the distribution before committing, because the average hides the case that breaks it."
How do you deploy a new version with no downtime?
"Rolling deploy behind the load balancer." Correct and incomplete. The hard part is database changes and in-flight requests, and stopping there suggests you have only done it on a stateless service.
"Rolling deploy, taking instances out of the pool, draining in-flight requests (skip connection draining and you drop live traffic during a zero-downtime deploy), then updating, waiting for health checks, and putting them back, keeping enough capacity in the pool throughout. The part that makes it work is that the new version must tolerate the old version running beside it, because for several minutes both are live. That means database migrations get split: add the new column, deploy code that writes both and reads the old, backfill, deploy code that reads the new, then remove the old column. That is three or four releases instead of one. Cost: every schema change is now a multi-deploy sequence, which is slower and is the price of not taking downtime. And I would rather roll forward to a known-good build than roll back, because undoing a migration is much harder than undoing code."
Now it must work offline and sync when the connection returns.
"We would cache data locally and sync later." That is the easy direction. The hard direction is writes made offline that conflict with writes made elsewhere, and sync order.
"Reads offline are straightforward: cache locally with a clear indication of how stale it is. Writes are the design. Every offline change becomes a queued operation with a client-generated id, applied optimistically to local state so the app feels responsive, and replayed on reconnection. Then the two hard problems. Conflicts: the same record was edited on the device and on the server, so I need a resolution rule — last-write-wins is simple and silently discards someone's work, per-field merging is better where fields are independent, and for heavily collaborative data a merge-friendly data type is the right tool. Ordering: operations must replay in the order they were made, and one failing must not silently skip the rest. Also, an operation queued offline may no longer be valid (the item was deleted, the permission revoked), so the server must be able to reject it and the client must be able to show that rejection to the user rather than losing it silently. What I would flag: this is a much bigger change than it sounds, because it turns every write path into a distributed multi-leader system."
Your storage bill doubles every six months. What do you do?
"We would delete old data." Which data, and who agreed you could? Retention is usually a product and legal decision, not an engineering one.
"First, find out what is actually growing, because it is rarely uniform — usually one dataset, often logs or raw events or derived copies nobody remembers creating. Then, in order of cheapness: lifecycle policies moving cold data to colder tiers, which is cheap to implement and can cut the bill sharply when most data is rarely read after its first week; compression and columnar formats, which for event data is a 5-10x reduction; tiered granularity, keeping recent data fine-grained and rolling older data up; and only then deletion, which needs a retention policy agreed with the product and legal side rather than decided by me. I would also check for duplication — a derived index, a backup of a backup, an old migration copy — because a surprising share of storage growth is copies. The one thing I would not do is delete raw data that lets me reprocess when a job turns out to be wrong, because that is the safety net that makes everything else correctable."
Two halves of your cluster can see clients but not each other. What now?
"The system would heal itself when the network came back." It might heal, or it might have accepted conflicting writes on both sides for ten minutes and now has two versions of the truth with no way to merge them.
"This is a partition, which is the case CAP is about. Both halves are up and serving, so the danger is not availability, it is divergence — if both sides accept writes, I get two conflicting histories. The defence is a majority rule: only the side that can reach more than half the nodes may accept writes, so the minority side refuses. That means some users get errors even though their servers are healthy, and I accept that on purpose for anything contested — inventory, balances, unique names. For non-contested data I would keep both sides available and reconcile afterwards, because the worst case is a merge rather than a double-sell. The failure to avoid is split brain: the old leader not knowing it lost, so I need fencing tokens so the storage layer rejects writes from a leader that has been superseded."
The topics this answer used
You have read a good answer. That is worth very little until you can produce one on a blank page — so go and write this problem without looking, and compare afterwards.