Chat and messaging
Design a messaging service. One-to-one and group chats, messages delivered in order, delivery and read receipts, and history that survives a phone being lost. Assume mobile clients on unreliable networks.
Where’s the slack?
Delivery: under a second. Receipts: a few seconds. History sync: tens of seconds.Two people are looking at the same screen expecting the message to appear. There is almost no slack on delivery. There is plenty on everything around it, and the design depends on using it.
Delivery has near-zero slack when both people are in the conversation. A message that takes three seconds feels broken, because the sender is watching for the tick and the receiver is waiting to reply. So the delivery path has to be a push down an already-open connection, not a poll.
Everything else has slack, and the design spends it. Read receipts can be batched and sent every few seconds, because a delay of a few seconds before the tick turns blue is not something users notice. Push notifications to an offline device go through a third party with its own latency, so seconds are already baked in. History sync after reinstalling can take a minute, because the user knows they just reinstalled.
The one place with no slack at all, and no way to buy any, is ordering within a conversation. A reply appearing above the message it answers is a correctness problem, and making things faster does not fix it. That constraint, more than throughput, decides how you partition.
So: Persistent connections for delivery, a queue for everything after it, and all writes for one conversation routed through one partition so the sequence is unambiguous. Receipts and notifications are batched, because a few seconds of delay on them goes unnoticed.
Work it through
Five stages, gated. You write your answer for each stage before its model answer unlocks. That is the core of the method.
Reference material for this problem. The diagrams below are here to check your own against once you have drawn it, not to read first.
The lifecycle
The numbers
So the hard part is fan-out and connection management. 120,000 writes a second is ordinary; turning them into 360,000 ordered deliveries across 50 million live sockets is not.
The architecture
The decision this turns on
- —Opening the app is one read of a precomputed list, which is fast where users notice it.
- —Exact unread counts, because the mailbox is countable.
- —A 100,000-member group is 100,000 writes per message, flooding the shared queue.
- —Mailbox writes and storage grow with messages times members.
- —One write per message regardless of group size, so enormous groups cost no more to write to.
- —Reading means querying every conversation you are in, then merging.
- —Unread counts become approximate unless you track a read pointer per conversation.
- —One cached log key serves everyone, which caches very well.
Follow-ups you should expect
Many interviews are decided in the follow-ups, not the first diagram.
Same shape, other products
One action by one person creates work for thousands of other people.
The question is always the same: do you do that work when they write, or when everyone else reads? And what do you do about the one user with 200 million followers?
Others with this shape: Twitter timeline · Chat delivery · Notification system · Activity feed
How it is really built
Write your own answer first. Then read how teams who run this at scale describe it. Their constraints were real, and the write-ups often say what their choices cost.
- Slack Engineering ↗Engineering write-ups on message delivery, presence and fan-out at scale.