Dev Learning
Big files

Video upload and playback

Design a service where people upload videos and other people watch them. Uploads come from phones on unreliable networks. Playback should start quickly and adapt to the viewer's connection.

hardCDNsMessage queues and backpressureIdempotency, and why exactly-once is a mythCachingChange data captureObject storage and presigned uploadsCapacity planning and costBatch and stream processing

Where’s the slack?

Processing: minutes. Playback start: under 2 seconds. Upload: hours if needed.

Uploading and processing have hours of slack. Starting playback has almost none. Designing both paths the same way is the usual way to fail this problem.

Uploads have hours of slack. Users do not expect a video to be watchable the instant the progress bar completes; video products show "processing" and users accept it. That makes the design tractable: transcoding into five resolutions is expensive, parallelisable, and off the user's critical path. You can retry it, queue it behind other work, and run it on cheap interruptible capacity.

Playback start has a hard budget of about two seconds. Beyond that, a growing share of viewers give up before the video starts, and video services track this number closely. The read path is built around that budget: content near the viewer, small first segments, a low starting bitrate that steps up.

Mid-playback has a different budget again. The player keeps a buffer, typically 10 to 30 seconds, which is slack it can spend fetching the next segments and adapting quality. Adaptive streaming depends on that buffer.

So there are two systems. One is optimised for throughput and cost, with hours to play with; the other is optimised for latency, with two seconds. They share only a storage bucket.

So: Hours of processing slack means transcoding is fully asynchronous, retryable, and can run on interruptible capacity. A two-second playback budget means segments must be static files on a CDN, which is why video is chopped into segments in the first place.

Work it through

Five stages, gated. You write your answer for each stage before its model answer unlocks. That is the core of the method.

Loading your progress…

Reference material for this problem. The diagrams below are here to check your own against once you have drawn it, not to read first.

The lifecycle

UploadingphoneStoredobject storageQueuedsystemTranscodedworkersPublishedsystemConnection drops at 80%chunked and resumable: ask which chunks exist, send the restProcessing event lostevent written in the same transaction; sweeper finds stuck videosWorker crashes mid-jobretry; deterministic output paths make it idempotentFile is unplayablenot retryable: fail explicitly and tell the uploader whyOnly some renditions donepublish on first usable rendition, add the rest as they landTakedown requestedpurge propagates across edges with real delay; say so
A video's life. "Stuck in processing forever" is the branch that users notice and that is most often missing from a whiteboard.

The numbers

Ingest~8 GB / sec
Egress to viewers~1,500 GB / sec
Storage added per day~2 PB / day (5 renditions)

Egress is ~180x ingest and it never stops. So the hard part is bandwidth, and the answer is that almost none of it may come from your origin. That is why video is segmented into static files in the first place.

The comparison that decides the design: what you serve dwarfs what you receive.

The architecture

get URLchunks, directmissmissUploaderAPIpresigned URLsObject storagesource + segmentsTranscode queueTranscode workersinterruptibleMetadata DBCDNimmutable segmentsRegional cachecoalesces missesViewer
Bytes never pass through the application servers. The upload path and the playback path share only a storage bucket.

The mechanism

ProducerfastQueuebacklogWorkerslow
four inone out — the queue absorbs the difference, and grows
Transcoding: uploads arrive faster than workers finish. Hours of slack mean the queue can absorb this, as long as you watch its age, not its length.

The decision this turns on

A
Direct to object storage (presigned)
  • —Bytes never touch your servers, so you never build a bandwidth tier.
  • —Storage handles resumption, parallel chunks and retries natively.
  • —Scales with upload volume without extra work from you.
  • —Validation and scanning happen after the fact, not inline.
B
Through your application servers
  • —Full control: validate, scan and transform the stream as it arrives.
  • —Simpler client, one endpoint, no presigning dance.
  • —You now carry 8 GB/sec of ingest bandwidth and the servers to terminate it.
  • —Every upload holds a connection for minutes, which wrecks your capacity model.
Which one: Direct to storage, once the payload is large. The control you give up can be recovered asynchronously; the bandwidth bill you take on cannot. The same reasoning applies to any big-file problem: file storage, image hosting, backups.
Where the upload bytes go. This is the first fork, and the wrong branch is expensive forever.

Follow-ups you should expect

Many interviews are decided in the follow-ups, not the first diagram.

What does this cost per month?The cache just died. What happens?10x traffic tomorrow. What fails first?One user reports a slow request. How do you debug it?Why a queue instead of just calling the service directly?A whole region is down. What do users see?Now make it multi-region.There is an option that is 40% cheaper but 10% slower. Do you take it?

Same shape, other products

The payload is far bigger than the metadata, and it should not go through your servers at all.

The moment you draw the file passing through your application server, you have made bandwidth your problem. Let the client talk to storage directly.

Others with this shape: Video upload and playback · Cloud file storage · Image hosting

Before you leave
Try this problem again in a week without reading anything. Comparing what you write then with what you wrote today shows you what stuck.
Stuck, or new to this?Read the full worked solutionAll five stages end to end, with the diagrams and the costs, written out the way you would say it. Best read after you have had a go.Read it →

How it is really built

Write your own answer first. Then read how teams who run this at scale describe it. Their constraints were real, and the write-ups often say what their choices cost.