Object storage and presigned uploads
Big files do not belong in your database, and they should not pass through your servers either.
What problem does object storage solve?
A photo, a video, a PDF, a backup. These are far bigger than the row that describes them, and a database is the wrong place for them. Storing a 200 MB file as a database blob makes every backup, every replica and every restore enormously slower.
Object storage (S3, GCS, Azure Blob) is built for this. You put a file in under a key, you get it back by that key, and it is cheap, effectively unlimited, and durable without you doing anything.
What object storage costs you
Every concept in here states a cost. A technique with no price is a technique you have not understood yet — and naming the cost out loud is most of what separates a senior answer from a junior one.
When to reach for object storage, and when not to
Use it when
- Any user-uploaded file: images, video, documents, avatars.
- Anything you generate and hand back: exports, reports, invoices, backups.
- Static assets that a CDN will sit in front of.
- Big append-only data you rarely read: logs, archives, raw events.
Do not, when
- Small structured data you query by fields. That is a database.
- Anything needing a transaction with your other data. There is no transaction across a database and a bucket.
- Very frequent small updates. You cannot edit part of an object. You replace the whole thing.
How object storage actually works
The key pattern here is the presigned URL. It answers most questions that involve a large upload. Your service creates a URL that grants permission to write one specific key, for a few minutes, and hands it to the client. The client uploads straight to the storage service. Your servers never see the bytes.
Say why that matters, because the reason is the interview answer. If a 2 GB upload goes through your application server, then that server is holding a connection for minutes, you are paying for the bandwidth twice, and you cannot scale uploads without scaling application servers. With a presigned URL, your involvement is a few hundred bytes of signing. Downloads work the same way in reverse, for private files.
Multipart upload is the companion. A big file is cut into parts, each part is uploaded on its own and can be retried on its own, and the storage service reassembles them. Without it, a failure at 95% of a 5 GB upload means starting again. Parts can also be uploaded at the same time, which is how you make full use of a fast connection.
Next, the consistency problem, which is the usual follow-up. Your database row and your file are two separate writes with no transaction across them. The usual fix is a status field. Create the row as pending before handing out the URL. The client tells you when it is done, or the storage service fires an event, and you flip the row to ready. Anything still pending after an hour is an abandoned upload, and a cleanup job deletes both. Never show a pending file to anyone.
Storage classes and lifecycle rules are how the cost story stays sane. Hot storage is instant and costs the most. Colder tiers are cheaper per month and slower or more expensive to read. A lifecycle rule moves objects between them by age automatically. If you are asked about cost on a system that stores a lot, the answer is "raw uploads go to cold storage after 30 days, and the derived versions people watch stay hot".
Two things that catch people out. Object storage is not a filesystem. There are no real directories, just keys that happen to contain slashes, and listing many keys with a common prefix is slow. Keep an index of your objects in your database, and never make the bucket listing part of a user-facing request. And you usually cannot append or edit in place. On most object stores, changing one byte means writing the whole object again, which is why logs go to object storage in whole files, not line by line.
- Storing files as database blobs, then wondering why backups and replicas are so slow.
- Streaming a large upload through the application server because it is easier to write.
- Signing a URL from a client-supplied key, letting one user write over another user's object.
- No cleanup for pending rows and abandoned multipart uploads. They cost money forever.
The object storage follow-up an interviewer will ask
"The user uploaded a 2 GB video. Walk me through it."
The client asks my API to start an upload. I create a row in the database marked pending, with the object key and the owner, and I return a presigned multipart upload, so the client can cut the file into parts, upload them in parallel straight to object storage, and retry any single part that fails without restarting the whole thing. My servers never see the video. When the client completes the upload, either it calls me or the storage service fires an event, and I flip the row to uploaded and put a job on a queue to transcode it. That job writes the output renditions back to storage as separate objects and marks the row ready. The costs I would name: the row and the object can disagree, so I need a sweeper that deletes pending rows older than an hour and their orphaned parts, because incomplete multipart uploads keep costing money silently. And the presigned URL is a real permission, so it is scoped to one key, expires in minutes, and the key is generated by me rather than taken from the client, or someone uploads over another user's file.
Your turn
Why hand out a presigned URL instead of just accepting the file through your API? Give two reasons.
Learn it from the source
This page gets you interview-ready. These take you to where the idea is actually specified or measured — read them when you want the real depth.
- Amazon S3 — User Guide ↗The service that defined the category, including its consistency and durability guarantees.