Dev Skills
← Ride matching·Worked solution

Ride matching — a full worked answer

The whole thing, start to finish, in the order you would actually say it. Read it to learn the shape of a strong answer — then go and produce one yourself, because reading is not the same skill as producing.

hard
Common trap
Read this second, not first. Reading a good answer feels like learning and only builds recognition. If you have not tried this problem yet, close this page and write your own answer first. It will feel worse and it is the part that works.

The question

Design the matching part of a ride-hailing service. A rider requests a ride from where they are standing; the system finds a nearby driver, offers them the trip, and confirms the match. Drivers move continuously and can decline.

What kind of problem this is

Matching under time pressure. Two sides need to find each other, and one of them is a human who is waiting.

Supply moves, demand is impatient, and the interesting design is what happens when nobody accepts, rather than the happy path.

Before anything: where is the slack?

How long the system has before a person notices. Ask this before you draw a single box — it decides more of the design than scale does.

Match: ~10 seconds, and every second is visible.

A person is standing on a street with a phone in their hand. There is no cooking time and no processing window to hide latency inside. Few problems in this app have less slack.

Compare this to food delivery, which looks like the same problem. There, a courier does not need to be assigned the moment the order is placed. The restaurant needs 15 minutes to cook, and the system can spend several of those minutes finding the best courier, batching orders together, and waiting for a better option to come free. Those 15 minutes are slack, and they make a different design possible: batched optimisation rather than immediate assignment.

Ride hailing has none of it. The rider is outside, possibly in the rain, watching a spinner. Ten seconds without a driver feels broken; after thirty they may open a competing app. So matching must be greedy and immediate: take a good driver now, not the best driver in forty seconds. That rules out global optimisation, which is the elegant answer and the wrong one here.

There is one small pocket of slack: the few seconds between offering a trip to a driver and their tap. That window lets you sequence offers rather than blasting all candidates at once. But the rider is waiting through it and other drivers are moving away, so it is measured in seconds and cannot be spent freely.

Everything not on the matching path (receipts, ratings, driver earnings, analytics) has hours of slack and should be nowhere near this design.

So: Zero slack means greedy immediate matching, not batch optimisation. It also means the failure branches (nobody accepts, everybody declines) are the design, because they happen while a human is watching.

1

Requirements

about 5 min

Why this stage exists: Because the question is deliberately vague, and every box you draw before this is a guess.

What you are being asked for: State your assumptions, ask one or two questions that change a box, and propose the scope.
What a strong answer sounds like

Assumptions: one city at a time as the unit of scale, so I design for a city and replicate the design rather than building one global matcher. Hundreds of thousands of active drivers per large city at peak. Drivers report location every 4 seconds. Payments and identity are elsewhere. Riders are impatient: a match must happen in under about 10 seconds.

First question, which changes correctness rather than polish: do we offer a trip to one driver at a time and wait, or broadcast to several and take the first to accept? Broadcasting means two drivers can accept the same ride, which is a contested-resource problem needing a lock or a conditional write; sequential offering avoids that entirely but is slower when drivers ignore their phones. I will assume sequential offers with a short timeout, falling back to a small parallel batch if the first few time out.

Second question: does the rider get to see drivers moving on the map before requesting? That turns a matching problem into a continuous streaming problem for every open app, which is a much larger read load than matching itself. I will assume yes, since most ride-hailing apps do it, and treat it as a separate lower-priority path.

Scope: request a ride, find candidate drivers, offer and confirm, and handle nobody accepting. Out of scope: pricing and surge, routing and navigation, payments, and the trip itself once it has started.

The costs named here
City as the unit of scale → costs: Cross-city trips and airport edges need special handling. In exchange, every shard is independent and a city outage is contained.
Sequential offers → costs: Slower when drivers do not respond, since each timeout costs seconds from a 10-second budget. Avoids two drivers accepting the same ride.
Say this
"Assuming I design for one city and replicate it. Matching does not need to cross cities. Drivers ping every four seconds, so writes hugely outnumber reads. One question: do we offer to one driver at a time or broadcast? That is a correctness question, because broadcasting means two people can accept the same trip. I will assume sequential with a short timeout."
Common trap
Asking about car types, or ratings, or the pricing model. None of them change a box in the matching design. The offer protocol does.
What an interviewer is ticking off in this stage
  • Scoped to a city rather than designing one global matcher
  • Stated the location update frequency, since it drives the write load
  • Asked sequential-versus-broadcast offering and explained it is a correctness question
  • Named the rider patience budget as a hard constraint
  • Proposed scope and excluded pricing, routing and payments explicitly
2

Actors and lifecycle

about 6 min

Why this stage exists: Because the happy path is the easy half. The failure branches are where the design actually lives.

What you are being asked for: List the actors — including the system — then write the trip request's life as a chain, and add the failure branches. The branches are the whole problem here.
What a strong answer sounds like

Actors: rider, driver, and the system, which is unusually active here. The system decides which drivers are candidates, in what order to offer, how long to wait for each, when to widen the search, when to give up, and whether a driver who ignores offers should keep receiving them.

The chain: requested → candidates found → offered to a driver → accepted → driver en route → picked up. The design lives in the first four states.

Failure branches, which are the substance of the problem. At candidates found: there are no drivers in range. Widen the radius, then tell the rider honestly rather than spinning forever. At offered: the driver does not respond within the timeout. Withdraw the offer and move to the next candidate. The offer must expire server-side, or a driver accepting late collides with a driver who already took it. At offered: the driver declines. Move to the next candidate and record the decline, because a driver declining everything is a supply problem, not a matching one. At offered: every candidate declines or times out. Candidates often skip this branch, and it is common. The answer is to re-run the search, since the world has moved on in ten seconds, widen the radius, and after a bounded number of rounds tell the rider no driver is available and stop, because an endless spinner is the worst outcome.

At accepted: two drivers accept, which is only possible in the broadcast variant and is why the accept must be a conditional write: first accept wins, second is told the trip is gone. At accepted: the rider cancels in the same moment the driver accepts, so both operations race. The trip state machine must have a single owner and reject invalid transitions, rather than two services each deciding. At en route: the driver goes offline or their app crashes. Detect it via missed heartbeats and re-match, which means a trip can go back to the offering state, and the rider must be told.

One more the system must handle on its own: a driver whose location is 45 seconds stale should not be a candidate at all, because offering to a driver who is no longer where you think they are burns the rider's patience budget for nothing.

Common trap
Writing "request → match → ride" and calling it a lifecycle. Nobody accepting is not an edge case. At peak it happens often, and the interviewer is listening for it.
What an interviewer is ticking off in this stage
  • Named the system as an active decision maker and listed its decisions
  • Wrote the request lifecycle as a chain of states
  • Handled no drivers in range: widen, then be honest
  • Handled driver timeout with a server-side offer expiry
  • Handled the everybody-declines case with bounded rounds and a real "no drivers" outcome
  • Handled two drivers accepting via a conditional write
  • Handled the rider-cancels-while-driver-accepts race with a single state owner
  • Handled the driver disappearing after accepting
  • Excluded drivers with stale locations from candidacy

The lifecycle, drawn

RequestedriderCandidates foundsystemOfferedsystemAccepteddriverEn routedriverNo drivers in rangewiden radius, then say so honestly — never spin foreverDriver times outserver-side expiry, withdraw, next candidateEveryone declinesbounded rounds, re-search each round, then a real no-driver resultTwo drivers acceptconditional write on trip state — first wins, second told clearlyRider cancels simultaneouslysingle owner of trip state rejects the invalid transitionDriver goes offlinemissed heartbeats for 30s → back to offering, tell the rider
The trip request. Every branch below happens routinely at peak. "Nobody accepted" is not an edge case; it happens every day.
3

Numbers

about 6 min

Why this stage exists: Because one part of this is hard and the rest is easy, and they look identical until you do the arithmetic.

What you are being asked for: Estimate location writes, matching reads, and storage. Then finish "So the hard part here is ___."
What a strong answer sounds like

Take a large city at peak: 100,000 active drivers, each reporting every 4 seconds. That is 25,000 location writes per second, from one city, continuously, all day.

Ride requests in the same city at peak: perhaps 500 per second, fifty times fewer than the location writes. So this is a write-dominated system, the opposite of most products, and that fact decides the storage choice.

Each match reads the candidate drivers in a small area: maybe 50 to 200 drivers examined per request, so 500 requests a second is around 50,000 candidate reads a second. Significant, but small next to the writes.

Storage: a live position is a driver id, a coordinate, a timestamp and a status — under 100 bytes. Current positions for 100,000 drivers is 10 MB. That fits in memory easily, and it is the most useful conclusion in this estimate: the live index is a memory-sized problem, so it should not be in a database at all.

Historical positions are a different matter: 25,000 a second at 100 bytes is 2.5 MB a second, about 200 GB a day per city. That is a real data volume, and it exists for analytics and disputes, not for matching. So it goes to a separate append-only path and never touches the matching system.

So the hard part here is a very high write rate against an index that must stay queryable by location — and doing it inside a rider's ten-second patience budget. Not storage, not read throughput, and not the matching algorithm, which is simple once the index is right.

Say this
"Twenty-five thousand location writes a second against five hundred ride requests a second. Writes beat reads fifty to one, which is unusual. But all live positions are only ten megabytes, so the live index belongs in memory, not a database. So the hard part here is sustaining that write rate on a geo index while answering a match in seconds."
Common trap
Designing a durable, transactional store for driver locations. A position that is four seconds old is worthless and will be overwritten in four more seconds. Durability here spends money protecting data with no value.
What an interviewer is ticking off in this stage
  • Calculated location writes per second and noticed they dominate
  • Compared them to ride requests: roughly 50:1
  • Concluded the live position index fits in memory
  • Separated historical position storage from the live index entirely
  • Finished the sentence: high-frequency writes into a geo-queryable index, under a hard latency budget

The numbers, drawn

Driver location pings~25,000 / sec
Candidate reads~50,000 / sec (~100 per request)
Ride requests~500 / sec
All live positions in memory~10 MB

All live positions fit in 10 MB. That number says the live index belongs in memory, not in a database, and that durability for a position that expires in 4 seconds is money spent protecting nothing.

One large city at peak. Location writes outnumber ride requests fifty to one, the opposite of most systems.
4

High-level design

about 12 min

Why this stage exists: Because now — and only now — you have a reason for every box you are about to draw.

What you are being asked for: Draw the design. Justify each box, and be specific about how "find nearby drivers" actually works.
What a strong answer sounds like

Location ingestion: drivers send positions over a persistent connection to a location service, which updates the in-memory geo index and separately fires the position onto a stream for the historical path. Justified by 25,000 writes a second: a persistent connection avoids per-ping HTTP overhead, and the two destinations have different requirements.

The geo index, which is the heart of it. Encode each position as a geohash of 6 characters (a cell of about 1.2 km by 0.6 km) and keep, per cell, a sorted set of drivers with their exact coordinates and last-seen time. Redis is the natural fit: in-memory, fast writes, and native support for this shape. A driver moving between cells is a remove and an add. Finding candidates means reading the rider's cell plus the eight neighbouring cells, because a driver 100 metres away can be just over a cell boundary, and missing them is a real bug.

Then filter and rank: drop anyone whose position is older than about 15 seconds, drop anyone not available, compute a real distance, and rank by estimated arrival time rather than straight-line distance. A driver 200 metres away across a motorway with no crossing is further away in practice than one 600 metres down the same road. The routing call is on the hot path, so it is a batch call for the top handful of candidates only, not for all 200.

Matching service: takes the ranked candidates and runs the offer protocol: offer to the top candidate, wait a few seconds, withdraw and move on, widening the search every couple of rounds, giving up after a bounded number. Greedy, not optimal, justified directly by the ten-second budget from Stage 1.

Trip state machine: one service owns trip state, in a durable store, with transitions enforced as conditional updates. This is the one part that must be strongly consistent and durable, because it is the record of who is going where and who gets paid. Justified by the Stage 2 races: two accepts, and cancel-versus-accept.

Push to drivers and riders: persistent connections, so an offer reaches a driver immediately and the rider sees status change without polling. Justified by the patience budget: polling every 2 seconds could waste a fifth of it.

Historical positions: the stream feeds an append-only store for analytics and dispute resolution. Separate on purpose, with no coupling to matching, so a problem in the analytics pipeline cannot slow down a match.

Partitioning: everything is sharded by city, and within a large city by geohash prefix, so a match only touches the shard covering that area. Justified by the numbers: 25,000 writes a second per city is fine on one in-memory node, and city isolation bounds the blast radius.

The costs named here
In-memory geo index → costs: A node restart loses all positions. Acceptable, because drivers re-ping within 4 seconds, so it self-heals faster than a recovery process would.
Greedy matching instead of batch optimisation → costs: Globally worse assignments, with longer average pickup times than a system that waits and optimises. Accepted because rider patience leaves no time to optimise.
Ranking by ETA, requiring a routing call → costs: A third-party dependency on the hot path, so it needs a timeout and a fallback to straight-line distance. Only called for the top few candidates.
Common trap
Storing driver locations in the main transactional database with a lat/lng index. At 25,000 writes a second you have just made your most important database the busiest one in the system, for data that is worthless four seconds later.
What an interviewer is ticking off in this stage
  • Explained why plain lat/lng columns do not work and named a geospatial approach
  • Live positions in memory, not a durable database
  • Searched neighbouring cells, not just the rider's cell
  • Filtered out stale positions before ranking
  • Ranked by estimated arrival time, not straight-line distance
  • Offer protocol is greedy with timeouts, justified by the patience budget
  • A single owner for trip state, with conditional transitions
  • Persistent connections for offers rather than polling
  • Historical position stream separated from the matching path
  • Sharded by city and by geo cell

The architecture, drawn

candidatesrank by ETAaccept = conditionalDriver appping / 4sLocation svcGeo indexin-memory, by cellPosition streamHistory storeanalytics onlyRider appMatching svcoffer protocolTrip statedurable, conditionalRoutingETA, top N only
Live positions live in memory and are never durable. Trip state is durable and strongly consistent. The two are separate systems on purpose.
Writeshash(key)Shard Aa–hShard Bi–pShard Cq–z
the key decides the shard — same key, same box, every time
Positions are routed to the shard owning their geo cell. A match reads its own cell plus the eight neighbours, because a driver just over a boundary can still be 100 metres away.
5

Deep dive and tradeoffs

about 14 min

Why this stage exists: Because naming what a decision costs is the single clearest signal of seniority there is.

What you are being asked for: Go deep on the two hardest parts. Name a cost after every decision.
What a strong answer sounds like

Hard part one: the offer protocol under a hard deadline. Sequential offers with a 5-second timeout mean three failed offers consume 15 seconds, already past the budget. So the protocol adapts: offer to one driver, and if the first two time out, switch to offering a small batch of three or four simultaneously and take the first accept. That converts a latency problem into a correctness problem, which I then solve explicitly. Accept is a conditional update on trip state (assign the driver only if the trip is still unassigned), so the first accept wins and the others are told the trip is gone, with a clear message rather than an error. Cost: drivers occasionally tap accept and lose, which is a bad experience and needs handling in the app; and a batch offer means several drivers stop looking at other work for a few seconds.

The offer must expire on the server, not just in the driver's app. If expiry lives only on the client, a driver with a frozen screen accepts 40 seconds later and collides with a completed match. Cost: server-side expiry means holding per-offer state with timers, which is more moving parts than a stateless design.

Nobody accepting, in detail, because it is a common real failure: after a bounded number of rounds (say three, with the radius widening each time), stop and tell the rider no drivers are available, with a retry option. Between rounds, re-run the search rather than reusing the candidate list, because in ten seconds drivers have moved and some have become free. Cost: re-running means more index reads per request, and the read estimate leaves room for that.

Hard part two: peak and hot cells. Demand is not uniform. A stadium emptying creates a single geohash cell with thousands of requests and few drivers. That cell's shard becomes hot for both reads and writes, and the index reads return the same overloaded candidate set to everyone. Defences: sub-partition hot cells so the load spreads. More importantly, once a cell is starved, stop pretending. Show honest wait times rather than searching endlessly: a rider told "8 minutes" can decide to wait, and a rider shown a spinner tends to leave. Cost: this is a product-visible degradation, and it needs the pricing and supply systems to respond, which is outside this design but must be acknowledged.

Driver disappearing after accepting: heartbeats every few seconds, and if they stop for more than about 30 seconds, move the trip back to offering and tell the rider we are finding another driver. Cost: a false positive on a bad network cancels a driver who was actually on their way, so the threshold must be generous and the driver must be able to reclaim the trip if they come back before it is reassigned.

Idempotency, because mobile networks retry constantly: a ride request carries a client-generated key so a retry does not create two trips, and an accept carries the offer id so a duplicated accept is a no-op rather than a second assignment. Cost: an idempotency key store on the write path with a retention window. It is cheap, and mandatory here.

Consistency per feature: driver positions are stale by seconds on purpose and that is fine. The candidate list is a best-effort snapshot. Trip state is strongly consistent and durable, because it decides who gets paid. Saying those three things separately is the answer. One blanket consistency statement would be wrong for two of the three.

What I would monitor: time-to-match at p50 and p95, the rate of requests ending in no-driver-found, and offer acceptance rate per area. Acceptance rate falling in one area is an early signal of a supply problem, and it tends to show up before riders complain.

The costs named here
Batch offers after two timeouts → costs: Several drivers can accept; all but one lose. Needs a clean in-app experience for losing, and it briefly occupies drivers who will not get the trip.
Server-side offer expiry → costs: Per-offer state and timers to manage. The alternative is late accepts colliding with completed matches.
Generous heartbeat threshold before re-matching → costs: A driver who has truly vanished holds the trip for up to 30 seconds. Tightening it would cancel drivers who merely went through a tunnel.
Say this
"Accept is a conditional update on the trip row: assign only if still unassigned. So when I batch offers, the first accept wins and the others get a clean 'this trip is taken'. Cost: some drivers tap accept and lose, which needs real handling in the app. And offers expire server-side, because a frozen client accepting 40 seconds late would otherwise collide with a completed match."
Common trap
Designing a beautiful global optimiser that assigns drivers to riders to minimise total wait. It is the better algorithm and the wrong product. The rider is standing outside, and there is no slack to run it in. Say why you rejected it; that scores.
What an interviewer is ticking off in this stage
  • Offer protocol adapts under the deadline: sequential, then small batch
  • Accept is a conditional write; losing drivers get a clear message
  • Offers expire server-side, not just on the client
  • Bounded rounds with re-search between them, then an honest failure
  • Handled hot cells at peak and named the product-visible degradation
  • Heartbeat detection for a vanished driver, with a generous threshold
  • Idempotency keys on request and accept
  • Gave a per-feature consistency answer
  • Named what to monitor, and why acceptance rate is the leading indicator
  • A cost named after every decision

The decision this turns on

A
Greedy immediate matching
  • —Offer to the best available driver right now and commit.
  • —Match in seconds, which is the only budget that exists here.
  • —Globally worse assignments, with longer average pickups.
  • —Simple to reason about and to debug during an incident.
B
Batched optimisation
  • —Collect requests for 30–60 seconds, then assign the whole set optimally.
  • —Noticeably better assignments overall, and it can pair trips going the same way.
  • —Requires slack that ride hailing does not have: the rider is outside, waiting.
  • —Right for food delivery, where 15 minutes of cooking hides the delay completely.
Which one: Greedy, because there is no slack to optimise inside. This is one of the clearest examples in the curriculum of a same-shaped problem getting opposite designs: food delivery has 15 minutes of cooking to hide latency in, so batching wins there. Ask "where is the slack?" before choosing.
The design fork that separates ride hailing from every other matching problem.

And then they push back

A complete answer is where the interview starts, not where it ends. These are the questions this design invites — the interview is decided here far more often than in the diagram.

One key gets 90% of the traffic. What breaks, and what do you do?
A weak answer

"Consistent hashing spreads the load evenly." It spreads keys evenly, not traffic. All requests for one key go to one place no matter how the keys are hashed. This answer confuses two different problems.

A strong answer

"Whichever node owns that key is saturated while the rest idle, and no partitioning scheme fixes it because partitioning distributes keys, not requests for a single key. Three options. Replicate the hot key across N nodes with a suffix and read a random one. It costs N times the memory and a fanned-out invalidation, and it is the right answer for a read-hot key. For a write-hot key, shard the counter and sum on read, accepting that the total is now a computed value. Or handle it on a separate path: an in-process cache on every server absorbs a read-hot key almost completely. I would also detect hot keys automatically with a count-min sketch rather than maintaining a manual list, because yesterday's hot key is not today's."

A whole region is down. What do users see?
A weak answer

"We failover to another region." That is the mechanism, not the answer. The question is what a user experiences, for how long, and what is lost.

A strong answer

"I would answer per path rather than globally. Reads: if replicas exist elsewhere, they keep working, possibly stale, and I would say how stale. Writes: if the leader was in the dead region, writes fail until a new leader is promoted. That takes seconds to a minute, and being aggressive about promotion risks promoting during a brief network blip and ending up with two leaders. For contested resources like inventory I would rather be unavailable for that region's writes than risk selling the same thing twice, and I would say that explicitly as a choice. For non-contested writes like posting a comment, take it locally and reconcile. Data loss: with async replication, anything committed in the last few seconds and not yet replicated is gone, so I would use semi-synchronous replication for anything I cannot lose. And DNS-based failover is slow because clients cache DNS for minutes, so I would not rely on it alone."

Two users click at the same millisecond. What happens?
A weak answer

"We would use a lock." Which lock, held where, and what happens if the holder pauses? A lock is a mechanism, and reaching for it before asking whether the resource is contested at all is a reflex rather than an answer.

A strong answer

"First I would ask whether the thing is contested. If it is a like button, both writes land, the count converges, and I would choose not to spend a transaction on it. If it is the last seat, exactly one must win, and I get that by making the decision at a single point of serialisation: a conditional update that only succeeds if the seat is still free, or a unique constraint so the second insert simply fails. I am not proposing to read and then write in application code, because that gap is the race itself. The loser gets a clear message and an alternative, not an error. Cost: that path is now strongly consistent, so it is slower and it becomes unavailable if the owning partition is unreachable, which I accept, because selling one seat twice is worse than an error message. I would specifically not add a distributed lock, because the database row is already a single point of serialisation and a lock adds a failure mode where a paused holder's lease expires and two processes both think they won."

Why this database and not Postgres?
A weak answer

"It scales better and is designed for high throughput." Vague and unfalsifiable. Postgres handles far more than most people assume, and an answer with no number in it sounds like a preference.

A strong answer

"The answer has to be a number or an access pattern. Something like: the write rate is around 80,000 a second sustained on one logical table, append-only, with no cross-row transactions. That is more than I would plan to commit on one Postgres primary, so I would be sharding it myself and hand-building rebalancing and failover that a distributed store already has. The read pattern is a single-key lookup, so I am giving up joins I do not need. And then what it costs: no cross-partition transactions, so any invariant spanning partitions moves into application code, and analytics needs a separate copy of the data. If I cannot produce a sentence like that, then the correct answer is that Postgres is fine. Saying so is the stronger answer: choosing the boring option on purpose shows more judgement than reaching for the distributed one."

Which alert fires first when this system starts failing?
A weak answer

"We would alert on CPU and memory." Those are symptoms of the machine, not of the user experience, and they fire either far too late or constantly for no reason.

A strong answer

"I would alert on user-visible symptoms first and causes second. The first alert should be error rate or latency at p99 on the main user path, because that is what maps to someone being unhappy. Then a small number of leading indicators that fire before users notice: queue age (not depth), because age maps directly to how late the work is; replication lag; cache hit rate dropping, which precedes database saturation; and connection pool wait time, which is the earliest sign of a slow dependency. For this system specifically, the earliest signal is usually cache hit rate, because everything downstream is sized assuming the cache absorbs most reads. I would not page on CPU. It is a dashboard metric, not an alert, because a machine at 90% CPU serving requests fine is not an incident, and a machine at 20% CPU timing out is."

Now it must support scheduled actions — do this thing at 8am next Tuesday.
A weak answer

"We would use a cron job." Cron per user does not scale, does not survive a machine dying, and has no answer for what happens when a scheduled time is missed.

A strong answer

"I would store scheduled items in a table indexed by due time, and have workers poll for items due now. That gives durability, survives restarts, and lets me see and cancel what is pending, none of which cron gives me. The interesting parts are the failure branches. If the system is down at 8am, the item is late rather than lost, and I need a policy: run it late, or skip it, and that differs per action — a reminder is worth sending late, a market order is not. Duplicate execution must be harmless, so each scheduled item carries an id and execution is idempotent. Time zones and daylight saving cause bugs here: '8am next Tuesday' means 8am in someone's local zone, which is a different UTC instant depending on the date, so I store the intent rather than a resolved timestamp. And scheduled times cluster on round hours like 9am, so the due-time index gets hot and I would jitter execution by a small window where the exact second does not matter."

The cache just died. What happens?
A weak answer

"Requests would go to the database instead, so it would be a bit slower." At the ratios that justify a cache, the database cannot take that load. This answer means you added a cache without checking what it was protecting you from.

A strong answer

"Every read that was being served from memory lands on the database at once. My cache was absorbing something like 99% of reads, so the database sees roughly 100x its normal load, and it will not survive that. A naive design goes down. What I would do about it, designed in advance: coalesce misses so a thousand simultaneous requests for the same key produce one database read rather than a thousand; keep a small in-process cache on each server as a second layer so the hottest keys never leave the machine; shed load at the edge rather than let the database die, because a degraded site beats a dead one; and bring the cache back warm rather than empty, because restarting into an empty cache reproduces the outage. Cost of the in-process layer: servers can briefly disagree with each other."

There is an option that is 40% cheaper but 10% slower. Do you take it?
A weak answer

"I would take the faster one because user experience matters." Reflexive. 10% slower may be entirely invisible, and 40% is a large amount of money that could fund things users would actually notice.

A strong answer

"It depends where in the system, and I would answer with the slack budget. On the playback start path, where the budget is two seconds and users leave past it, 10% slower is meaningful and I would not take it. On the transcoding pipeline, where I have hours of slack, 10% slower is invisible and I would take the 40%. That is what the slack is for. So my answer is: take it everywhere the user cannot perceive the difference, refuse it on the paths where a human is waiting. Concretely that means interruptible capacity for batch work, cheaper storage tiers for cold data, and premium capacity only for the request path. And I would want to know whether the 10% is on the median or the tail, because a 10% worse p99 can be much more damaging than a 10% worse median."

The topics this answer used

Now do it yourself

You have read a good answer. That is worth very little until you can produce one on a blank page — so go and write this problem without looking, and compare afterwards.