Performance vs scalability
Fast for one person is not the same as staying fast for a million.
What problem does the scalability question solve?
These two words get used as if they mean the same thing. They do not. Mixing them up sends people off to fix the wrong problem.
Performance asks: is it fast right now, for one user? Scalability asks: does it stay just as fast when there are a hundred times more users?
A system can be fast and still not scale. It is quick with ten users and unusable with ten thousand. A system can scale and still be slow. Every request takes two seconds, whether you have ten users or ten million. At least it is predictable.
What the scalability question costs you
Every concept in here states a cost. A technique with no price is a technique you have not understood yet — and naming the cost out loud is most of what separates a senior answer from a junior one.
When to reach for the scalability question, and when not to
Use it when
- Someone says "it is slow". Ask whether it is slow for everyone, or only when busy. That one question tells you which problem you have.
- Deciding whether to speed up the code or add machines.
Do not, when
- Do not use this split as an excuse to skip measuring. Both questions are answered with real numbers, not with arguments.
How the scalability question actually works
- —Slow even with one user and no load.
- —Usually a missing index, an N+1 query, or work being done that need not be.
- —Adding machines does not help. Every machine is equally slow.
- —Fix: profile it, find the slow part, make it do less work.
- —Fine when quiet, falls apart when busy.
- —Something shared is running out: a database, a lock, a connection pool.
- —Response times climb as users climb, then fall off a cliff.
- —Fix: remove the shared bottleneck, or spread it across more machines.
The easy way to tell them apart is to look at a graph of response time against load. A performance problem is a flat line that sits too high. It is bad everywhere, equally. A scalability problem is a line that is fine, fine, fine, then bends sharply upward. That bend is the point where a shared resource ran out.
There are two ways to grow, and the names matter. Scaling up (vertical) means a bigger machine: more CPU, more memory. It is simple and needs no code changes. It has a ceiling, and each resize usually means a restart. Scaling out (horizontal) means more machines. The ceiling is far higher, but it forces you to keep state off your servers and to think about how they share data.
Scaling up is underrated. The machines you can rent today are huge, and one big database is far easier to run than twenty small ones. Reach for it first. Move to scaling out when you can name the limit you are hitting.
- Adding servers to fix a problem that is already there at zero load.
- Assuming a system that is fast today will stay fast. Speed now tells you nothing about the shape of the curve.
The the scalability question follow-up an interviewer will ask
"Your service is slow. Do you add more servers?"
Not before I know which problem it is. If it is slow with one user and no load, more servers change nothing. Every new server is equally slow, and I have spent money to make the same request take the same time. That is a performance problem, so I would profile one request and find where the time goes. It is usually a missing index or a call made inside a loop. If it is only slow under load, then something shared is running out, and I want to know what before adding capacity. If the bottleneck is the database, adding application servers makes it worse, not better, because they pile on more connections.
Your turn
Your app takes 3 seconds per request whether 1 person or 1,000 people are using it. Performance or scalability problem — and does adding servers help?
Learn it from the source
This page gets you interview-ready. These take you to where the idea is actually specified or measured — read them when you want the real depth.
- A Word on Scalability ↗Werner Vogels, Amazon CTO, 2006. The definition this page uses, from the person who had to live with it.
- Google SRE Book — Service Level Objectives ↗How Google turns "is it fast enough" into a number you can be held to.