Getting everything ready for you…

Loading courses, problems and your progress. This takes just a moment.

0s

Notes › System Design

System Design Basics: Scalability

Scalability = the system keeps performing well as load grows (more users, more data, more requests).

Vertical vs horizontal scaling

Vertical (scale up) Horizontal (scale out)
How bigger machine — more CPU, RAM, SSD more machines behind a load balancer
Limit hardware ceiling almost unlimited
Failure single point of failure one node dies, others serve
Complexity simple, no code changes needs stateless design, load balancing, data distribution
Cost gets expensive fast commodity machines
Example upgrading the DB server from 16 GB to 128 GB 2 → 20 Node.js servers

Real systems start vertical (cheap and simple) and go horizontal when needed.

Stateless servers

To scale out, any request must be able to go to any server. So don't keep user state (sessions, uploaded files) in server memory or local disk:

Key metrics

Metric Meaning
Latency time for one request (ms). Track p50, p95, p99, not just the average
Throughput requests handled per second (RPS/QPS)
Availability % of time the system works
Durability data is never lost
Consistency every read sees the latest write

The "nines" of availability

Availability Downtime per year
99% 3.65 days
99.9% 8.76 hours
99.99% 52.6 minutes
99.999% 5.26 minutes

Two components in series (both needed): 99.9% × 99.9% = 99.8%. Two in parallel (either works): 1 − 0.001² = 99.9999%. Redundancy raises availability.

Latency numbers to know (roughly)

Operation Time
L1 cache / RAM read ~1 ns / ~100 ns
Read 1 MB from RAM ~10 µs
SSD random read ~100 µs
Round trip in the same data centre ~0.5 ms
Read 1 MB from SSD ~1 ms
Disk (HDD) seek ~10 ms
Round trip India ↔ US ~150–250 ms

Memory ≫ SSD ≫ network ≫ disk — hence caching.

Growing an app: 1 user → millions

  1. Single server: app + DB on one machine.
  2. Separate the database onto its own server — scale them independently.
  3. Load balancer + multiple app servers (stateless) — no single point of failure for the app tier. See Load Balancing.
  4. Database replication: one primary for writes, read replicas for reads (most apps are read-heavy).
  5. Cache (Redis) for hot data and a CDN for static files — see Caching, CDN and Databases at Scale.
  6. Message queue for slow work (emails, video processing) — the request returns immediately; workers process later. See Message Queues.
  7. Sharding the database when one primary can't handle writes or storage — see Sharding and Replication.
  8. Multiple data centres / regions, geo-DNS routing users to the nearest one.
  9. Split into services (microservices) owned by different teams, plus logging, metrics and alerting.
Users → DNS → CDN (static)
          ↓
     Load Balancer
     ↙    ↓    ↘
  App1  App2  App3   (stateless, auto-scaled)
     ↘    ↓    ↙
  Redis cache   Queue → Workers
        ↓
  DB primary → read replicas (→ shards)

Single point of failure (SPOF)

Any component whose failure brings everything down. Remove SPOFs with redundancy (2+ load balancers, DB failover, multiple zones) and health checks.

Back-of-the-envelope estimation

Useful shortcuts: 1 day ≈ 86,400 s ≈ 10⁵ s; 1 million requests/day ≈ 12 RPS. Example — a photo app with 10 M daily active users:

Answering a system design interview

  1. Clarify requirements (5 min): functional (what it does) and non-functional (scale, latency, availability, consistency).
  2. Estimate scale: users, QPS, storage, bandwidth.
  3. API design: main endpoints.
  4. Data model: entities, SQL vs NoSQL.
  5. High-level design: boxes and arrows — clients, LB, services, DB, cache, queue.
  6. Deep dive into 1–2 hard parts (feed generation, ID generation, hot keys).
  7. Bottlenecks and trade-offs: SPOFs, scaling each tier, failure handling, monitoring. Say trade-offs out loud — there's no single correct answer.

Interview one-liners

More System Design notes