Notes › System Design
Design a Chat Application
A classic interview question: design something like WhatsApp or Messenger. Follow the usual steps: requirements → estimates → API → high-level design → deep dives → scaling.
1. Requirements
Functional
- 1-to-1 chat and group chat (up to ~256 members).
- Messages delivered in real time; stored for offline users and delivered when they come online.
- Sent / delivered / read receipts (✓, ✓✓, blue ✓✓).
- Online / last seen status, typing indicator.
- Send images/files; push notifications.
Non-functional
- Low latency (< 200 ms delivery when both are online).
- Highly available; messages never lost.
- Ordered within a conversation.
- Scale to hundreds of millions of users.
2. Estimates (back-of-envelope)
- 100 M daily active users × 40 messages/day = 4 billion messages/day ≈ 46,000/s average, ~150,000/s peak.
- Average message 100 bytes + metadata ≈ 200 bytes → 4 B × 200 B ≈ 800 GB/day of text (~290 TB/year before replication). Media goes to object storage.
- Concurrent connections: say 50 M → if one chat server holds
100,000 WebSocket connections → **500 chat servers**.
3. Why WebSockets
| Option | How | Verdict |
|---|---|---|
| Short polling | ask "any messages?" every few seconds | wasteful, slow |
| Long polling | request stays open until a message arrives | better, but a new request each time |
| WebSocket | one persistent two-way TCP connection | ✅ best for chat |
| Server-Sent Events | server → client only | good for feeds, not chat |
| Clients keep a WebSocket to a chat server; everything else (login, profile, history, media upload) uses normal HTTPS APIs. |
4. High-level design
HTTPS ┌──────────────┐
Client ───────────────► API servers ─────────────► │ User / group │ (SQL)
│ │ service │
│ WebSocket └──────────────┘
▼
Load balancer ──► Chat servers ◄──► Session store (Redis: userId → chat server)
│ ▲
▼ │ pub/sub or queue between chat servers
Message service ──► Message DB (Cassandra / ScyllaDB)
│
├──► Push service (FCM / APNs) for offline users
└──► Media: upload to S3 via pre-signed URL, CDN for downloads
Presence service (Redis, heartbeats)
5. Message flow — A sends "hi" to B
- A's app creates a client message id (UUID) and sends
{to: B, clientMsgId, text}over its WebSocket to chat server S1. - S1 assigns a server message id (sortable, e.g. Snowflake: time + machine + sequence), saves the message to the message DB, and replies to A with an ack → ✓ sent.
- S1 looks up B in the session store: B is connected to server S7.
- S1 forwards the message to S7 (Redis pub/sub channel per server, or a message queue — see Message Queues and Event-Driven Systems).
- S7 pushes it down B's WebSocket. B's app acks → A gets ✓✓ delivered.
- When B opens the chat, B sends a read receipt → A sees read.
- If B is offline: the message stays stored as undelivered; the push service sends a notification; when B reconnects, the app asks "give me everything after my last message id" and syncs.
Deduplication: if A's connection drops and A resends, the server sees the same clientMsgId and doesn't store it twice.
6. Data model
Messages are written a lot and read by conversation, newest first → a wide-column store like Cassandra/ScyllaDB/HBase fits (massive write throughput, partition by conversation).
messages
partition key : conversation_id
clustering key: message_id DESC (time-sortable)
columns : sender_id, type, text, media_url, created_at
conversation_members (conversation_id, user_id, last_read_message_id)
user_conversations (user_id, conversation_id, last_message_at) ← chat list screen
- 1-to-1 conversation id: e.g.
min(A,B) + "_" + max(A,B)so both sides find the same chat. - Users, groups, contacts, settings → a relational DB (strong consistency, smaller data).
- Media → object storage (S3) + CDN; the message just stores the URL. Clients upload directly with a pre-signed URL.
7. Group chat
- Small groups (≤ ~500): fan-out on write — the server looks up members and delivers to each online member's chat server; offline members get pushes and sync later. Store the message once per group, not once per member.
- Huge channels (thousands+): fan-out on read — members pull new messages when they open the channel.
- Read receipts in groups: track
last_read_message_idper member instead of per-message flags.
8. Presence (online / last seen)
- Clients send a heartbeat every ~30 s; Redis stores
user → last_heartbeatwith a TTL. - No heartbeat for > ~60 s → offline; "last seen" = last heartbeat time.
- Publish presence changes only to friends who are currently online (and batch them) — broadcasting to everyone would be enormous.
- Typing indicators are fire-and-forget: not stored.
9. Ordering
Ordering is needed within a conversation, not globally. Use per-conversation sequence numbers or time-sortable ids assigned by the server; clients sort by them. Partitioning queues/DB by conversation_id keeps one chat's events together.
10. Reliability and scaling
- Chat servers are stateful (they hold connections): if one dies, clients reconnect (with backoff + jitter) through the LB to another server and sync missed messages by last message id.
- Session store and presence in a Redis cluster; message DB replicated (RF = 3) across zones (Database Sharding and Replication).
- Rate-limit sending to stop spam; validate message size.
- Multi-region: users connect to the nearest region; cross-region messages go through replicated queues.
11. Security
- TLS on every connection.
- End-to-end encryption (Signal protocol): keys live on devices; servers store and forward ciphertext only — so server-side search and moderation become harder.
- Authenticate the WebSocket with a token during the handshake.
12. API sketch
POST /api/v1/media/upload-url → pre-signed URL
GET /api/v1/conversations?cursor=… → chat list
GET /api/v1/conversations/:id/messages?before=<msgId>&limit=50 → history (cursor pagination)
WS send: {type:"message", to, clientMsgId, text}
WS receive: {type:"message", id, from, conversationId, text, sentAt}
WS receive: {type:"receipt", messageId, status:"delivered"|"read"}
Trade-offs to mention
| Decision | Choice | Why |
|---|---|---|
| Transport | WebSocket | real-time, two-way |
| Message store | Cassandra | huge write volume, query by conversation |
| Delivery | at-least-once + client dedupe | never lose messages |
| Groups | fan-out on write for small groups | fast delivery |
| Presence | heartbeats + TTL in Redis | cheap, self-expiring |
Interview one-liners
- WebSockets for real-time; HTTPS for everything else.
- A session store maps each user to the chat server holding their connection.
- Store first, then deliver; offline users sync by last message id.
- Partition messages by conversation id; order within a conversation.
- Related: Design a URL Shortener, Load Balancing, CAP Theorem and Consistency.