15:I[71953,["/_next/static/immutable/chunks/40bva27ovyvmm.js","/_next/static/immutable/chunks/2s36npum74o8y.js","/_next/static/immutable/chunks/2f2bp2lg0w4uq.js","/_next/static/immutable/chunks/3h38cxtcfwvqe.js","/_next/static/immutable/chunks/40zn0rwmi74gr.js","/_next/static/immutable/chunks/1un3ttr3ry4yy.js","/_next/static/immutable/chunks/19k7ec3wwj9uc.js","/_next/static/immutable/chunks/42fqcaagr7aqh.js"],"StudyApp"] 16:Te81,## Requirements Send and receive messages in 1:1 and group conversations with delivery and read receipts; show presence (online, last seen); sync history across a user's devices; notify offline users. Online delivery should feel instant (under 100 ms in-region), nothing may be lost, and messages within a conversation must appear in the same order for everyone. ## Estimates 500 million DAU × 40 messages ≈ 2 × 10^10 messages/day ≈ 230,000/s average, ~600,000/s peak. With ~3 recipients per message, ~2 million deliveries/s at peak. 10–20% online at once → 50–100 million persistent connections. Text storage ~4 TB/day; media (via object storage) 100× that. ## API A WebSocket (or similar) channel carries `send`, `ack`, `receipt`, `typing` and `presence` frames. REST endpoints cover `GET /conversations`, `GET /conversations/{id}/messages?before=`, `POST /media/upload-url`. Clients attach a locally generated message id to each send so retries are idempotent. ## Data model `messages` partitioned by `(conversation_id, time_bucket)` and clustered by `message_id`, so "load the last 50 messages in this chat" is one partition read in time order. `conversations(id, type, created_at)`, `members(conversation_id, user_id, joined_at, last_read_message_id)`. A presence/routing map `user_id → { gateway_id, device_ids, last_seen }` in Redis. ## High-level design ```mermaid flowchart LR A[Alice's app] <--> G1[Gateway 1] B[Bob's app] <--> G2[Gateway 2] G1 --> CS[Chat service] CS --> DB[(Message store)] CS --> RT[Router] RT --> P[(Presence / routing map)] RT --> G2 RT --> PN[Push notification service] ``` Alice's gateway forwards her message to the chat service, which validates membership, assigns a server-side message id and timestamp, persists it, and hands it to the router. The router looks up each recipient's gateway and pushes the message; recipients not connected get a push notification and will fetch on reconnect. Alice's gateway returns an ack with the server id so her client can mark it "sent". ## Deep dives **Ordering.** Assign ids per conversation from a single writer (a per-conversation sequencer or the partition owner) so every client sorts by the same id. Time-ordered ids (Snowflake) are a reasonable approximation when strict per-conversation sequencing is too costly. **Group fan-out.** For groups up to a few hundred members, the router expands membership and pushes to each. For huge channels (tens of thousands), pushing is wasteful; instead notify members that the channel has new messages and let clients pull when the channel is open, Slack-style. **Sync across devices.** Each device tracks the last message id it has seen per conversation and requests everything newer on reconnect. Because history is in a partitioned store keyed by conversation and time, this is a cheap range read. **Presence.** Clients heartbeat every ~30 s; the gateway updates `last_seen`. Friends subscribe to presence changes through a pub/sub channel; publish only on transitions, not on every heartbeat. ## Bottlenecks and failure modes - **Gateway crash:** tens of thousands of clients reconnect at once. Use jittered reconnect and let them resync from last acked id; nothing is lost because messages are persisted before delivery. - **Hot conversation:** a 50,000-member group is a hot partition for writes and a fan-out storm for reads; the pull model for large channels and time-bucketing the partition key handle it. - **Duplicates:** at-least-once delivery means a client may see a message twice; dedupe by message id. - **Media:** never through the chat path. Upload to object storage via presigned URL, send a message containing the reference.17:T1027,## Requirements A user installs a client on several devices; any file saved into the synced folder appears on the others within seconds. Files can be gigabytes; bandwidth and storage should be used efficiently; edits made offline must reconcile without losing anything; users can view history and share folders. Durability must be essentially perfect. ## Estimates 100 million users with ~200 files averaging 500 KB is ~10 PB of logical data. Deduplication (the same installer, the same photo forwarded around) cuts that meaningfully; replication or erasure coding multiplies it back. Request volume is dominated by small metadata operations (listing changes, committing versions), not bytes. ## API ```text GET /namespaces/{ns}/changes?cursor=... -> [{ path, version, chunkHashes[], deleted }], nextCursor POST /namespaces/{ns}/commit { path, parentVersion, size, chunkHashes[] } -> { version } | 409 conflict POST /chunks/presign { hashes[] } -> { missing: [{ hash, uploadUrl }] } PUT (raw bytes) GET ``` ## Data model `files(id, namespace_id, path, version, size, chunk_hashes[], modified_at, modified_by_device, deleted)` plus an append-only `journal(namespace_id, seq, file_id, version)` that clients page through by cursor. `chunks(hash PK, size, storage_key, refcount)` enables dedup. Namespaces are a user's root or a shared folder, and are the sharding unit. ## High-level design ```mermaid flowchart LR subgraph client[Client] W[FS watcher] --> CH[Chunker + hasher] CH --> IDX[(Local index)] end CH -->|missing chunks| BS[(Block storage)] CH -->|commit| MS[Metadata service] --> MDB[(Metadata DB, sharded by namespace)] MS --> J[(Change journal)] MS --> NS[Notification service] NS -->|wake up| C2[Other devices] C2 -->|GET changes, GET chunks| MS & BS ``` A device detects a change, chunks the file, asks which chunk hashes the server lacks, uploads only those, then commits the new file version. The server validates the parent version, appends to the journal and pings the user's other devices, which fetch the change list and download missing chunks. ## Deep dives **Chunking.** Fixed-size chunks break on inserts: adding one byte at the start shifts every boundary. Content-defined chunking uses a rolling hash to place boundaries at content patterns, so an edit only changes the chunks around it. Chunk hashes (SHA-256) are the content addresses used for dedup and integrity. **Conflicts.** Each commit carries the `parentVersion` the client based its edit on. If the server's current version differs, another device committed first; the server rejects with `409`, and the client saves its version as `report (conflicted copy from laptop).docx` and commits that as a new file. No data is ever lost; the user merges manually. Version vectors can refine this for multi-master offline edits. **Dedup and GC.** Chunks are shared across files and users via refcounts. Deleting a file decrements counts; a background collector removes chunks whose count hits zero after a grace period (so a quick undelete is free). **Notifications.** Long polling per namespace works well: a device holds a request open; when the journal for any of its namespaces advances, the server responds and the device syncs. WebSockets are an alternative, but long polling survives corporate proxies better. ## Bottlenecks and failure modes - **Metadata hot spots:** a user with millions of files makes listing and journal reads heavy; paginate aggressively and shard by namespace. - **Large-file uploads:** resumable by design, because a retry asks "which chunks are missing" and uploads only those. - **Notification storms:** a shared folder with 10,000 members gets one change → 10,000 wakeups; batch notifications and let clients debounce. - **Clock and ordering:** never use client timestamps for conflict decisions; use server-assigned versions. - **Durability:** chunks are immutable, which makes replication and verification simple; periodically re-hash stored chunks to detect bit rot.18:Te41,## Requirements Applications call `get`, `set` (with TTL), `delete`, a few atomic operations and multi-key reads, and expect sub-millisecond responses. The cache must hold terabytes across many nodes, keep working when a node fails, and allow adding capacity without flushing everything. It is a performance layer, not a system of record: losing it must degrade latency, not correctness. ## Estimates Caching 500 GB of hot objects averaging 1 KB means ~500 million keys. With ~50 GB usable per 64 GB node, that is ~10 primaries plus a replica each. Two million gets per second cluster-wide is ~200,000 per node, which a single-threaded Redis or multi-threaded Memcached node can serve. ## API ```text get(key) -> value | nil mget(keys[]) -> values[] set(key, value, ttl) delete(key) incr(key, n) cas(key, value, version) // compare-and-swap cluster.slots() -> [{ range, primary, replicas }] ``` ## Data model Each node is an in-memory hash table from key to `{ bytes, expiresAt, lruClock }`. Cluster metadata maps hash slots (Redis uses 16,384) or ring positions to a primary node and its replicas. Clients cache this map and refresh it on `MOVED` responses or topology change notifications. ## High-level design ```mermaid flowchart LR APP[Application + smart client] -->|hash(key) → slot → node| N1[Shard 1 primary] APP --> N2[Shard 2 primary] APP --> N3[Shard 3 primary] N1 --> R1[Replica] N2 --> R2[Replica] N3 --> R3[Replica] CFG[(Topology / health)] --> APP APP -- miss --> DB[(Database)] ``` Nodes do not talk to each other on the request path; the client (or a proxy such as mcrouter) picks the node. Replication within a shard is asynchronous; on primary failure the replica is promoted and the topology update propagates. ## Deep dives **Eviction and memory.** Exact LRU costs a linked list per key; Redis samples a handful of keys and evicts the least recently used among them, which approximates LRU well. Memcached uses slab classes to avoid fragmentation. Always set `maxmemory` and a policy so the node never swaps. **Stampede prevention.** When a hot key expires, thousands of requests miss simultaneously and all hit the database. Options: *single-flight* (one request per key fetches, others wait), *leases* (the cache hands a token to the first misser and tells others to retry briefly), *stale-while-revalidate* (serve the expired value while one request refreshes), or *probabilistic early expiration* so refreshes spread out before the TTL. **Hot keys.** A single key read a million times per second saturates one node. Add a short-TTL in-process L1 cache in the application, or replicate the key under several suffixes (`key#1`..`key#N`) and read a random one. **Consistency contract.** Cache-aside: the app reads cache, on miss reads the database and sets the cache; on write it updates the database and *deletes* the cache key. Document the staleness window and use leases to prevent the stale-set race. ## Bottlenecks and failure modes - **Node loss:** the shard's keys all miss until refilled; replicas limit this, and request coalescing keeps the database alive during warm-up. - **Resharding:** adding nodes with consistent hashing or slot migration moves only a fraction of keys; migrate slots live and have clients follow `ASK`/`MOVED` redirects. - **Partitions:** a cache should favour availability; a partitioned replica serving slightly stale data is better than errors. Never make the cache authoritative for anything. - **Large values:** a 10 MB value blocks a single-threaded node; cap value sizes and compress or chunk big objects.19:Tef3,## Requirements A store exposing `get`, `put` and `delete` on opaque keys with values up to about a megabyte. It must scale by adding commodity nodes, keep serving reads and writes when nodes fail or the network partitions, offer single-digit-millisecond latency, and let callers choose between stronger consistency and lower latency per request. ## Estimates 100 TB of logical data at replication factor 3 is 300 TB on disk. With 2–4 TB usable per node (leaving room for compaction), that is 100–150 nodes. One million operations per second across the cluster is ~10,000 per node, which an LSM-based node handles comfortably. ## API ```text put(key, value, context?) -> ok get(key, consistency=QUORUM) -> [{ value, version }] // may return siblings delete(key) ``` `consistency` is `ONE`, `QUORUM` or `ALL`, trading latency and availability for recency. ## Data model Keys and values are bytes. Each stored value carries a version (vector clock or timestamp), optional TTL, and a tombstone flag for deletes. Cluster metadata holds the ring (token ranges per node) and the replication factor. ## High-level design ```mermaid flowchart LR C[Client] --> N1[Node A, coordinator] N1 -->|hash key → ring| N2[Replica 1] N1 --> N3[Replica 2] N1 --> N4[Replica 3] N1 -. gossip .- N2 N2 -. gossip .- N3 N3 -. gossip .- N4 ``` Any node can coordinate. It hashes the key onto the ring, identifies the N replicas (the next N distinct nodes clockwise), sends the request to all, and responds once W (writes) or R (reads) have acknowledged. Membership and liveness spread by gossip; each node runs a failure detector. ## Deep dives **Partitioning.** Consistent hashing with virtual nodes: each physical node owns many small token ranges, so adding a node pulls a sliver from everyone and a failed node's load spreads across the cluster rather than onto one neighbour. **Quorums.** With N = 3, W = 2, R = 2, every read overlaps every successful write on at least one replica. W = 1 gives the fastest, most available writes at the cost of possibly reading stale data; W = 3 makes writes fail if any replica is down. Expose this per request. **Conflicts.** Under availability-first writes, two replicas can accept different values for the same key. Vector clocks (one counter per coordinator) let the store detect concurrency and hand both versions ("siblings") back to the client to merge; shopping carts are the classic example. Last-writer-wins by timestamp is simpler and what most deployments choose, accepting silent loss on true conflicts. **Repair.** *Read repair* writes the newest version back to stale replicas on each read. *Anti-entropy* compares Merkle trees between replicas in the background and streams differing ranges. *Hinted handoff* keeps a write destined for a down replica on another node and delivers it when the replica returns. **Storage engine.** Writes append to a commit log and an in-memory memtable; memtables flush to immutable sorted files (SSTables). Reads check the memtable, then SSTables newest-first, skipping files whose Bloom filter says the key is absent. Compaction merges SSTables and drops tombstones. ## Bottlenecks and failure modes - **Node loss:** replicas keep serving; a replacement node streams its ranges from peers. - **Hot keys:** one celebrity key overwhelms its three replicas; cache at the client or salt the key into several sub-keys. - **Latency spikes:** compaction I/O and JVM-style pauses show up at p99; throttle compaction and size heaps carefully. - **Tombstone accumulation:** heavy deletes slow reads until compaction; set `gc_grace` thoughtfully. - **Split brain:** without a leader there is none to split, but a partitioned minority will accept writes that must be reconciled later; that is the chosen tradeoff and you should say so.1a:Tff0,## Requirements Collect numeric metrics (CPU, request latency, queue depth, business counters) from tens of thousands of hosts and services. Engineers query them over time ranges with aggregation (sum by service, p99 by region) for dashboards, and define alert rules ("error rate > 1% for 5 minutes") that notify on-call. Ingest is millions of points per second; recent-data queries must return in under a second; retention is years, at decreasing resolution. ## Estimates 100,000 hosts × 100 series × one sample per 10 s = **1 million samples/s**. A raw sample (timestamp + float64) is 16 bytes → 16 MB/s, ~1.4 TB/day. Gorilla-style compression gets this to ~1–2 bytes per sample, so ~100–200 GB/day of hot storage, and downsampled tiers are a fraction of that. Series count (10 million here) matters as much as sample rate, because every series needs an index entry and an open write buffer. ## API Ingestion either pushes (`POST /write` with batches of `name{labels} value timestamp`) or pulls (the system scrapes `/metrics` endpoints on a schedule). Queries use a language with selectors and functions (`rate(http_errors_total{service="api"}[5m])`), range and instant queries, and aggregation operators. Alert rules are stored expressions with a duration and severity; dashboards are saved query layouts. ## Data model A series is identified by its name and sorted label pairs; hash them to a series id. An inverted index maps each `label=value` to the set of series ids so queries like `service="api"` resolve to series quickly. Samples are stored in time-partitioned blocks (say two hours) per series, compressed. Downsampled tiers store per-interval `count, sum, min, max` (and sketches for percentiles) with their own retention. ## High-level design ```mermaid flowchart LR A[Agents / scrapers] --> IG[Ingest gateway: validate, limit cardinality] IG --> K[(Stream)] K --> W[TSDB writers, sharded by series hash] W --> HOT[(Hot store, SSD, raw ~15 days)] HOT --> CP[Compactor + downsampler] CP --> COLD[(Cold store, object storage, rollups)] Q[Query service] --> HOT & COLD D[Dashboards] --> Q AE[Alert evaluator] --> HOT AE --> N[Notification system] ``` ## Deep dives **Compression.** Timestamps in a series arrive at near-regular intervals, so store the delta of deltas, which is usually zero and encodes in a bit or two. Values change slowly, so XOR each with the previous and store only the differing bits. Gorilla reported ~1.37 bytes per sample with this, enabling weeks of data in memory. **Downsampling and retention.** A background job reads raw blocks and writes 1-minute and 1-hour rollups. Policies such as raw for 15 days, 1-minute for 90 days, 1-hour for 2 years keep storage bounded. Queries over long ranges automatically read the coarsest tier that satisfies the requested resolution. **Cardinality control.** A label like `user_id` turns one metric into millions of series and sinks the index. Enforce per-tenant series limits at the gateway, reject or drop high-cardinality labels, and surface the top offenders so teams fix instrumentation. **Alerting independence.** Evaluators run rules every N seconds against the hot store, keep state for "for 5 minutes" conditions, and hand firing alerts to the notification system with grouping and deduplication. Keep this path small and separate from dashboard queries so a heavy ad hoc query cannot delay a page. ## Bottlenecks and failure modes - **Ingest spikes** (a deploy restarts everything at once): the stream absorbs the burst; writers catch up. - **Query abuse:** an unbounded `rate(...)` over a year across all series must be refused or chunked; apply query limits and cache panel results. - **Writer failure:** series are sharded across writers with replication; a lost writer's shards are replayed from the stream. - **Self-monitoring:** the metrics system must not depend on itself; run a tiny independent monitor that alerts if ingestion stops. - **Clock skew:** samples with far-future or far-past timestamps are rejected to protect block boundaries.1b:Te11,## Requirements Users publish posts (text plus optional media), follow other users, and open a feed showing posts from people they follow, newest first or ranked. The feed must load in well under a second and a new post should appear for followers within a few seconds. Reads vastly outnumber writes. ## Estimates Take 300 million DAU, 2 posts/user/day → 6 × 10^8 posts/day ≈ 7,000 posts/s. Ten feed opens per user per day → 3 × 10^9 reads/day ≈ 35,000 feed reads/s, ~100,000 at peak. With an average of 200 followers, pushing every post to every follower is 7,000 × 200 = 1.4 million feed-cache writes per second. That number is why the design has a decision to make. ## API ```text POST /posts { text, mediaIds[] } -> { postId } GET /feed?cursor=... -> { items: [post...], nextCursor } POST /users/{id}/follow ``` ## Data model `posts(id, author_id, text, media_refs, created_at)` sharded by post id (Snowflake ids encode time, making recent-post scans cheap). `follows(follower_id, followee_id)` indexed both ways: "who do I follow" and "who follows me". The **feed cache** is per user: a sorted list of `(post_id, score)` capped at a few hundred entries, in Redis sorted sets or a wide-column table. ## High-level design ```mermaid flowchart LR U[Client] --> GW[API gateway] GW --> PS[Post service] --> PDB[(Posts)] PS --> EV[(Post events)] EV --> FO[Fan-out workers] FO --> FC[(Feed cache per user)] GW --> FS[Feed service] FS --> FC FS --> PC[(Post cache)] FS --> RK[Ranker] ``` Writing a post stores it and emits an event. Fan-out workers look up the author's followers and insert the post id into each follower's feed list. Reading a feed fetches the user's list, hydrates the post objects from a post cache, ranks them, and returns a page with a cursor. ## Deep dives **Fan-out on write vs on read.** Push (precompute every follower's feed) makes reads a single cache lookup but costs writes proportional to follower count. Pull (at read time, fetch recent posts from every followee and merge) makes writes trivial but reads expensive for users who follow many accounts. The hybrid: push for authors under a threshold (say 10,000 followers), and for celebrities above it do not fan out; instead the feed service pulls their recent posts at read time and merges them into the precomputed list. Twitter described exactly this split. **Feed storage.** A sorted set per user keyed by post id with a score of creation time (or rank). Trim to the newest 500 on insert. Memory: 300M users × 500 entries × ~16 bytes ≈ 2.4 TB across a cache cluster, acceptable; skip inactive users to cut it. **Ranking.** Keep retrieval and ranking separate. Retrieval returns a few hundred candidates; a ranking service scores them with features (recency, author affinity, engagement velocity) and the client shows the top N. If the ranker is slow or down, fall back to chronological. **Own posts.** Insert the author's own post into their feed synchronously so they see it immediately; everyone else can wait a few seconds. ## Bottlenecks and failure modes - **Celebrity storms:** handled by the pull path above. - **Inactive users:** fan-out to someone who has not opened the app in a month is wasted work. Skip them and rebuild their feed by pulling when they return. - **Feed cache loss:** rebuild lazily on read from followees' recent posts; degrade to chronological pull. - **Hot posts:** engagement counters (likes, comments) on viral posts are a write hotspot; aggregate them asynchronously and cache them separately from the post body.1c:Te7c,## Requirements Internal services (orders, social, security) ask the notification service to tell a user something. The service chooses channels based on the user's preferences and the notification type, renders content, delivers through third-party providers, retries failures, avoids duplicates, respects quiet hours and rate limits, and records delivery status. Transactional notifications (login codes, payment receipts) must be fast and reliable; marketing can wait. ## Estimates Say 10 million push, 1 million email and 100,000 SMS per day: tens to a few hundred per second on average, but events like "your favourite team scored" create bursts of tens of thousands per second. Each notification logs ~1 KB → ~10 GB/day, keep 90 days. ## API ```text POST /v1/notifications { userId, type: "order_shipped", data: {...}, channels?: [...], priority: "high"|"normal"|"low", idempotencyKey } -> { notificationId, status: "queued" } GET /v1/notifications/{id} GET/PUT /v1/users/{id}/preferences ``` ## Data model `devices(user_id, platform, token, last_seen)`, `contacts(user_id, email, phone, verified)`, `preferences(user_id, type, channel, enabled, quiet_hours)`, `templates(type, channel, locale, body)`, and a `notification_log(id, user_id, type, channel, status, provider_message_id, created_at, delivered_at, error)`. ## High-level design ```mermaid flowchart LR S[Internal services] --> API[Notification API] API --> PREF[Preferences + dedup + rate limit] PREF --> LOG[(Notification log)] PREF --> QP[(Push queue)] PREF --> QE[(Email queue)] PREF --> QS[(SMS queue)] QP --> WP[Push workers] --> APNS[APNs / FCM] QE --> WE[Email workers] --> SES[Email provider] QS --> WS[SMS workers] --> TW[SMS provider] WP & WE & WS --> LOG ``` The API validates the request, checks the idempotency key, resolves the user's channels from preferences, writes a log row per channel with status `queued`, and enqueues one message per channel. Channel workers render the template in the user's locale, call the provider, and update the log. Failures retry with exponential backoff; after N attempts the message goes to a dead-letter queue for inspection. ## Deep dives **Provider abstraction.** Each channel worker talks to an interface (`send(message) → providerId | error`) with multiple implementations. Track per-provider error rates and latency; on degradation, shift traffic to a backup vendor automatically. **Priorities.** Separate queues (or priority lanes) per urgency. A marketing blast of 5 million emails must never delay a 2FA code. Reserve worker capacity for the high-priority lane. **Deduplication and rate limiting.** The idempotency key stops duplicate requests from the same producer. A per-user limiter ("at most 5 marketing pushes a day") and collapse keys ("replace the older 'new message' notification") stop fatigue. Digest mode buffers low-priority events and sends one summary per hour. **Templates.** Store templates with placeholders; render at send time so copy changes do not need code deploys, and version them so the log can show exactly what was sent. ## Bottlenecks and failure modes - **Broadcast fan-out:** one event → millions of notifications. Expand the audience in a fan-out worker that enqueues in batches with throttling, rather than inside the API request. - **Provider outage:** messages queue up; retries with backoff plus vendor failover. Monitor queue age, not just depth. - **Invalid device tokens:** APNs and FCM report them; prune on feedback or every push wastes a call. - **Provider rate limits:** SMS vendors throttle per number; shard sends across sender pools and respect their limits with a token bucket per provider.1d:T105d,## Requirements Customers pay for orders with cards or wallets; the platform holds funds and pays merchants out on a schedule; refunds and disputes flow back. Money must move exactly once, balances must be exactly right, every movement must be auditable, and card data should never touch the platform's servers. Throughput is modest; correctness is everything. ## Estimates One million orders per day is ~12 payments per second average, perhaps a few hundred per second during a flash sale. Payouts are batched daily. This is a small system by volume, which is why the design spends its complexity on consistency and reconciliation rather than scale. ## API ```text POST /payments Idempotency-Key: { orderId, amount, currency, paymentMethodToken } -> 201 { paymentId, state: "pending" | "succeeded" | "failed" } GET /payments/{id} POST /payments/{id}/refunds Idempotency-Key { amount } POST /webhooks/psp (signed events from the provider) ``` ## Data model `payments(id, order_id, amount_minor, currency, state, psp_reference, idempotency_key, created_at, updated_at)` with a `payment_events` log of every transition. A double-entry `ledger_entries(id, txn_id, account_id, debit_minor, credit_minor, currency, created_at)` table where each `txn_id` groups entries whose debits equal credits. Accounts include customer receivables, platform cash at PSP, merchant payable, fees and refunds. Amounts are integers in minor units; never floats. ## High-level design ```mermaid flowchart LR CO[Checkout] --> PS[Payment service] PS --> IDK[(Idempotency keys)] PS --> PDB[(Payments + state log)] PS --> PSP[Payment provider] PSP -->|webhook| PS PS --> LED[Ledger service] --> LDB[(Double-entry ledger)] PS --> EV[(Payment events)] EV --> PO[Payout scheduler] --> PSP REC[Reconciliation job] --> LDB & PSP ``` ## Deep dives **Idempotency end to end.** Store the key and request hash *before* calling the PSP; on a repeat, return the stored outcome. Pass your own idempotency key to the PSP so their retries are safe too. Webhooks carry event ids; process each id once. **The unknown-outcome problem.** The PSP call times out. Did the charge happen? Never retry blindly (that risks a double charge) and never assume failure (that risks an unpaid order). Mark the payment `pending_unknown`, then resolve by querying the PSP with your reference, or wait for the webhook, with a reconciliation sweep for anything stuck. **Double-entry ledger.** Every money movement is a transaction with at least two entries whose debits equal credits: a customer payment debits "cash at PSP" and credits "merchant payable" and "platform fees". Entries are never updated or deleted; corrections are new entries. Balances are sums over entries (cached with a version for speed, recomputed for audits). This gives you an audit trail by construction and makes "where did the money go" a query. **State machine.** `created → pending → succeeded | failed`, with `succeeded → partially_refunded → refunded` and `disputed` branches. The payment service is the only writer, and every transition is validated and logged. **PCI scope.** The browser or app sends card details straight to the PSP, which returns a token. The platform only ever stores the token, keeping most of PCI DSS out of scope. ## Bottlenecks and failure modes - **PSP outage:** accept the order, hold the payment as pending, retry with the same key, and tell the customer honestly. Consider a second PSP for failover, routed by BIN or region. - **Duplicate webhooks and out-of-order events:** idempotent by event id; apply transitions only if valid from the current state. - **Reconciliation gaps:** nightly compare your ledger against PSP settlement reports; every mismatch is an alert and a ticket with tooling to post correcting entries. - **Refund after payout:** the merchant's payable goes negative; net it against future payouts or claw back per contract, all as ledger entries. - **Hot merchant balance row:** many concurrent payments to one merchant contend on a cached balance; append entries freely and update the cached balance asynchronously or with optimistic versioning.1e:Tcad,## Requirements Enforce limits such as "1,000 requests per minute per API key" and "10 login attempts per hour per IP" consistently across every API server. Checks must add about a millisecond, the limiter must not become the outage, and clients need enough information to back off intelligently. Rules change without redeploys. ## Estimates If the API fleet serves 100,000 requests/s, the limiter performs 100,000 checks/s. Each check is one small Redis command on one key, which a modest Redis cluster handles. With 10 million active keys at ~100 bytes each, counters take ~1 GB of memory. ## API Internally the middleware calls `allow(key, rule) → { allowed, remaining, resetAt }`. Externally a rejected request gets `429 Too Many Requests` with `Retry-After: ` and `RateLimit-Limit`, `RateLimit-Remaining`, `RateLimit-Reset` headers on every response. ## Data model Rules live in a config store and are cached in every server: `{ scope: apiKey | ip | user, endpoint pattern, limit, window, burst }`. Counters live in Redis keyed by `rl:{ruleId}:{clientKey}` with a TTL slightly longer than the window so idle keys disappear. ## High-level design ```mermaid flowchart LR C[Client] --> GW[API gateway + limiter middleware] GW -->|allow?| R[(Redis cluster, sharded by key)] GW --> S[Backend services] CFG[Rules config] --> GW GW --> M[Metrics] ``` The gateway loads rules, hashes the client key to a Redis shard, and runs an atomic script. Allowed requests proceed; rejected ones return 429 immediately. ## Deep dives **Token bucket in Lua.** Store `tokens` and `last_refill` in a hash. The script reads both, adds `rate × elapsed` tokens up to `capacity`, and if `tokens ≥ 1` decrements and returns allowed. One round trip, atomic, and the bucket's state is two numbers. Use Redis `TIME` inside the script so app server clocks do not matter. **Sliding window counter.** Keep counts for the current and previous fixed windows. Estimate = `current + previous × (overlap fraction)`. Two integers per key and smooth behaviour at boundaries; slightly approximate, which is fine for API quotas. **Local pre-limiting.** A single abusive client can generate more checks than Redis wants to see. Each server keeps a tiny in-memory limiter per key as a first filter and only consults Redis when the local limiter allows. **Multiple rules.** A request may match several (per key, per endpoint, per IP). Evaluate all; reject if any fails; report the most restrictive remaining count. ## Bottlenecks and failure modes - **Redis down.** Decide explicitly: fail open (serve traffic, protect yourself with a local backstop) for a bounded time with loud alerts, or fail closed for abuse-sensitive endpoints like login. - **Hot shard.** One client key always hashes to one shard; if a single client is 20% of traffic, that shard is hot. Local pre-limiting and giving huge clients dedicated limits mitigate it. - **Accuracy vs latency.** A central store is exact; local-only is fast but allows N× the limit across N servers. The hybrid (lease batches of tokens) is the usual production answer. - **Observability.** Count throttled requests per rule and per client; a sudden rise usually means a client bug or an attack.1f:Tf9c,## Requirements Drivers continuously report their location and availability. A rider enters pickup and destination and gets matched with a nearby driver within seconds. Both parties see each other's position live until pickup; the trip proceeds through states until completion and payment. The system must be highly available per city, and a driver must never be assigned two trips at once. ## Estimates One million online drivers reporting every 4 seconds is **250,000 location writes per second**, far more than any relational database should see on a hot path. Trips are tiny by comparison (tens of thousands per minute worldwide). Reads for matching (nearby lookups) and tracking (riders polling or subscribed to one driver) are in the millions per second but local to a region. ## API Drivers hold a persistent connection: they stream `location { lat, lng, heading, status }` frames and receive `offer { tripId, pickup, eta, expiresAt }` frames to accept or decline. Riders call `POST /trips { pickup, destination }` and subscribe to trip updates; `GET /trips/{id}` returns the current state, driver position and ETA; `POST /trips/{id}/cancel`. ## Data model Live state lives in memory, partitioned by region: `driver:{id} → { cell, lat, lng, status, updatedAt }` and `cell:{h3} → set(driverIds)`. Durable state in a database: `trips(id, rider_id, driver_id, state, pickup, dropoff, requested_at, matched_at, started_at, ended_at, fare)`. Raw location history is appended to a stream for analytics and dispute resolution, not written to the trip database. ## High-level design ```mermaid flowchart LR D[Driver app] <--> GW[Gateway] R[Rider app] <--> GW GW --> LOC[Location service] LOC --> GEO[(In-memory geo index per region)] GW --> TRIP[Trip service] --> TDB[(Trips DB)] TRIP --> MATCH[Matching / dispatch] MATCH --> GEO MATCH --> ETA[Routing / ETA service] MATCH --> PRICE[Pricing] TRIP --> NOTIF[Push to clients] LOC --> LOG[(Location history stream)] ``` ## Deep dives **Geo index.** Convert each location to an H3 (or geohash) cell at a resolution around 0.5–1 km. Updating a driver means removing them from the old cell set and adding to the new one: two in-memory operations. Matching queries the pickup cell and its ring of neighbours, yielding tens of candidates, then computes exact distance and asks the routing service for ETAs. **Dispatch.** Rank candidates by ETA (and acceptance history, fairness). Atomically set the best driver's status from `available` to `offered` (compare-and-set); if that fails, try the next. Send the offer with a 10–15 s expiry. On accept, transition the trip to `matched` and the driver to `on_trip`; on decline or timeout, release the driver and offer the next candidate. Every transition is idempotent and recorded. **Trip state machine.** `requested → matching → matched → arriving → in_progress → completed | cancelled`. The trip service is the single writer for a trip; clients send intents, the service validates the transition. **Regional sharding.** Matching is inherently local, so shard the live index, matching and gateways by city or region. Cross-region is only needed near borders; include neighbouring region cells in the query when the pickup is near an edge. ## Bottlenecks and failure modes - **Location write volume:** in-memory, region-partitioned stores absorb it; sample to durable storage at a lower rate. - **Double assignment:** the compare-and-set on driver status (or a single dispatcher process per cell group) is the invariant; say it explicitly. - **Hot cells:** a stadium emptying creates thousands of requests in one cell; widen the search ring progressively and use surge pricing to rebalance supply. - **Region failure:** replicate live state to a standby; on failover drivers reconnect and re-report location within seconds, so the index rebuilds itself. - **ETA service slowness:** fall back to straight-line distance ranking rather than blocking matches.20:Tf12,## Requirements As the user types, show the ten most popular queries starting with what they have typed so far, ranked by popularity, with results arriving fast enough to feel instant (under 100 ms including network). Popularity should reflect recent behaviour, with hourly or daily refreshes and a faster path for trending terms. Offensive or private strings must never appear. ## Estimates With 10 million searches/day and ~5 keystrokes per search, there are ~50 million prefix lookups/day: ~600/s average, peaks of a few thousand per second. If there are 100 million distinct queries and we store the top 10 completions for every prefix up to, say, 20 characters, the structure is tens of gigabytes: fits in memory across a small cluster. ## API `GET /suggest?q=des&limit=10` returns `["design patterns", "design system", ...]`. Responses depend only on the prefix (no user state), so they cache perfectly at the CDN with a short TTL. The client debounces keystrokes (50–100 ms), cancels in-flight requests when a new key arrives, and caches results locally for backspacing. ## Data model Two equivalent choices: 1. **Trie with per-node top-k.** Each node represents a prefix and stores its ten best completions with scores. Lookup walks the prefix (O(length)) and returns the stored list; no subtree traversal at query time. 2. **Flat key-value table** `prefix → [top-k]` for every prefix. Simpler to shard and serve from Redis or a wide-column store; costs more storage because prefixes repeat completions. Both are built offline from an aggregated `query → weighted count` table where weights decay over time. ## High-level design ```mermaid flowchart LR subgraph offline[Offline pipeline] L[(Query logs)] --> AGG[Aggregate + decay + filter] AGG --> B[Build trie / prefix table] B --> ART[(Published snapshot)] end subgraph online[Online path] C[Browser] --> E[CDN edge cache] E --> S[Suggest service, in-memory shards] T[(Trending counts)] --> S end ART --> S ``` The offline job runs hourly, applies the blocklist, computes decayed popularity, builds the structure, and publishes a new snapshot. Serving nodes load the snapshot into memory with a double-buffer swap so no request sees a half-built structure. ## Deep dives **Why precompute top-k.** Computing the top 10 completions under a trie node at query time means traversing possibly millions of descendants. Storing the answer at each node makes every lookup a few pointer hops; the cost moves to the build step, which is offline and parallel. **Sharding.** Hash on the first character or two. "a", "s" and "t" prefixes are far hotter than "x" or "z", so split hot shards further (e.g. "a" → "aa"–"am", "an"–"az") and replicate every shard several times; the data is read-only so replication is trivial. **Trending.** A streaming job counts queries over the last few minutes. The serving node merges a small trending list into results when a recent term's velocity exceeds a threshold, so "earthquake" appears within minutes rather than after the next hourly build. **Personalisation.** Keep the global results cacheable and let the client (or a thin layer) re-rank or prepend the user's own recent searches, which it already has locally. ## Bottlenecks and failure modes - **Hot prefixes:** single-letter prefixes dominate traffic; edge caching with a 60-second TTL absorbs most of it. - **Snapshot publish:** load the new structure in the background and swap atomically; keep the old one until all requests using it finish. - **Bad data:** blocklist at build time, plus a kill switch to remove a term from serving immediately without a rebuild. - **Latency budget:** network ~30–50 ms, edge or service ~5 ms, client render ~10 ms. Most of the budget is the network, which is why the CDN matters more than the data structure.21:Td5b,## Requirements Ids must be unique across every datacenter, fit in a signed 64-bit integer (database-friendly), sort roughly by creation time so indexes and pagination work well, and be generated at millions per second with microsecond latency. No central service may be on the critical path. ## Estimates A node generating up to ~4,000 ids per millisecond needs 12 sequence bits (4,096). Supporting 1,024 nodes needs 10 bits. That leaves 41 bits for a millisecond timestamp: 2^41 ms ≈ 69 years from a custom epoch. ## API In-process library: `nextId() → int64`, plus `decode(id) → { timestamp, datacenterId, machineId, sequence }` for debugging. For systems that cannot embed, a small HTTP service `GET /ids?count=100` that returns a batch. ## Data model The id layout (Snowflake): ```text | 1 bit unused | 41 bits timestamp (ms since epoch) | 5 bits datacenter | 5 bits machine | 12 bits sequence | ``` A node-id registry (etcd, ZooKeeper, or a config table) assigns each generator a unique `(datacenter, machine)` pair via a lease. ## High-level design ```mermaid flowchart LR REG[(Node id registry, leased)] --> S1[Service instance + generator] REG --> S2[Service instance + generator] S1 -->|nextId, no network| ID1[id] S2 -->|nextId, no network| ID2[id] ``` Every instance obtains a node id at startup, then generates ids entirely locally. Uniqueness follows from (timestamp, node id, sequence) being unique per node per millisecond and node ids being unique across the fleet. ## Deep dives **Clock problems.** Generators depend on wall-clock time. If NTP steps the clock backwards, a naive generator could reuse a timestamp and collide with ids it already issued. Fixes: remember the last timestamp used and if `now < last`, either wait until the clock catches up (for small drifts) or refuse to generate and alert (for large ones). Never issue ids with a timestamp earlier than one already used. **Hot milliseconds.** If the sequence overflows within a millisecond, spin until the next millisecond. At 4,096 per ms per node that is rare. **Node id assignment.** Static config breaks under autoscaling. Use a lease in etcd or ZooKeeper: an instance claims a free id, renews periodically, and the id is reclaimed if the lease lapses. Verify on startup that no other live instance holds the same id. **Alternatives.** - *UUIDv4:* unique without coordination but random, 128 bits, and terrible for B-tree locality. - *UUIDv7 / ULID:* time-ordered 128-bit ids with no coordination. Great when 128 bits is acceptable; they are the modern default for many systems. - *Database ticket server:* a table with an auto-increment column handing out ranges to clients (Flickr's approach). Simple, but a central component and ordering only within a range. - *Database per-shard sequences with shard prefix:* ordered within a shard, but requires knowing the shard at generation time. ## Bottlenecks and failure modes - **Duplicate node ids** from misconfiguration are the realistic failure; leases plus startup verification prevent them. - **Clock skew across nodes** means ids are only *roughly* ordered globally; within a node they are strictly ordered. Say this explicitly. - **Information leakage:** ids reveal creation time and approximate rate; if that matters (public URLs), hash or encrypt them at the API boundary. - **Epoch exhaustion:** 69 years; pick a recent epoch and document it.22:Tbff,## Requirements Functional: create a short link for a long URL, redirect when the short link is visited, optional custom alias, optional expiry, click counts per link. Non-functional: redirects must be fast (they sit in front of every page view the link drives) and highly available; codes should not be guessable; creation can be slightly slower. ## Estimates 100 million links per month is ~40 writes/s; a 100:1 ratio gives ~4,000 reads/s, peak ~40,000. Ten years of links at ~500 bytes is ~6 TB. Small. The interesting constraints are latency and availability, not throughput. ## API ```text POST /api/links { longUrl, alias?, expiresAt? } -> { code, shortUrl } GET /{code} -> 302 Location: longUrl (404 if unknown/expired) GET /api/links/{code}/stats -> { clicks, byDay[] } ``` ## Data model `links(code PK, long_url, owner_id, created_at, expires_at)` and `link_clicks_daily(code, day, count)`. The code is the primary key because every redirect is a point lookup by code. ## High-level design ```mermaid flowchart LR U[Browser] --> LB[Load balancer] LB --> API[Redirect / API servers] API --> C[(Redis cache)] C -- miss --> DB[(Links store)] API -- click event --> Q[(Event stream)] Q --> AGG[Aggregator] --> DB API --> IDG[Code allocator] ``` Redirect servers are stateless; the cache holds `code → long_url` for hot links; the store is a key-value database or a relational database with the code as primary key. Click events are published asynchronously so the redirect path never waits on analytics. ## Deep dives **Code generation.** Three common schemes: 1. *Base62 of a unique 64-bit id* from a Snowflake-style generator: no collisions, but sequential codes let anyone enumerate your links and estimate volume. Mitigate by encrypting or bit-shuffling the id first. 2. *Random 7-character code with a uniqueness check*: 62^7 ≈ 3.5 trillion codes, so collisions are rare; retry on conflict. 3. *Pre-generated key pool*: a service fills a table with unused random codes; API servers lease batches. Fast and collision-free at the cost of a component to run. **301 vs 302.** A 301 (permanent) lets browsers and CDNs cache the redirect, so repeat visits never reach you: lower load, but you lose click counts. A 302 (temporary) sends every click through you, giving full analytics. Most commercial shorteners use 302 for that reason. **Analytics.** Never increment a counter row synchronously on each redirect; it serialises writes on hot links. Emit an event (code, timestamp, referrer, user agent) to a stream and aggregate per minute or per day. ## Bottlenecks and failure modes - A viral link produces millions of redirects per hour: the cache absorbs it, and a CDN can cache 302s briefly too. - If the cache tier dies, the database must survive the full read load for a few minutes: size it for that or fail some requests gracefully. - Abuse: scan destination URLs against phishing lists, rate-limit creation per user, and block known bad domains. - Expiry: lazy-delete on read plus a background sweeper.23:Tfc9,## Requirements Creators upload videos of any length (multi-gigabyte files); the platform processes them into several qualities; viewers stream smoothly on networks from 3G to fibre, worldwide; metadata (title, description, view counts, comments) is searchable and updatable. Playback must be highly available and cheap per view, because egress is the dominant cost. ## Estimates Peak of 5 million concurrent viewers at an average 3 Mbps is **15 Tbps** of egress. 500 hours of video uploaded per minute at ~1.4 GB/hour of source is ~1 PB/day of uploads, and renditions multiply stored bytes by 3–4×. These two numbers say: the CDN *is* the product's delivery architecture, and storage must be object storage with lifecycle tiering. ## API ```text POST /videos { title, size, contentType } -> { videoId, uploadId, partUrls[] } PUT (client uploads parts directly to storage) POST /videos/{id}/complete { parts: [{ etag, number }] } GET /videos/{id} -> { title, status, duration, manifestUrl, thumbnails } GET /{id}/master.m3u8 -> manifest listing renditions GET /{id}/720p/seg-00042.ts ``` ## Data model `videos(id, owner_id, title, description, status: uploading|processing|ready|failed, duration, created_at, manifest_key)`, `renditions(video_id, quality, codec, bitrate, segment_count)`, `view_counts(video_id, day, count)` updated asynchronously. Object storage holds `source/{id}` and `renditions/{id}/{quality}/seg-{n}.ts` plus manifests and thumbnails. ## High-level design ```mermaid flowchart LR subgraph ingest[Ingest] C[Creator] -->|presigned multipart| OS[(Object storage: source)] OS -->|event| ORC[Transcode orchestrator] ORC --> W[Worker fleet: split, encode, stitch, thumbs] W --> OR[(Object storage: renditions + manifests)] W --> MD[(Metadata DB)] end subgraph playback[Playback] V[Viewer] --> API[Metadata API] --> MD V --> CDN[CDN edge] CDN --> SH[Origin shield] --> OR end ``` ## Deep dives **Parallel transcoding.** Split the source into chunks at keyframe boundaries (say 10 seconds each). Encode every chunk at every rendition in parallel across hundreds of workers, then stitch and generate manifests. A one-hour video that would take hours to encode serially finishes in minutes. Model the pipeline as a DAG with idempotent steps so any failed step can be retried without redoing the rest. **Adaptive bitrate streaming.** HLS and DASH split each rendition into short segments (2–6 s) and describe them in a manifest. The player measures throughput and buffer level and requests the next segment from whichever rendition fits, so quality steps down on a congested network instead of stalling. Segments are plain HTTP objects, which is exactly what CDNs cache best. **Fast availability.** Encode the lowest rendition (e.g. 360p) first and publish a manifest with just that; add higher renditions as they finish. The creator sees "your video is live" in a couple of minutes. **Delivery.** Edge PoPs serve segments from cache; misses go through an origin shield that collapses requests from many PoPs into one origin fetch. For the hottest content, caches embedded in ISP networks remove transit cost entirely. ## Bottlenecks and failure modes - **Egress cost:** every percentage point of CDN hit rate is money; tune TTLs (segments are immutable, cache forever) and pre-warm trending videos. - **Viral spikes:** a video going from zero to millions of viewers hits the shield hard for a few minutes; the shield plus request coalescing protects the origin. - **Transcode storms:** a burst of uploads queues jobs; autoscale workers and prioritise by creator tier or expected popularity. - **Corrupt or hostile uploads:** validate containers and codecs before encoding; sandbox the transcoder; dead-letter inputs that fail twice. - **Metadata hot spots:** view counters on a viral video are a write hotspot; batch increments in a stream and update the database periodically.24:Te6d,## Requirements Starting from seed URLs, download pages, store them, extract links and keep going; revisit pages so the stored copy stays fresh. Be polite (never overload a host, obey `robots.txt`), be robust against broken or hostile sites, scale horizontally, and make it possible to add new content handlers (PDFs, images) later. ## Estimates One billion pages a month is ~400 pages/s. At ~500 KB per page that is ~200 MB/s of download bandwidth (1.6 Gbps) and ~500 TB/month of raw HTML, perhaps 100 TB compressed. A seen-URL set of 10 billion entries at ~100 bytes is ~1 TB in a database; a Bloom filter at 10 bits per URL is ~12 GB and fits in memory. ## API Internal interfaces: `frontier.next()` returns a URL whose host is allowed to be fetched now; fetchers publish `(url, status, headers, body)` to a parse queue; parsers publish extracted links and content records. Operators add seeds, set per-domain limits and view crawl progress. ## Data model `urls(url_hash PK, url, host, status, priority, last_fetched_at, change_rate, content_hash, depth)`. Page bodies go to object storage keyed by `url_hash` plus fetch timestamp. A `links(from_hash, to_hash)` table (or a graph store) supports ranking and discovery. Per-host state: `robots.txt` cache with expiry, crawl-delay, last request time. ## High-level design ```mermaid flowchart LR SEED[Seeds] --> F[URL frontier] F --> FE[Fetchers] DNS[(DNS cache)] --> FE ROB[(robots.txt cache)] --> FE FE --> PQ[(Parse queue)] PQ --> P[Parsers + extractors] P --> DD[Dedup: URL seen set + content fingerprints] DD --> ST[(Content store + URL table)] DD -->|new URLs| F ``` Every stage is a pool of stateless workers connected by queues. Fetchers only fetch; parsers only parse; the frontier decides order. ## Deep dives **Frontier design.** The classic two-level scheme: *front queues* partition URLs by priority (PageRank-like importance, freshness need, operator boosts); a selector pulls from higher-priority queues more often. *Back queues* hold URLs for exactly one host each, with a min-heap of "earliest next fetch time" per host so a fetcher always gets a URL whose host has waited long enough. This enforces politeness with one request in flight per host and respects `crawl-delay`. **Deduplication.** Normalise URLs first (lowercase host, remove fragments, sort query params, strip tracking params) and check a Bloom filter backed by the URL table for exact matches. For content, compute a SimHash fingerprint and compare Hamming distance to recently stored fingerprints to catch near-duplicates (mirrors, print versions). **Freshness.** Record whether each fetch changed the content. Pages that change often get shorter recrawl intervals; static pages drift out to weeks or months. Prioritise recrawls by importance × expected change. **Robustness.** Hard timeouts, size limits, redirect limits, and per-host URL caps so a calendar that generates infinite "next month" links cannot consume the crawl. Treat `429` and `503` as signals to back off that host. ## Bottlenecks and failure modes - **DNS:** 400 lookups/s to a shared resolver will throttle; run a local caching resolver. - **Hot hosts:** large sites have millions of pages; politeness means they crawl slowly by design, so spread across many hosts in parallel. - **Frontier size:** billions of pending URLs do not fit in memory; keep the hot head in memory and spill to disk or a database. - **Crash recovery:** checkpoint frontier state and fetch progress; make fetch and store idempotent so replaying a queue is safe. - **JavaScript pages:** headless browsers cost 10–100× more per page; use them selectively for high-value sites.25:T753,Video is the workload where bandwidth, not storage or QPS, decides the architecture. **Peak egress** 5 × 10^6 viewers × 3 Mbps = 15 × 10^6 Mbps = **15 Tbps**. For scale: a large datacenter might have a few Tbps of total internet uplink. You cannot serve this from one place, or from ten. It has to be served from hundreds of edge locations, ideally from caches sitting inside ISP networks so the traffic never crosses a paid transit link. This is why Netflix built Open Connect and why YouTube runs caches at ISPs. **Daily transfer** Assume 50 million DAU watching 2 hours each. 3 Mbps × 3,600 s = 10.8 Gb per hour ≈ 1.35 GB/hour. 50 × 10^6 × 2 h × 1.35 GB ≈ **135 PB/day**, roughly 4 EB per month. At cloud egress list prices (cents per GB) that is tens of millions of dollars per month, so every point of CDN hit rate is worth real money. **Origin load** If the edge serves 98% of bytes, origin egress at peak is 2% × 15 Tbps = **300 Gbps**, still substantial, which is why a mid-tier (origin shield) sits between edge and origin to collapse misses. **Storage vs egress** Adaptive bitrate requires encoding every title into 6–10 renditions (240p through 4K). That multiplies *storage* roughly 3–4× relative to the top rendition, but each viewer downloads only one rendition at a time, so *egress* is unchanged. Storage for a 10,000-title catalog at ~20 GB per title per rendition set is ~200 TB, trivial next to egress. **Live vs on-demand** On-demand has a long tail but popular titles cache well. Live streams are perfectly cacheable (everyone wants the same 4-second segment at the same time) but the segments are new every few seconds, so the edge must fetch each one once and fan it out; origin load scales with the number of PoPs, not viewers. Say the Tbps number, say "this cannot come from a datacenter", and the rest of the design follows.26:T7ab,Cache sizing is where you convert a vague "we'll cache it" into a number of nodes. **Reads per second** 10 × 10^9 / 86,400 ≈ 1.16 × 10^5 → **~115,000 reads/s average**, ~300,000 at peak. No relational database serves that directly; the cache is not optional. **How much to cache** Access is skewed. The Pareto heuristic (20% of items get 80% of reads) is a reasonable default for content; real feeds are usually *more* skewed because recent posts dominate. Hot set = 20% × 500 × 10^6 = **100 million posts**. Payload = 100 × 10^6 × 1 KB = **100 GB**. Add overhead for keys, pointers, expiry metadata and fragmentation, typically 30–50%: **~150 GB**. **Nodes** Memory per cache node is commonly 64–256 GB usable. 150 GB fits in **2–3 nodes**, or one large node, but you want at least 3 for availability and to spread the ~300,000 peak requests per second (a Redis node handles roughly 100,000–200,000 simple GETs per second). Use consistent hashing across nodes and a replica per shard. **Remaining database load** Misses = 20% × 115,000 ≈ **23,000 reads/s average**, ~60,000 at peak. Still heavy: this points at read replicas (each handling perhaps 10,000–20,000 point reads per second) or a wide-column store designed for the volume. If the cache hit rate reaches 95%, the database sees ~6,000/s average, which is a very different infrastructure bill, so measuring and improving hit rate pays off directly. **A better partition.** Instead of "20% of all posts", cache **everything from the last 48 hours**: if posting rate is ~5 million/day, that is 10 million posts, 10 GB, and likely serves >90% of reads because feeds show recent content. Cheaper and a higher hit rate. Say both approaches and pick the recency-based one. **Protecting the cache.** A nightly backfill reading all 500 million posts would evict the hot set. Bypass the cache for batch jobs, or use an admission policy that only caches items on their second request.27:T811,Chat is the estimation exercise where fan-out and connection count matter more than raw writes. **Messages per second** 500 × 10^6 × 40 = 2 × 10^10 messages/day. 2 × 10^10 / 86,400 ≈ 2.3 × 10^5 → **~230,000 writes/s average**. Peak at 2–3× → **~500,000–700,000 writes/s**. That is well beyond one database; message storage must be partitioned (by conversation id is the natural choice, so a chat's history is one partition). **Fan-out** Each message is delivered to every other member of the conversation. With a mix of 1:1 chats and groups, say an average of 3 recipients: 230,000 × 3 ≈ **700,000 deliveries/s average, ~2 million at peak**. Every delivery is a lookup of where the recipient is connected plus a push over that connection, or a push notification if offline. This is the hottest path in the system and must be in-memory. **Concurrent connections** Assume 10–20% of DAU are connected at any moment at peak: **50–100 million persistent connections** (WebSocket or similar). A tuned gateway server holds on the order of 100,000–1,000,000 idle connections (WhatsApp famously pushed past a million per server on Erlang), so **hundreds to ~1,000 gateway servers**. The gateways are stateless apart from the connection map, which must be shared (a presence/routing service keyed by user id → gateway). **Storage** Text messages are ~100–300 bytes with metadata. 2 × 10^10 × 200 B = 4 × 10^12 → **~4 TB/day**, ~1.5 PB/year. Discord stores trillions of messages this way in a wide-column store partitioned by channel and time bucket. Media: if 2% of messages carry a 1 MB image, that is 4 × 10^8 × 1 MB = **400 TB/day**, 100× the text. Media goes to object storage via presigned uploads; the message only carries a reference. **What the numbers tell you.** Hundreds of thousands of writes per second → partitioned message store. Millions of deliveries per second → in-memory routing and gateways. Connection count → a dedicated connection tier separate from business logic. Media → off the hot path entirely.28:T740,You will never be asked for exact figures, but you will be expected to know the *shape* of the ladder: each rung is roughly 10 to 1,000 times slower than the one before. | Operation | Approximate time | Relative | | --- | --- | --- | | L1 cache reference | 1 ns | 1 | | Branch mispredict | 3 ns | 3 | | L2 cache reference | 4 ns | 4 | | Mutex lock/unlock | 20 ns | 20 | | Main memory reference | 100 ns | 100 | | Compress 1 KB (fast codec) | 2 µs | 2,000 | | Send 1 KB over 10 Gbps network | 1 µs | 1,000 | | Read 1 MB sequentially from memory | 10 µs | 10,000 | | SSD random read (4 KB) | 100 µs | 100,000 | | Read 1 MB sequentially from SSD | 1 ms | 1,000,000 | | Round trip within a datacenter | 0.5 ms | 500,000 | | HDD seek | 10 ms | 10,000,000 | | Read 1 MB sequentially from HDD | 20 ms | 20,000,000 | | Round trip California to Netherlands | 150 ms | 150,000,000 | **Human scale.** If an L1 hit took one second, memory would take two minutes, an SSD read a day, a datacenter round trip a week, a disk seek four months, and a transatlantic round trip five years. **What it means for design** - Anything served from memory (caches, in-process data) is effectively free compared to anything that touches disk or network. This is why caching dominates read-path design. - A request that makes ten sequential same-region calls spends ~5 ms in network alone; do them in parallel. - One synchronous cross-region call costs as much as 300 in-region ones. Replicate data to where the users are instead. - Sequential disk reads are fast; random ones are not. Log-structured storage exists because of this row. - Sending a megabyte across a 1 Gbps link takes ~10 ms, so compressing payloads is worth it over WAN but rarely within a rack. Say the ladder in three groups: nanoseconds (CPU and memory), microseconds (SSD, LAN), milliseconds (disk, WAN).29:T5bf,Estimation is multiplication, and multiplication is easy when you round to powers of ten and know a few object sizes by heart. **Powers of two** | Power | Exact | Approximate | Name | | --- | --- | --- | --- | | 2^10 | 1,024 | 1 thousand | kilo (KB) | | 2^20 | 1,048,576 | 1 million | mega (MB) | | 2^30 | 1.07 × 10^9 | 1 billion | giga (GB) | | 2^40 | 1.1 × 10^12 | 1 trillion | tera (TB) | | 2^50 | 1.1 × 10^15 | 1 quadrillion | peta (PB) | Also useful: 2^32 ≈ 4.3 billion (an unsigned 32-bit id runs out faster than you think), 2^64 ≈ 1.8 × 10^19 (never runs out). **Primitive sizes.** `char` 1 byte; `int` 4 bytes; `long`, `double` and a millisecond timestamp 8 bytes; UUID 16 bytes (36 as a string). A row with ten such columns plus overhead is roughly 100 bytes. **Typical objects** | Object | Size | | --- | --- | | Short text post or chat message | 100–500 bytes | | Email without attachments | 10–75 KB | | JSON API response | 1–10 KB | | Thumbnail image | 10–30 KB | | Compressed photo (phone camera) | 1–3 MB | | Full web page with all assets | ~2 MB | | One minute of 1080p video | 50–100 MB | | One hour of 4K video | ~20 GB | **Using them.** 500 million posts per day × 300 bytes ≈ 150 GB/day of text, ≈ 55 TB/year. The same users uploading one 2 MB photo each is 1 PB/day, three orders of magnitude more. Media dominates storage; text dominates nothing. That single comparison tells you where the storage design effort goes.2a:T5d5,

All decks