Latency Numbers

Every architectural decision — "do we need a cache?", "should this call be async?", "can we fit this in RAM?" — collapses to the same question: how long does the operation actually take? Engineers who can't estimate this pick the wrong tool constantly. Engineers who can know, without running a benchmark, that reading 1MB from disk will dominate a request and therefore must go through cache.

The numbers below are Jeff Dean's famous list, updated for 2025 hardware. Memorize the orders of magnitude.

If an L1 access took 1 second:

• L2 cache: 14 seconds
• RAM: 3.3 minutes
• SSD random read: 9 hours
• SSD sequential 1MB: 23 days
• DC round-trip: 11.5 days
• Disk seek: ~7.6 months
• CA → Netherlands round-trip: ~9.5 years

This is why reading from disk to serve a user request is insane when the data could be in RAM. It's also why a cross-ocean call must never happen synchronously in a hot path.

Why caches exist. RAM is ~200× faster than SSD. A 1ms DB query becomes a 5μs Redis hit. On a page that makes 20 backend calls, that's 20ms vs 100μs — the difference between "snappy" and "laggy."

Why CDNs exist. A round-trip from Singapore to Virginia is ~180ms. You cannot beat this with code. The only answer is to move the data closer — CDN edges cut this to ~20ms.

Why asynchronous processing exists. A single synchronous cross-datacenter call per request limits you to ~6–7 RPS per thread. An async architecture (queue + workers) decouples request latency from processing latency — the user gets a fast ACK, work happens in the background.

Why compression is cheap. Snappy compresses 1KB in 2μs. Sending it over a 1 Gbps link takes 10μs, but sending 400 bytes (gzipped) takes 4μs. Net win: 4μs total instead of 10μs. For anything crossing the network, compress.

Why sequential beats random. 1MB sequential SSD read = 1ms. 1MB of random 4KB reads = 256 × 16μs = 4ms. 4× slower. This is why B-tree indexes (sequential) outperform pointer-chasing (random) even when the total bytes read are equal.

Every problem sets explicit latency targets in its requirements. These targets drive whether you need CDN, cache, in-memory indices, or async processing.