Backpressure & Flow Control

A producer emits 100k events/sec. A consumer can process 10k/sec. With no coordination, the consumer's queue grows unboundedly — memory bloats, latency cliffs, eventually the process OOMs. Backpressure is the mechanism by which downstream components signal upstream to slow down. Without it, any fast producer will eventually crash any slow consumer. This is THE problem in streaming systems.

TCP-level — sockets have send + receive buffers. When receiver's buffer fills, TCP's sliding window shrinks; sender's send() blocks. Happens transparently. Covers "I'm overwhelmed at the transport layer."

HTTP 429 / 503 — server signals "too many requests" or "service unavailable" with Retry-After header. Caller honors (see retry with backoff).

Reactive Streams spec (Java, JS, .NET) — consumer calls request(n) to pull at most n items. Producer can only emit up to the outstanding demand. Built-in to RxJava, Project Reactor, akka-streams, Node.js streams.

Credit-based flow control — consumer grants producer N "credits." Each sent message consumes one credit. Producer stops when credits hit 0; more credits sent as consumer processes. Used by AMQP, gRPC streaming, HTTP/2.

Pull-based queues (Kafka, SQS long-poll) — consumer decides when to fetch. Producer writes to durable queue; consumer reads at its own pace. Queue itself acts as buffer with disk as the overflow floor.

import mmh3

class CountMinSketch:
    def __init__(self, width=10_000, depth=4):
        self.w, self.d = width, depth
        self.grid = [[0] * width for _ in range(depth)]

    def add(self, item, count=1):
        for i in range(self.d):
            self.grid[i][mmh3.hash(item, i) % self.w] += count

    def estimate(self, item):
        # MIN across rows — over-estimate only, never under
        return min(self.grid[i][mmh3.hash(item, i) % self.w]
                   for i in range(self.d))

# Trade: ~N×W×8 bytes for constant-space approx counts at any scale

Classic example: a Node.js stream pipeline.

source.on("data", chunk => destination.write(chunk));
// destination is slow. Chunks accumulate in memory. OOM.

The fix Node provides:

source.pipe(destination);
// .pipe() reads write()'s return value. false = slow; pauses source until
// destination emits "drain". Transparent backpressure, same API.

Same idea everywhere:

• RxJava Flowable (backpressure-aware) vs Observable (not).
• gRPC server streaming respects client's read() pace.
• Kafka consumers poll at their own speed; broker never pushes.

Backpressure-aware APIs look almost identical to eager ones. The difference is correctness under load. Always choose the backpressure-aware variant. Cost: a little more code complexity. Benefit: your pipeline doesn't implode at peak traffic.

Related but different:

• Rate limiting = enforce a preset ceiling ("max 1000 RPS"). Protect against abusive clients, fair-share resources.
• Backpressure = dynamically signal real-time downstream pressure. Protects the system from itself when legit traffic outpaces capacity.

You want both. Rate limits set the outer envelope; backpressure handles normal variance within it.

Distributed queue's core design challenge is backpressure between producers and consumers. Distributed logging uses credit-based backpressure to avoid overwhelming the storage tier. Live streaming applies per-viewer backpressure to smooth traffic.