Concurrency Models

"Handle 50k concurrent connections" is an easy requirement to write and a hard one to meet. Every language offers a concurrency model — threads, event loops, actors, coroutines, or some combo. Each has different costs: threads consume memory, event loops can't use multiple cores, actors trade simplicity for message overhead. Picking wrong caps your system's throughput by 10×.

Interview answer: "how does your language/runtime handle concurrency?" should be instant, with tradeoffs.

One thread. It runs a loop: pick an event from the queue, run its handler, go back. I/O is async — the handler issues read() and returns immediately; the runtime re-queues the continuation when the read completes.

Why it's fast: no thread context switches, no stack allocations per connection. 50k connections = 50k small closures in memory. A single Node process can hold 100k+ WebSocket connections on modest hardware.

Why it's fragile: any blocking CPU work freezes everyone. One JSON.parse of a 10MB string stalls every connection on the loop. Mitigations: worker threads for CPU work, stream-based parsing, never sleep.

Limit: one CPU core. Need more? Run N Node processes behind a load balancer. Each has its own loop.

Java's traditional model: one thread per request. 10k requests = 10k threads × 2MB stack = 20 GB RAM just for stacks. Context switches dominate. Can't do it.

Go's model: goroutines. User-space coroutines with ~2 KB starting stack (grows as needed). 10k connections = 10k goroutines × 2 KB = 20 MB. Go's runtime multiplexes them onto a pool of N OS threads (N = CPU count). Each syscall yields the goroutine; the scheduler picks another one.

Result: you write synchronous-looking code (conn.Read(), conn.Write()) and get async performance for free. No callback hell, no async/await propagation, no special "blocking = bad" discipline. The language hides the machinery.

This is why Go is the default for high-concurrency servers in 2025 — Kubernetes, Docker, Prometheus, CockroachDB, Consul all written in Go. Java 21+ Loom (virtual threads) brings the same model to JVM, retrofitting 30 years of Java apps.

Each actor is a single-threaded computation with a mailbox. Actors communicate by sending messages — they never share mutable state directly. No locks, no data races (by construction).

Erlang's killer feature: actors can supervise each other. An actor that crashes doesn't take the process down — its supervisor restarts it. "Let it crash" is a legitimate error-handling strategy. WhatsApp famously scaled to 2M connections per box on Erlang because actor-crashes are cheap and local.

Cost: every interaction is a message. Small, fast operations pay overhead vs direct method calls. Most languages don't integrate actors naturally — Akka (JVM) and Orleans (.NET) add them as libraries; Erlang and Elixir bake them in.

WhatsApp's scale hinges on Erlang actors. Video conferencing signaling uses Go for goroutine-per-room. Stock exchange matching engines use single-threaded event loops for determinism. PUBG game-servers use actor-like entity models.