API Gateway

You have 40 microservices behind the scenes. A mobile client wanting to render a home screen shouldn't call 20 different hostnames, each with different auth, rate limits, and retry policies. An API gateway is the single door to the backend: it handles auth, rate limiting, routing, protocol translation, and request/response shaping — so each service can focus on its own logic.

It's part of the classic "one thing to rule them all" infra tier, alongside the load balancer.

1. Authentication & authorization. Validate JWTs, OAuth tokens, API keys. Reject unauthenticated requests before they touch backends.
2. Rate limiting & quotas. 1000 RPS per API key, 100 RPS per IP for unauthenticated. Centralized — individual services don't implement rate limiting.
3. Routing. /api/v1/orders/* → orders service. /api/v1/users/* → users service. Path-based, host-based, header-based.
4. Protocol translation. Client speaks REST/JSON; upstream speaks gRPC. Gateway translates.
5. Aggregation. Client wants home screen (user + recent orders + recommendations). Gateway calls 3 services, merges responses into one. Reduces mobile round-trips.
6. Circuit breaking & retries. Downstream service is failing — gateway stops hammering it; returns cached or degraded response.
7. Observability. Every request logged, traced, metered at one point.

Practically: an API gateway often is a reverse proxy with plugins (Nginx + Lua, Envoy + filters, Kong, AWS API Gateway). Plus a load balancer underneath routing to service instances. It's a spectrum, not a strict hierarchy.

Belongs at the gateway: auth, rate limit, routing, TLS termination, request ID injection, access logs, basic validation (size limits, required headers).

Belongs in services: business logic, domain validation, data access, service-to-service calls.

The anti-pattern: business logic creeping into the gateway. "Let's add a special rule for orders worth over $10,000" — the gateway has now hidden a business rule where nobody looks. When in doubt, keep the gateway dumb.

The gateway sits on every request's critical path. Even a 5ms overhead, compounded across 100M requests/day = 500k seconds of user-perceived latency. Measure religiously.

Sources of overhead:

• Auth lookup — hitting an external auth service per request is 5–10ms. Cache JWT validation results (signed tokens = no lookup needed, just signature verify).
• Rate limit counter increment — Redis INCR is fast (<1ms) but adds a round-trip.
• TLS termination — 1–3ms per new TLS handshake. Session resumption + HTTP/2 minimize this.
• Request body inspection — parsing for validation or transformation costs CPU.

Best practice: keep cache of decoded tokens (TTL ≈ token expiry). Batch rate-limit writes. Use local rate-limiter (token bucket in memory) with eventual sync to Redis for cross-instance coordination. Most gateways do all of this out of the box.

News feed uses an API gateway for mobile BFF aggregation. E-commerce uses it for all external traffic + rate limiting + auth. WhatsApp API uses a gateway for partner integrations. Notification system exposes a gateway for third-party webhooks.