map / networking / load-balancing

Networking · principle 5 of 6

L4 vs L7 load balancing

An L4 balancer routes connections without reading them; an L7 balancer parses requests, which buys smarter routing at the cost of more work per request.

01 · Why it’s true

An L4 (transport-layer) load balancer forwards connections based on addresses and ports without reading their contents. It’s fast and works for any protocol, but it balances connections, not requests.

An L7 (application-layer) load balancer terminates the connection and reads each request, so it can route by path or header, retry, and spread the requests from one connection across many backends — at the cost of more work per request.

02 · Picture it

L4: balances connectionsReads only IP addresses and portsPicks a backend once per connectionFast, works for any protocolA long-lived connection pins all itsrequests to one backendL7: balances requestsTerminates the connection, parses eachrequestRoutes by path, header, or cookieCan retry, rate-limit, and terminateTLSMore CPU per request, and it sits inevery request’s pathvsHTTP/2, gRPC and WebSockets reuse one connection for many requests — behind L4, that canoverload one backend while others idle.
L4: balances connectionsReads only IP addresses and portsPicks a backend once per connectionFast, works for any protocolA long-lived connection pins allits requests to one backendL7: balances requestsTerminates the connection, parseseach requestRoutes by path, header, or cookieCan retry, rate-limit, andterminate TLSMore CPU per request, and it sitsin every request’s pathHTTP/2, gRPC and WebSockets reuse oneconnection for many requests — behindL4, that can overload one backendwhile others idle.
What the balancer can see decides what it can balance.

03 · In the interview

  • Design questions: where TLS terminates, how routing works, and what health checks test.
  • Explaining uneven load with long-lived connections (HTTP/2, gRPC, WebSockets) behind an L4 balancer.
  • Health checks that are too strict (flapping) or too lenient (sending traffic to broken backends).

04 · Practice question

“We run gRPC behind an L4 load balancer. One backend is overloaded while the others are idle. Why?”

A strong answer covers

  1. gRPC uses long-lived HTTP/2 connections that carry many requests each.
  2. An L4 balancer chooses a backend once per connection, so all of a client’s requests land on the same backend for the life of that connection.
  3. Fixes: an L7 balancer or proxy that balances per request, client-side load balancing, or capping connection lifetime so clients reconnect and rebalance.
  4. Confirm by comparing connection counts and request rates per backend.