map / systems / scheduling

Systems · principle 2 of 7

Scheduling

The scheduler time-slices runnable threads across cores; when there are more runnable threads than cores, work waits in a run queue.

01 · Why it’s true

A CPU core runs one thread at a time. The kernel’s scheduler keeps runnable threads queued and switches between them many times a second, giving each a slice of time, so it looks like everything runs at once.

When there are more runnable threads than cores, the extras wait in a run queue. That waiting doesn’t show up as extra CPU usage (which can’t go past 100%), but it does show up as latency. On Linux, the load average counts runnable threads plus threads in uninterruptible sleep, which is usually waiting on disk I/O.

02 · Picture it

1Runnablethreadshave workto doqueue2Run queuewaiting fora free coretime slice3CPU coresone threadper core ata timeblocks4Blockedwaiting ondisk,network,locksMore runnable threads than cores → the run queue grows → latency rises, even though CPUusage can never show more than 100%.
1Runnable threadshave work to doqueue2Run queuewaiting for a free coretime slice3CPU coresone thread per core at a timeblocks4Blockedwaiting on disk, network, locksMore runnable threads than cores → therun queue grows → latency rises, eventhough CPU usage can never show morethan 100%.
Runnable threads compete for cores; the wait shows up as latency, not CPU.

03 · In the interview

  • “The box is slow”: telling CPU saturation (long run queue) apart from I/O waiting.
  • Reading load average relative to the number of cores.
  • Recognizing CPU throttling in containers when a CPU limit is hit.

04 · Practice question

“A 4-core server has a load average of 16, but CPU usage is only 30%. What’s going on?”

A strong answer covers

  1. On Linux, load average includes threads in uninterruptible sleep — usually waiting on disk or network storage — not just threads wanting CPU.
  2. High load with low CPU suggests lots of threads blocked on I/O rather than CPU contention.
  3. Check vmstat (the b column and wa), iostat -x for device latency and utilization, and look for processes in D state.
  4. Likely causes: a slow or saturated disk, a hung network filesystem, or heavy swapping.