Context Switching & the Dispatcher

A context switch is the act of saving the CPU state of the thread that is currently running and restoring the CPU state of a different thread, so that one physical CPU can be shared among many threads over time. The word "context" means exactly the bundle of information that defines where a thread is in its execution: the contents of the registers, including the program counter that says which instruction is next. Freeze that context and you have frozen the thread; reload it later and the thread continues as if nothing ever interrupted it.

Here is why it must exist. The CPU has exactly one set of registers per core, but the OS is juggling far more threads than it has cores. Those registers are a shared resource that only one thread can occupy at a time. To run thread B while thread A is only half-finished, the OS has nowhere to keep A’s live registers except by copying them out to memory first — and it needs a place to put them. That place is the process (or thread) control block.

Context

The complete CPU-visible state of a thread: its registers, program counter, stack pointer and status flags — everything needed to resume it exactly where it stopped.

Context switch

Saving the current thread’s context and restoring another thread’s, so the CPU can change which thread it is executing.

PCB (process control block)

The kernel’s per-process record; among other things it holds the saved register state (for threads, a per-thread control block does the same job).

Recall the timer interrupt from the hardware chapter. When it fires, the CPU is dragged out of user mode into the kernel, and the OS gets a chance to decide whether the current thread should keep running. If it decides to switch, this save-and-restore dance is what happens next. The timer is the heartbeat; the context switch is the actual changing of the guard.

The core of the saved context is the register file. Every general-purpose register (the ones holding your working values), the program counter, the stack pointer, and the status/flags register must be copied out to the thread’s control block. Miss even one and the thread resumes with corrupt state — a wrong value in a register or, worse, a program counter pointing at the wrong instruction. On real CPUs there is more to capture too: the floating-point and vector (SIMD) register state, which is large, so kernels often save it lazily and only when a thread actually uses it.

Now the crucial distinction. Switching between two threads of the same process is comparatively cheap, because those threads share one address space — the same page tables, the same view of memory. The OS swaps the registers and the stack pointer and it is essentially done. Switching between two different processes is more expensive, because on top of the registers the OS must also switch the memory map: it points the CPU at the new process’s page tables so that virtual addresses now translate to the new process’s physical frames.

Register state

General registers, program counter, stack pointer and flags — always saved and restored on any switch.

FPU / SIMD state

The large floating-point and vector register set; often saved lazily to avoid paying for it when a thread never touches it.

Address-space switch

Repointing the CPU at a new process’s page tables. On x86 this means loading the CR3 register with the new page-table base.

On x86 that memory-map switch is concretely a write to the CR3 register, which holds the physical address of the current page table. Loading CR3 is the moment the machine stops seeing process A’s memory and starts seeing process B’s. As we will see in the cost section, that single register write has an expensive side effect on cached address translations — which is a big part of why a process switch hurts more than a thread switch.

A context switch never happens for no reason. Something must pull the CPU into the kernel and give the scheduler an opening to change its mind about who should run. There are four everyday triggers, and it is worth being able to name each one when you see it in a trace.

• ▸Timer interrupt (preemption): the thread has used up its time slice, the timer fires, and the scheduler forcibly hands the CPU to someone else. This is what keeps one CPU-bound loop from freezing the machine.
• ▸Blocking system call / I/O wait: the thread asks for something that is not ready — a disk read, a network response, a lock held by another thread — so it cannot make progress. The kernel parks it and switches to a thread that can run.
• ▸A higher-priority thread becomes ready: an interrupt or a wakeup makes a more important thread runnable. A preemptive scheduler will switch away from the current thread immediately rather than wait for its slice to end.
• ▸Voluntary yield or exit: the thread explicitly gives up the CPU (a yield call, or waiting on a condition), or it simply finishes and terminates, and the OS must pick a successor.

It is useful to group these into two families. Involuntary (preemptive) switches — the timer tick and the higher-priority wakeup — are the OS taking the CPU away from a thread that would happily have kept running. Voluntary switches — blocking on I/O, yielding, exiting — are the thread giving the CPU up because it has nothing useful to do right now. The same save-and-restore machinery runs in every case; only the reason differs.

It helps to split the work into policy and mechanism. The scheduler is the policy: it decides which thread should run next according to priorities, fairness, and deadlines. The dispatcher is the mechanism: it is the small, tightly-optimised piece of the kernel that actually carries out the switch once the scheduler has chosen. The scheduler answers "who?"; the dispatcher does the "how".

Concretely, the dispatcher performs three jobs: it switches context (saving the outgoing thread’s registers and restoring the incoming thread’s), it switches to user mode if the new thread runs in user space, and it jumps to the correct location in the new thread’s code — that is, it reloads the program counter and lets execution resume there. It is deliberately kept lean, because it sits on the hot path of every single switch the system ever makes.

Scheduler

The policy component that decides which thread should run next (covered in depth in the scheduling chapter).

Dispatcher

The mechanism that gives control of the CPU to the thread the scheduler chose: switch context, switch mode, jump to the thread’s next instruction.

Dispatch latency

The time from when the scheduler decides to stop one thread to when the newly chosen thread actually begins executing.

Dispatch latency is the gap between the decision and the resumption — pure overhead during which no useful work happens. For most software it is negligible, but for real-time and latency-sensitive systems it is a headline number: if an urgent thread becomes ready, the worst-case time before it actually runs depends directly on how small and how bounded the dispatch latency is. Keeping it low (and predictable) is a major goal of real-time kernel design.

The direct cost of a context switch is the obvious part: the CPU cycles spent saving one register set, running a bit of scheduler logic, and restoring the next register set. On its own this is small — a fraction of a microsecond. If that were the whole story, switching would be cheap and we could switch as often as we liked. It is not the whole story.

The indirect cost is where the real damage lives, and it is all about the memory hierarchy from the hardware chapter. While thread A ran, it warmed the CPU caches with its own data and instructions. The instant thread B takes over, those caches are full of the wrong data; B runs slowly at first, suffering cache misses that stall the CPU on RAM until it re-warms the caches with its own working set. A process switch is worse still: loading CR3 to change the address space effectively invalidates the TLB (the cache of virtual-to-physical translations), so the new process pays a wave of expensive page-table walks before address translation gets fast again.

Direct cost

The cycles to save and restore registers and run the scheduler/dispatcher — small and fixed per switch.

Indirect cost

The performance lost afterwards to cold caches and (on a process switch) a flushed TLB — often far larger than the direct cost.

TLB flush

Discarding cached address translations when the address space changes; the new process must rebuild them via page-table walks.

This is why too-frequent switching quietly wrecks throughput, and why spawning more threads than you have cores can make a program slower rather than faster. Past the point where threads outnumber cores, adding threads does not add parallelism — it just adds context switches, and each switch throws away warm cache state. The CPU ends up spending its time re-warming caches instead of computing. This is exactly the situation profilers surface as high "system" or "sys" CPU time with disappointing real work done.

$ vmstat 1 3
 procs -----------memory---------- ---system-- ------cpu-----
  r  b   swpd   free   buff  cache   in   cs  us sy id wa
  3  0      0 1.2G   180M   3.1G  9200 48000  61 34  5  0
  4  0      0 1.2G   180M   3.1G  9600 51200  59 37  4  0

$ pidstat -w 1 1
   UID     PID   cswch/s  nvcswch/s  Command
  1000    4821   1120.00     880.00  app-server
  1000    4990      3.00       2.00  logger

Here is the confusion that trips up almost everyone. When your program makes a system call, the CPU flips from user mode into kernel mode, does the privileged work, and flips back. That is a mode switch — a change in privilege level, exactly the user/kernel boundary crossing we met in the hardware chapter. It is real work and it has a cost, but crucially it happens within the same thread. The same thread that called read() is the thread that returns from it, with its registers and address space intact.

A context switch is different: it changes which thread is running. A mode switch does not, by itself, cause a context switch. If read() finds its data already in the page cache, the kernel services the call and returns to the very same thread — one mode switch, zero context switches. A context switch only follows if that syscall has to block (the data is not ready), at which point the kernel parks your thread and dispatches another. So a blocking syscall can lead to a context switch, but the mode switch and the context switch are two separate events.

Mode switch

A change of CPU privilege level (user ↔ kernel) within the same thread — what a system call, trap or interrupt causes.

Context switch

A change of which thread the CPU is running — save one thread’s context, restore another’s.

Key relationship

Every context switch runs through the kernel (so it involves kernel mode), but most mode switches are not context switches — they return to the same thread.

Getting this right changes how you read performance data. A syscall-heavy workload pays mode-switch overhead on every call even when no thread ever changes; that is why batching I/O and avoiding a syscall per byte matters, as we noted with the mode boundary earlier. A switch-heavy workload pays the far larger context-switch tax of cold caches and TLB flushes. They are different problems with different fixes, and calling both "context switching" hides which one you actually have.