Threads

For decades a process ran as a single stream of instructions. That was fine when a machine had one CPU and did one thing at a time. But two changes broke that model. First, CPUs stopped getting dramatically faster per core and started getting more cores instead — a modern phone has eight, a server has dozens. A single-threaded program uses exactly one of them and leaves the rest idle. Second, users stopped tolerating programs that freeze: click a button, and the window must not lock up while the work happens.

Threads answer both problems. To use many cores, you split the work across many threads and the OS runs them on different cores literally at the same time — that is true parallelism, and it is how you turn an eight-core machine into an eight-times-faster encoder. To stay responsive, you keep one thread free to handle the user interface while other threads do the slow work in the background — that is responsiveness, and it is why your editor keeps scrolling while it saves a file.

You might ask: we already have processes, so why not just start more of them? You can, and sometimes you should. But creating a process is heavy — the OS must build a whole new address space — and two processes cannot share data without asking the kernel to set up special channels between them. Threads are cheaper to create and they share memory by default, so passing data between them costs nothing. When several streams of execution need to work on the same data, threads are the natural, efficient tool.

Thread

An independent stream of execution inside a process; one process can hold many, each running its own path through the code.

Parallelism

Multiple threads truly executing at the same instant on different CPU cores — the way to actually use a multicore machine.

Concurrency

Multiple threads making progress over the same period by interleaving, even on a single core; the broader idea parallelism is one case of.

Responsiveness

Keeping one thread free for the user while others do slow work, so the program never appears frozen.

The whole personality of a thread comes from one split: some things belong to the process and are shared by every thread in it, and a few things are private to each thread. Get this table straight and most thread behaviour — the good and the terrifying — follows directly from it.

The shared side is almost everything. All threads in a process see the same program code, the same heap (so a pointer to a malloc’d object is valid in every thread), the same global and static variables, and the same table of open file descriptors (so a file one thread opens, another can read). This is why threads communicate for free: they are already looking at the same memory.

The private side is small but essential. Each thread needs its own stack, because each is running its own chain of function calls with its own local variables; if they shared a stack, they would corrupt each other instantly. Each also has its own register set and its own program counter — the CPU state that says which instruction it is on — so that the OS can pause one thread and resume another. Finally, threads can have thread-local storage (TLS): named globals that are secretly per-thread, like the errno each thread sees privately.

Shared: code

The program’s instructions — all threads execute from the same text, just at different points in it.

Shared: heap

Dynamically allocated memory; a pointer created in one thread is meaningful in all of them.

Shared: globals & static data

Process-wide variables every thread can read and write — convenient, and the classic source of races.

Shared: open files

The file-descriptor table belongs to the process, so an fd opened by one thread works in another.

Private: stack

Each thread’s own call stack, holding its local variables and return addresses.

Private: registers & PC

Each thread’s CPU state, including the program counter marking the instruction it is executing.

Threads and processes are two answers to the same question — how do I run several things at once — with opposite trade-offs. Processes are isolated: each has its own address space, so one crashing or misbehaving process cannot corrupt another’s memory. That safety is the whole reason your browser puts each tab in its own process — one bad page kills its tab, not the browser. The price of isolation is cost: creating a process is expensive, and two processes can only exchange data through deliberate inter-process communication (pipes, sockets, shared-memory segments) that the kernel sets up.

Threads flip both. They are cheap to create because there is no new address space to build — just a new stack and some bookkeeping. And they communicate at memory speed because they already share everything. But that same sharing removes the safety net: there is no isolation between threads, so a wild pointer in one thread can trample another thread’s data, and one thread crashing the process takes down all of them. Worse, shared mutable data invites the race conditions and deadlocks that the next part of this course is entirely devoted to taming.

So how do you choose? Reach for threads when the work is naturally cooperative — many streams operating on one shared dataset, where fast communication matters and you are willing to synchronise carefully. Reach for separate processes when the work should be isolated — untrusted plugins, independent services, or anything where one part crashing must not sink the rest. Real systems mix both: a web server may run a few worker processes for isolation, each with a pool of threads inside for cheap parallelism.

Creation cost

A thread is far cheaper to spawn than a process, which must build a fresh address space.

Communication

Threads share memory directly (free but perilous); processes need explicit IPC (safe but slower).

Isolation

Processes are walled off from each other’s memory; threads inside a process are not walled off at all.

Blast radius

A crash in one thread takes down the whole process and all its threads; a crash in one process leaves others alive.

A thread is an idea; something has to actually implement it. There are two places it can live. A kernel-level thread is created and scheduled by the OS itself — the kernel knows it exists, counts it, and places it on a CPU. A user-level thread is created and scheduled by a library inside your process, entirely above the kernel, so the OS does not even know these threads exist; it just sees one process running.

Pure user-level threads sound attractive: switching between them is just a function call in your own code, with no trip across the user/kernel boundary, so it is blazingly fast, and you can create thousands cheaply. But they carry one crippling flaw — the blocking-syscall problem. Because the kernel sees only the single process, the moment one user thread makes a blocking system call (say it reads from a socket with no data yet), the kernel blocks the whole process. Every other user thread, ready to run or not, is frozen alongside it, because from the kernel’s view there is only one thing to block.

Kernel-level threads do not have this problem. When one kernel thread blocks in a syscall, the kernel simply schedules a different thread of the same process onto the CPU — the others keep running. The cost is that every thread operation (create, destroy, switch) goes through the kernel and is therefore heavier than a user-space function call. This tension — user threads are fast but block badly, kernel threads block well but cost more — is exactly what the threading models in the next section try to resolve.

Kernel-level thread

A thread the OS knows about and schedules directly; blocking one does not block its siblings.

User-level thread

A thread managed by a library inside the process; invisible to the kernel, so switching is cheap but limited.

Blocking-syscall problem

A blocking call in one pure user thread stalls the entire process, because the kernel can only block the one entity it sees.

Thread library

The user-space code (or the thin layer over kernel threads) that offers create/join/switch operations to your program.

Given user threads and kernel threads, the design question is how to map one onto the other. There are three classic models, and understanding them explains why almost every mainstream OS today made the same choice.

The many-to-one model maps many user threads onto a single kernel thread. Thread management is all in user space, so it is fast and portable — but it inherits the blocking-syscall problem (one blocking call freezes all threads) and, fatally, it cannot use more than one core at a time, since the kernel sees only one schedulable thread. It was common in early thread libraries and is now largely a historical footnote.

The one-to-one model gives every user thread its own kernel thread. Now a blocking call stalls only that thread, and the kernel can run your threads on as many cores as you have — real parallelism. The cost is that every thread is a kernel object, so creating thousands is heavier. The many-to-many model tries to have it both ways, multiplexing many user threads onto a smaller pool of kernel threads, but it is genuinely complex to build and get right.

In practice, one-to-one won. Linux (via the NPTL implementation of pthreads) and Windows both map each thread you create directly to a kernel thread. The reason is pragmatic: kernel threads became cheap enough that the simplicity and true-parallelism of one-to-one beat the complexity of many-to-many for almost everyone. The many-to-many idea did not die, though — it re-emerged in language runtimes as the goroutine/virtual-thread scheduling mentioned earlier, built above the OS rather than inside it.

Many-to-one

Many user threads on one kernel thread; fast switching but no multicore use and blocking stalls everyone. Mostly historical.

One-to-one

Each user thread backed by its own kernel thread; true parallelism and independent blocking, at a higher per-thread cost.

Many-to-many

Many user threads multiplexed onto a smaller pool of kernel threads; flexible but complex to implement.

NPTL

The Native POSIX Thread Library — Linux’s modern one-to-one pthreads implementation.

On Unix-like systems the standard threading API is POSIX threads, universally called pthreads. Two calls carry most of the weight. pthread_create starts a new thread that begins executing a function you name; pthread_join waits for a thread to finish, the way wait() reaps a child process. Everything else — mutexes, condition variables — builds on top of these.

The one idea that trips up newcomers is how you tell a new thread what to do. You do not pass it a block of code inline; you pass it a function pointer — the address of a "start routine" with the fixed signature void *f(void *) — plus a single void * argument that becomes that function’s parameter. The void * is a generic pointer to anything: an int, a struct, an array. This is C’s way of letting one thread-creation call launch any function with any payload. The return value of the start routine (also a void *) is later collected by pthread_join.

#include <pthread.h>
#include <stdio.h>

/* The start routine: fixed signature void *(void *). */
void *worker(void *arg) {
    int id = *(int *)arg;          /* recover our argument */
    printf("thread %d running\n", id);
    return NULL;
}

int main(void) {
    pthread_t t[2];
    int ids[2] = {0, 1};

    for (int i = 0; i < 2; i++)   /* create: pass function + its arg */
        pthread_create(&t[i], NULL, worker, &ids[i]);

    for (int i = 0; i < 2; i++)   /* join: wait for each to finish */
        pthread_join(t[i], NULL);

    printf("all threads done\n");
    return 0;
}

Read it top to bottom. pthread_create takes the address of a pthread_t handle to fill in, attributes (NULL for defaults), the function to run, and the argument to hand it — here a pointer to each thread’s own id so the two do not read the same variable. The threads run concurrently, in an order the OS decides, so the two "running" lines may print in either order. The pthread_join loop then blocks main until both have finished, guaranteeing "all threads done" prints last. Omit the join and main might return and tear the whole process down while the workers are still mid-print.

$ cc -pthread threads.c -o threads
$ ./threads
thread 0 running
thread 1 running
all threads done

Threads are cheap compared to processes, but they are not free — creating and destroying one still costs real time and memory. A server that spawns a brand-new thread for every incoming request pays that cost on every request and, under a traffic spike, can spawn so many threads that the machine spends all its time switching between them instead of doing work. The fix is a thread pool: create a fixed set of worker threads once, at startup, and feed them a queue of tasks. Each worker loops — pull a task, do it, come back for the next — so threads are reused instead of recreated.

This turns the classic thread-per-request server design into a pool-of-workers design. The pool caps concurrency at a sane number (say, a few per core), which both amortises creation cost and protects the machine: extra work waits politely in the queue instead of spawning unbounded threads and melting the box. Java’s ExecutorService, the worker pools in databases, and the thread pools inside most application servers are all this same pattern.

There is a rival design worth knowing. Instead of many threads each blocked waiting on I/O, Node.js runs your JavaScript on a single event-loop thread and never blocks it: I/O is handed off and a callback fires when it completes. One thread juggles thousands of connections because it is never sitting idle waiting. Neither approach is universally better — thread pools shine for CPU-bound work that must run in parallel across cores; the event loop shines for I/O-bound work with huge connection counts. Many real systems combine them: an event loop out front, a thread pool behind it for the heavy CPU work.

• ▸Thread pool: a fixed set of reusable workers pulling tasks from a shared queue — reuse instead of per-task creation.
• ▸Thread-per-request: simple but spawns unbounded threads under load and pays creation cost every time.
• ▸Event loop (Node): one non-blocking thread multiplexing many I/O-bound connections via callbacks.
• ▸Oversubscription: running far more busy threads than cores, so time drains into context switches instead of work.
• ▸Common hybrid: an event loop for I/O with a thread pool behind it for CPU-heavy tasks.

Threads are powerful, but an engineer has to respect their costs. Creating a thread takes real time. Switching between threads — a context switch — is not free either: the OS must save one thread’s registers, load another’s, and the switch disturbs the CPU caches, so a machine drowning in threads can spend more time switching than computing. And every thread needs its own stack, which the OS reserves up front — commonly one to eight megabytes each. Ten thousand threads at 1 MB of stack is ten gigabytes of memory before your program has done a thing. This single fact is why unbounded thread-per-request designs fall over, and why pools and event loops exist.

There is a language-specific twist every Python engineer eventually hits: the Global Interpreter Lock, or GIL. CPython allows only one thread to execute Python bytecode at a time, holding a single process-wide lock. So Python threads give you responsiveness and are fine for I/O-bound work (the lock is released while waiting on I/O), but they cannot run CPU-bound Python code in parallel across cores — for that you use multiple processes instead. It is a vivid reminder that "threads" mean subtly different things in different runtimes, and you must know your platform’s rules.

The deepest pitfall, though, is the one we have flagged all chapter: shared state. Because threads share the heap and globals, two of them updating the same data without coordination produces a race condition — a bug whose outcome depends on the exact, unrepeatable timing of who ran first. These bugs vanish when you look for them and reappear in production, and taming them with locks, semaphores, and condition variables is the entire subject of the next part of this course.

Context-switch cost

Saving and restoring thread state plus the cache disturbance it causes; heavy when threads vastly outnumber cores.

Stack memory per thread

Each thread reserves its own stack (often 1–8 MB), so thread count directly drives memory use.

Oversubscription

Having many more runnable threads than cores, so throughput drops as switching overhead climbs.

GIL (Python)

CPython’s single lock that lets only one thread run bytecode at once — fine for I/O, useless for CPU-bound parallelism.

Race condition

A timing-dependent bug from unsynchronised access to shared data — the danger that comes bundled with sharing memory.