Concurrency vs Parallelism

The big picture#

Everything below expands on these four points. Read the core sections top to bottom for the full mental model; the collapsible "Go deeper" boxes hold the advanced bits (threads vs async, race conditions, the math of speedup) you can skip on a first pass and come back to later.

Start here: why the distinction matters#

A computer is always asked to do more than one thing at a time: serve many users, download a file while the UI stays responsive, run a background job while answering requests. The naive picture is 'it does them all at once.' But a single processor core is fundamentally sequential — it fetches one instruction, runs it, fetches the next. It cannot literally do two things in the same instant. So how does one core appear to juggle hundreds of tasks?

The answer is that it doesn't run them simultaneously — it interleaves them. It runs a little of task A, saves where it was, switches to task B, runs a little of that, switches back. The switches happen thousands of times a second, far faster than a human notices, so it looks simultaneous. That illusion is concurrency. Whether any of it is actually parallel — happening in the same instant — depends entirely on how many cores you have.

How a single core takes turns#

The component that decides who runs next is the scheduler. It keeps a set of ready tasks and hands the core to one of them for a short slice of time called a quantum (often a few milliseconds). When the quantum expires — or the task voluntarily gives up the core — the scheduler performs a context switch and gives the core to the next ready task.

1. A task runs on the core until its quantum runs out, it finishes, or it has to wait for something (like disk or network).
2. The scheduler saves that task's state — its registers, its place in the code — so it can be resumed exactly where it left off.
3. It picks the next ready task (round-robin scheduling just rotates through them fairly so none is starved) and loads its saved state.
4. The core resumes that task. Repeat thousands of times a second. To a human it looks like everything runs at once.

In the simulator's one-core scenario you can watch this directly: tasks A and B alternate, one tick each, and the schedule fills A B A B A B. Only one runs at any instant — that's the defining property of concurrency on a single core. Add a second core and the picture changes completely.

The cost model: speedup, utilization, and Amdahl's law#

Two numbers describe how well a schedule uses its cores. Serial time is how long the work would take on a single core — the sum of all the CPU work. Wall-clock time is how long the schedule actually takes from the first tick to the last. Speedup is serial ÷ wall-clock: how many times faster you finished by spreading the work out.

• On one core, wall-clock equals serial, so speedup is exactly 1× — interleaving rearranges the work but can't make it finish sooner. Concurrency alone buys responsiveness, not throughput.
• Add cores and wall-clock drops toward the length of the longest single task — never below it. Two equal tasks on two cores finish in half the time (2× speedup); that's parallelism paying off.
• Utilization is the fraction of available core-time actually spent doing useful work. Idle cores (no ready work) and blocked cores (stuck waiting on I/O) both drag it down — and low utilization is exactly why speedup falls short of the core count.

Ways to actually achieve concurrency#

There's more than one way to structure a program to handle many tasks. They differ in who does the switching, how expensive a switch is, and whether they can use multiple cores. Treat this as a tour; the deep dive has the trade-offs.

• Threads — the OS scheduler gives each thread time slices and switches between them (preemptively). Threads can run on different cores, so they give you real parallelism — but each one costs memory, and switching between thousands of them is expensive.
• Async / event loop — a single thread juggles many tasks cooperatively: each task runs until it hits an I/O wait, then voluntarily yields so another can run. Switches are cheap (no OS involvement), so one thread can handle tens of thousands of connections — but it's concurrency on one core, not parallelism. This is Redis and Node.js.
• Multiprocessing — separate processes, each with its own memory, often one per core. True parallelism with strong isolation (a crash in one doesn't take down the others), at the cost of pricier communication between them.

What concurrency buys, and what it costs#

Concurrency buys responsiveness and throughput; parallelism buys raw speed on divisible work. But both add a cost that sequential code never has: the tasks can interfere with each other.

• Strength — responsiveness: interleaving means no single slow task freezes everything. The UI stays alive while a download runs; a slow request doesn't block the other thousand.
• Strength — throughput and speed: with real parallelism, divisible work finishes faster in proportion to the cores you can keep busy.
• Cost — shared state is dangerous: when two tasks touch the same data, the order of their interleaving matters. A race condition is a bug that appears only on certain timings; a deadlock is two tasks each waiting forever for a lock the other holds. These are notoriously hard to reproduce because they depend on the scheduler.
• Cost — overhead and diminishing returns: context switches, coordination (locks), and the serial fraction all eat into the speedup. Past a point, adding concurrency makes things slower, not faster.

Comparisons at a glance#

Two comparisons worth holding in your head: concurrency versus parallelism as concepts, and the two main ways to implement concurrency (threads versus async).

Where this shows up#

Once you can tell concurrency from parallelism, a lot of systems design vocabulary stops being mysterious.

• Redis is single-threaded — and fast. It's concurrent (an event loop juggling thousands of clients) but not parallel for command execution. Because it never blocks and never context-switches between OS threads, one core handles enormous load. This is the async story in production.
• Node.js uses one event loop per process; to use all cores you run cluster mode — N processes, one per core. Async concurrency times multiprocessing parallelism.
• Web servers (Nginx, Go services) combine both: a small pool of worker threads/processes (parallelism across cores) each handling many connections concurrently (so a slow client doesn't tie up a whole core).
• Data processing (MapReduce, Spark, GPU work) is the parallelism-heavy end: split a big divisible job across many cores or machines. Here Amdahl's law is the daily reality — the serial setup/shuffle steps cap how much the parallel part can help.

Common questions & gotchas#

In an interview#

Lead with the definition that most people get wrong: concurrency is dealing with many tasks by interleaving them (a structure that works on a single core); parallelism is running them in the same instant (which needs multiple cores). State plainly that one core can be concurrent without ever being parallel.

Then show the cost model. Speedup is serial ÷ wall-clock, and it's bounded by Amdahl's law — the serial fraction caps how much more cores can help, and uneven work plus coordination make real speedup worse than the theoretical limit. Mentioning utilization (idle and blocked cores) shows you understand why adding cores so often underdelivers.

Close by matching the tool to the bottleneck: I/O-bound work wants async concurrency (one core, never blocking — Redis, Node) so no core is wasted waiting; CPU-bound divisible work wants real parallelism across cores or machines. The strongest answers note that high-performance servers do both: async I/O per core, times one process per core.

References & further reading#