Parallelism · throughput
Find independent work, then decide how much hardware can exploit it.
A scalar five-stage pipeline can complete at most one instruction per cycle in steady state. Instruction-level parallelism finds independent work to exceed that limit. Pipeline depth raises frequency; issue width raises instructions per cycle. Neither creates independence in the program.
Make each stage smaller.
Splitting IF, ID, EX, MEM, and WB into shorter stages can raise clock frequency because each stage performs less logic.
- Gain: shorter critical path and a faster clock.
- Cost: more register overhead, bypass paths, and hazard latency.
- Branch risk: more younger work sits behind an unresolved branch.
Issue more than one instruction per cycle.
A 2-wide or 4-wide design fetches and decodes blocks, checks dependencies, and dispatches independent operations to parallel units.
- Gain: IPC can exceed one when useful ILP exists.
- Cost: wider rename, scheduling, register, bypass, and commit structures.
- In-order limit: one blocked older instruction can idle younger slots.
| Design lens | Deeper pipeline | Wider superscalar |
|---|---|---|
Core lever | Shorten stage delay to raise clock frequency | Duplicate lanes and units to raise instructions per cycle |
ILP use | Temporal: advance work through more, shorter stages | Spatial: execute several independent operations together |
Main limiter | Branch resolution, hazard latency, and pipeline-register overhead | True dependencies, fetch supply, issue readiness, and unit availability |
Hardware cost | More clocked state, checkpoints, and forwarding distance | More ports, comparators, bypass links, and commit bandwidth |
Amdahl lens | Only frequency-scaled work improves; latch, clock, and hazard time remain | Only independent slots improve; serial and dependency-bound work remain |
Modern balance | Moderate depth limits recovery and clock-distribution cost | OoO scheduling fills lanes, but width is bounded by energy and wire delay |
Interactive speedup model
Amdahl’s law makes the serial fraction visible.
Speedup = 1 / ((1 − P) + P / S)P is the improvable fraction. S is the speedup applied only to that fraction of the original runtime. The workload stays fixed. When S is a core count, this model assumes ideal parallel scaling without communication, synchronization, or bandwidth overhead.
The 40.0% serial fraction sets the limit, even if the accelerated portion gets thousands of workers.
Live speedup curve
More processors bend toward the serial ceiling.
At 8 processors: 2.11×. Infinite parallel resources approach 2.5×.
Overall speedup by processor count. Each bar uses the live model; its length is relative to the serial ceiling.
- 1 processor1.00×
- 4 processors1.82×
- 16 processors2.29×
- 64 processors2.44×
- 256 processors2.49×
- 1024 processors2.50×
- 4096 processors2.50×
Selected: 8 processors → 2.11×. Serial ceiling: 2.5×.
Overall speedup is 2.11 times with 60.0 percent parallel work and 8 processors. The serial ceiling is 2.5 times.
What is the key tradeoff in a deeper pipeline?
Smaller stages can support a higher clock frequency, but the design pays more pipeline-register overhead and keeps more work in flight. Dependencies take more stages to resolve, forwarding gets more complex, and a bad branch prediction discards work from a longer speculative window. The best depth is therefore a frequency-versus-IPC-and-energy decision, not “deeper is faster.”
What limits a simple in-order superscalar processor?
Issue width helps only when the next program-order group contains independent operations and matching execution resources. A RAW dependency, cache miss, branch, or unavailable unit at the head of the group can leave otherwise capable slots idle. Out-of-order scheduling looks beyond that local blockage.
Worked problems · interview reasoning
Show the denominator, state the assumptions, and explain the tradeoff.
The arithmetic is short. A strong architecture answer makes the serial term, predictor initialization, and protocol state explicit so another engineer can reproduce the result.
40% serial, 60% parallel
8 cores: 1 / (0.40 + 0.60/8)2.105×
32 cores: 1 / (0.40 + 0.60/32)2.388×
Serial work 2× faster: 1 / (0.40/2 + 0.60/8)3.636×
Halving the serial work wins in this comparison. It changes the bottleneck; adding 24 cores only shrinks an already-small parallel contribution.
Find the ceiling and processor count
Maximum = 1 / (1 − 0.95)20×
18 = 1 / (0.05 + 0.95/S)S = 171
99% of 20× = 19.8×S = 1,881
Moving from 18× to 19.8× needs eleven times as many processors. Near the serial ceiling, marginal hardware returns collapse.
Architecture reasoning loop
Opportunity → speculation → ordering → proof
- Opportunity: identify independent work and the serial fraction.
- Speculation: predict control and memory dependencies to keep units busy.
- Ordering: use rename, ROB, LSQ, and coherence state to preserve contracts.
- Proof: trace cycles, predictor state, exceptions, and cache-line ownership.
Put it into practice
Explain the mechanism. Test your reasoning.
Use the experiment above to support your answer, then apply the idea in the question bank.
Continue the learning path
Next: Out-of-order execution
Trace one instruction through rename, reservation stations, physical registers, the ROB, speculative memory ordering, recovery, and precise retirement.