Parallelism · throughput
Find independent work, then decide how much hardware can exploit it.
A scalar five-stage pipeline can complete at most one instruction per cycle in steady state. Instruction-level parallelism finds independent work to exceed that limit. Pipeline depth raises frequency; issue width raises instructions per cycle. Neither creates independence in the program.
Make each stage smaller.
Splitting IF, ID, EX, MEM, and WB into shorter stages can raise clock frequency because each stage performs less logic.
- Gain: shorter critical path and a faster clock.
- Cost: more register overhead, bypass paths, and hazard latency.
- Branch risk: more younger work sits behind an unresolved branch.
Issue more than one instruction per cycle.
A 2-wide or 4-wide design fetches and decodes blocks, checks dependencies, and dispatches independent operations to parallel units.
- Gain: IPC can exceed one when useful ILP exists.
- Cost: wider rename, scheduling, register, bypass, and commit structures.
- In-order limit: one blocked older instruction can idle younger slots.
| Design lens | Deeper pipeline | Wider superscalar |
|---|---|---|
| Core lever | Shorten stage delay to raise clock frequency | Duplicate lanes and units to raise instructions per cycle |
| ILP use | Temporal: advance work through more, shorter stages | Spatial: execute several independent operations together |
| Main limiter | Branch resolution, hazard latency, and pipeline-register overhead | True dependencies, fetch supply, issue readiness, and unit availability |
| Hardware cost | More clocked state, checkpoints, and forwarding distance | More ports, comparators, bypass links, and commit bandwidth |
| Amdahl lens | Only frequency-scaled work improves; latch, clock, and hazard time remain | Only independent slots improve; serial and dependency-bound work remain |
| Modern balance | Moderate depth limits recovery and clock-distribution cost | OoO scheduling fills lanes, but width is bounded by energy and wire delay |
Interactive speedup model
Amdahl’s law makes the serial fraction visible.
Speedup = 1 / ((1 − P) + P / S)P is the improvable fraction. S is the speedup applied only to that fraction. The untouched serial term remains on every run.
The 40.0% serial fraction sets the limit, even if the accelerated portion gets thousands of workers.
Live speedup curve
More processors bend toward the serial ceiling.
At 8 processors: 2.11×. Infinite parallel resources approach 2.5×.
Overall speedup is 2.11 times with 60.0 percent parallel work and 8 processors. The serial ceiling is 2.5 times.
What is the key tradeoff in a deeper pipeline?
Smaller stages can support a higher clock frequency, but the design pays more pipeline-register overhead and keeps more work in flight. Dependencies take more stages to resolve, forwarding gets more complex, and a bad branch prediction discards work from a longer speculative window. The best depth is therefore a frequency-versus-IPC-and-energy decision, not “deeper is faster.”
What limits a simple in-order superscalar processor?
Issue width helps only when the next program-order group contains independent operations and matching execution resources. A RAW dependency, cache miss, branch, or unavailable unit at the head of the group can leave otherwise capable slots idle. Out-of-order scheduling looks beyond that local blockage.
Worked problems · interview reasoning
Show the denominator, state the assumptions, and explain the tradeoff.
The arithmetic is short. A strong architecture answer makes the serial term, predictor initialization, and protocol state explicit so another engineer can reproduce the result.
40% serial, 60% parallel
8 cores: 1 / (0.40 + 0.60/8)2.105×
32 cores: 1 / (0.40 + 0.60/32)2.388×
Serial work 2× faster: 1 / (0.40/2 + 0.60/8)3.636×
Engineer B wins. Halving the serial term changes the bottleneck; adding 24 cores only shrinks an already-small parallel contribution.
Find the ceiling and processor count
Maximum = 1 / (1 − 0.95)20×
18 = 1 / (0.05 + 0.95/S)S = 171
99% of 20× = 19.8×S = 1,881
Moving from 18× to 19.8× needs eleven times as many processors. Near the serial ceiling, marginal hardware returns collapse.
Architecture reasoning loop
Opportunity → speculation → ordering → proof
- Opportunity: identify independent work and the serial fraction.
- Speculation: predict control and memory dependencies to keep units busy.
- Ordering: use rename, ROB, LSQ, and coherence state to preserve contracts.
- Proof: trace cycles, predictor state, exceptions, and cache-line ownership.