02 · Concurrency and performance
QoS, Fairness, and Backpressure
Treat latency, bandwidth share, starvation, and credit conservation as correctness properties under sustained contention.
A weighted arbiter can meet aggregate bandwidth while still starving one class, leaking credits, or violating a bounded-latency promise.
demand pending → eventual grant within class bound · free + in_flight = issued credits
Why it matters
- A chip can move every bit correctly and still be unusable because its performance policy is broken.
- For example, a GPU stream can starve CPU memory traffic even though every individual transaction is protocol-correct.
- QoS verification must prove that each priority class receives its specified bandwidth and latency behavior under congestion.
What is difficult
- Long-tail starvation may appear only once in a very large sample, such as a request waiting 10,000 cycles for a grant.
- Weighted Round Robin and related arbiters are intentionally difficult to predict cycle by cycle under random stimulus.
- Fairness requires statistical or bounded evidence, not a few successful transfers.
- A credit leak can degrade throughput slowly enough that short tests pass.
- A watchdog must distinguish an actively starved class from one with no pending demand.
Failure signatures
- A low-priority request waits beyond its allowed starvation bound.
- A high-priority stream misses its required bandwidth percentage.
- Grant ratios do not converge toward programmed WRR weights.
- A low-priority request blocks a high-priority request through a shared buffer or single-entry queue.
- READY deassertion on the final beat causes duplication, loss, or state corruption.
- Flow-control credits leak and total throughput decays over a long run.
Two viable approaches—and their cost
Statistical performance subscriber
Collect latency and bandwidth distributions from accepted and completed transactions.
- Non-intrusive and implementation-independent.
- Provides a global view of latency, bandwidth share, and tail behavior.
- Needs explicit thresholds or post-processing to turn statistics into pass-fail decisions.
- A finite sample cannot prove every possible starvation trace.
Credit-based tracking
Mirror the DUT's internal flow-control credit accounting in the testbench.
- Detects credit leaks that slowly reduce throughput.
- Can localize a performance failure to a conservation error.
- Tightly coupled to RTL implementation details.
- Sensitive to arbiter and buffering changes.
Interview answer, built from the mechanism
- Use pressure-cooker stimulus: saturate the arbiter with several high-priority streams plus at least one low-priority stream.
- Record request, grant, and completion timestamps. Build latency distributions, tail percentiles, throughput, and grant-share metrics per priority.
- Fail a class that exceeds its latency or starvation bound, and fail a high-priority class that misses its specified bandwidth share.
- Run long regressions and track credit conservation and aggregate throughput so slow credit leaks cannot hide behind short functional tests.
Component responsibility contract
| Component | Responsibility | Required change |
|---|---|---|
| Agent config | Traffic shaping | Add Weight and Priority knobs for sequences. |
| Monitor | Timestamps | Record req_time, gnt_time, and data_done_time. |
| Subscriber | Statistics | Collect minimum, maximum, average, histogram, and tail latency by priority. |
| Watchdog | Starvation | Check only priority classes with pending demand against their timeout. |
| Credit model | Conservation | Track issued, consumed, and returned credits over long runs. |
Implementation patterns
realtime total_latency[int];
int trans_count[int];
realtime last_access_time[int];
realtime MAX_LATENCY_THRESHOLD = 500ns;
realtime STARVATION_TIMEOUT = 2000ns;
virtual function void write(mac_item t);
realtime latency = t.complete_time - t.accept_time;
int p = t.priority;
total_latency[p] += latency;
trans_count[p]++;
last_access_time[p] = $realtime;
if (latency > MAX_LATENCY_THRESHOLD)
uvm_report_warning("QOS_TAIL", "Latency exceeded 500 ns");
uvm_report_info(
"PERF_STATS",
$sformatf("Prio=%0d Avg=%0t Count=%0d",
p, total_latency[p] / trans_count[p], trans_count[p]),
UVM_HIGH
);
endfunctionThe source formula is latency = complete_time - accept_time, and the running average is total_latency[p] / trans_count[p]. The example threshold is 500 ns.
virtual task run_phase(uvm_phase phase);
forever begin
#100ns;
foreach (last_access_time[p]) begin
if (pending_count[p] > 0 &&
($realtime - last_access_time[p]) > STARVATION_TIMEOUT)
uvm_report_error(
"QOS_STARVATION",
$sformatf("Priority %0d waited more than %0t",
p, STARVATION_TIMEOUT)
);
end
end
endtaskThe source checks every 100 ns with a 2000 ns timeout. The pending-demand guard prevents an idle traffic class from being mislabeled as starved.
virtual function void report_phase(uvm_phase phase);
foreach (trans_count[p]) begin
uvm_report_info(
"QOS_REPORT",
$sformatf("Prio %0d: %0d packets, Avg latency %0t",
p, trans_count[p],
total_latency[p] / trans_count[p]),
UVM_LOW
);
end
endfunctionAverage latency is useful context, but signoff also needs tail, maximum, bandwidth share, starvation, and credit-conservation results.
Stress recipe
- Saturate the fabric with several high-priority streams and one continuously pending low-priority stream.
- Run at least the source example of 10,000 transactions for a WRR grant-ratio check.
- Measure request-to-grant and request-to-completion latency, bandwidth share, p99, maximum, and starvation duration by priority.
- Drop READY on the final beat and other state-transition boundaries.
- Run long enough to reveal a slow credit leak and compare total throughput over successive windows.
Follow-up questions
How do you verify Weighted Round Robin?
Run a large window, such as 10,000 transactions, count grants by class, and compare the observed grant ratios with the programmed weights using a justified statistical tolerance.
How do you expose backpressure bugs?
Use a Ready-Toggle sequence that deasserts READY at difficult boundaries, especially the final beat of a burst, then prove state and payload remain stable until the transfer completes.
What common QoS bug should you look for?
Priority inversion, where a low-priority request blocks high-priority work through a shared buffer, dependency, or single-entry queue.