Performance Verification Framework

Mechanism and evidence

Guided verification lab

The tail disappears when you stop observing it.

Compare latency boundaries and capture policies on the same 1,000-transaction workload. Every number is calculated from the selected records.

STEP THROUGH

MEASUREMENT DISTRIBUTIONFull capture · start → done / 1 of 3

1000 observations · 10 ns bins · [10k, 10(k+1))

Non-empty latency bins

20–30 ns
250
30–40 ns
240
40–50 ns
249
50–60 ns
240
230–240 ns
10
250–260 ns
10
540–550 ns
1

Inspect all 55 bins, including zero counts

0–10 nsobservations0
10–20 nsobservations0
20–30 nsobservations250
30–40 nsobservations240
40–50 nsobservations249
50–60 nsobservations240
60–70 nsobservations0
70–80 nsobservations0
80–90 nsobservations0
90–100 nsobservations0
100–110 nsobservations0
110–120 nsobservations0
120–130 nsobservations0
130–140 nsobservations0
140–150 nsobservations0
150–160 nsobservations0
160–170 nsobservations0
170–180 nsobservations0
180–190 nsobservations0
190–200 nsobservations0
200–210 nsobservations0
210–220 nsobservations0
220–230 nsobservations0
230–240 nsobservations10
240–250 nsobservations0
250–260 nsobservations10
260–270 nsobservations0
270–280 nsobservations0
280–290 nsobservations0
290–300 nsobservations0
300–310 nsobservations0
310–320 nsobservations0
320–330 nsobservations0
330–340 nsobservations0
340–350 nsobservations0
350–360 nsobservations0
360–370 nsobservations0
370–380 nsobservations0
380–390 nsobservations0
390–400 nsobservations0
400–410 nsobservations0
410–420 nsobservations0
420–430 nsobservations0
430–440 nsobservations0
440–450 nsobservations0
450–460 nsobservations0
460–470 nsobservations0
470–480 nsobservations0
480–490 nsobservations0
490–500 nsobservations0
500–510 nsobservations0
510–520 nsobservations0
520–530 nsobservations0
530–540 nsobservations0
540–550 nsobservations1

Inspect the contract

All completions

N: 1000
Mean: 43.5 ns
p99: 258 ns
Max: 546 ns
Observed BW: 23.2727 Gb/s
Window: [0, 11000)

All completions inside the half-open window contribute to the histogram, latency statistics, and observed bandwidth.

Evidence to inspect

The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.

Checkpoint 1 / 3

Inspect the checker Selected checkpoint pseudocode

Conceptual checker / selected checkpoint

latency = done - start;
p99 = sorted[ceil(0.99 * N) - 1];
bandwidth = observed_bits / window_ns;

Example contract & limitations

Example contract. All times are in ns, payload is 256 bits per transaction, p99 uses nearest rank, and bins are [10k,10(k+1)). The deterministic sample takes index 100b+b for b=0…9 and misses all planted tail events; it is not an unbiased estimate. Sampled bandwidth is observed bits/time, not estimated workload throughput. Checkpoints show selected state changes, not equally spaced simulation cycles.

Read the complete walkthrough

Full capture · start → done

All completions. All completions inside the half-open window contribute to the histogram, latency statistics, and observed bandwidth. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
First measurement window. All completions inside the half-open window contribute to the histogram, latency statistics, and observed bandwidth. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
Final measurement window. All completions inside the half-open window contribute to the histogram, latency statistics, and observed bandwidth. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.

1 in 100 · start → done

All completions. Only selected records contribute. Missing the long tail makes this capture look deceptively fast; report sample count and policy with every result. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
First measurement window. Only selected records contribute. Missing the long tail makes this capture look deceptively fast; report sample count and policy with every result. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
Final measurement window. Only selected records contribute. Missing the long tail makes this capture look deceptively fast; report sample count and policy with every result. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.

Full capture · accept → done

All completions. All completions inside the half-open window contribute to the histogram, latency statistics, and observed bandwidth. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
First measurement window. All completions inside the half-open window contribute to the histogram, latency statistics, and observed bandwidth. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
Final measurement window. All completions inside the half-open window contribute to the histogram, latency statistics, and observed bandwidth. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.

1 in 100 · accept → done

All completions. Only selected records contribute. Missing the long tail makes this capture look deceptively fast; report sample count and policy with every result. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
First measurement window. Only selected records contribute. Missing the long tail makes this capture look deceptively fast; report sample count and policy with every result. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.
Final measurement window. Only selected records contribute. Missing the long tail makes this capture look deceptively fast; report sample count and policy with every result. Evidence: The right boundary is excluded. The completion exactly at 10,000 ns belongs in the final window.

Related implementation: 10 ns latency histogram

class perf_analyzer extends uvm_subscriber #(trans);
  int latency_bins[int]; // 0-10ns, 10-20ns, ...

  virtual function void write(trans t);
    realtime lat = t.done_time - t.start_time;
    int bin = lat / 10ns;
    latency_bins[bin]++;
  endfunction

  virtual function void report_phase(uvm_phase phase);
    foreach (latency_bins[i])
      `uvm_info(
        "PERF",
        $sformatf(
          "Bin %0dns-%0dns: %0d hits",
          i * 10,
          (i + 1) * 10,
          latency_bins[i]
        ),
        UVM_LOW
      )
  endfunction
endclass

Each transaction contributes done_time minus start_time to one 10 ns bucket. The report prints every occupied interval and its hit count.

Full capture · start → done, checkpoint 1: All completions. Inspect the contract.

Understand the failure

Distribution, not one averageMake the long tail visible.

Lifecycle timestamps feed an in-memory histogram, a sliding bandwidth window, and separate average, p99, and maximum results.

start0 nsaccept18 nsdone146 ns

0–10 nsp99max

Average42 nsP99 tail138 nsMaximum211 nsSustained BWbits / window time

Transaction-to-performance report

The monitor records lifecycle timestamps once; the subscriber turns them into distributions, bandwidth windows, and a compact final report.

100 masters: Generate concurrent traffic, congestion, and backpressure conditions.
Timestamped monitor: Records start, accept, and done for each packet.
Stats subscriber: Computes latency and aggregates every event or a defined sample.
10 ns histogram: Preserves the latency distribution needed for average and tail analysis.
CSV / architecture model: Carries the final summary into post-processing and target comparison.

Traffic handshake -> start, accept, and done timestamps
done - start -> latency value
latency / 10 ns -> histogram bin
window bits / window time -> bandwidth
histogram + throughput metrics -> CSV and architectural comparison

Why it matters

Functional correctness is only the baseline. A fabric must also meet throughput, stall, latency, and gigabyte-per-second architecture targets.
Average latency alone can hide the 99th-percentile tail where architectural bottlenecks appear.
Timestamped transactions and a reusable subscriber turn performance into a measurable verification result rather than an informal waveform inspection.

What is difficult

Tracking every cycle of every transaction across 100 masters can produce enough data to slow simulation by 5x to 10x.
Recording only 1 out of 100 transactions reduces overhead but can miss rare tail-latency spikes.
Keeping all statistics in memory avoids log bloat, but a simulation crash can lose data that was never persisted.
Bandwidth, backpressure response, skid depth, and effective throughput require explicit windows and boundary definitions.

Failure signatures

The design passes functionally but misses sustained bandwidth or latency targets.
Statistical sampling misses a rare 99th-percentile stall.
Per-transaction logging dominates runtime and distorts the workload being measured.
In-memory histograms disappear when simulation terminates unexpectedly.
READY/VALID backpressure leaves extra data in flight beyond the intended skid depth.
Protocol headers and idle gaps consume cycles, reducing useful effective throughput.

Compare approaches

Two viable approaches—and their cost

Statistical sampling monitor

Record timing for 1 out of every 100 transactions.

Strengths

Very low collection overhead.
Can estimate average latency when sampling is unbiased and sufficiently large.

Costs

Can miss corner-case and 99th-percentile latency spikes.
Sampling policy can bias results during bursty or priority-dependent traffic.

In-memory analysis subscriber

Process every transaction in memory and emit only a final summary table or machine-readable report.

Strengths

Maintains full transaction-count accuracy.
Avoids per-event log-file bloat.

Costs

Unflushed data is lost if simulation crashes.
Histogram state can still consume substantial memory in a long, high-cardinality run.

Explain it in an interview

Interview answer, built from the mechanism

I build a performance framework that records start, accept, and done timestamps in each monitored transaction.
A UVM subscriber computes latency and aggregates it into in-memory histograms rather than printing every event. That preserves simulation throughput while retaining the distribution.
I report average latency and the 99th-percentile tail, then export CSV for comparison with the architectural golden model.
I also define a sliding bandwidth window, separate peak from sustained bandwidth, measure backpressure/skid response, and report effective throughput as useful data beats divided by total clock cycles.

Assign responsibilities

Component responsibility contract

Component responsibilities and required verification changes for Performance Verification Framework
Component	Responsibility	Required change
Transaction	Timestamps	Add realtime fields for start, accept, and done.
Subscriber	Stats engine	Use an associative array to bin latencies into a histogram.
Final Report	CSV/SQL output	Dump data in a format that Python or Excel can parse.

Build the checker

Implementation patterns

10 ns latency histogramsystemverilog

class perf_analyzer extends uvm_subscriber #(trans);
  int latency_bins[int]; // 0-10ns, 10-20ns, ...

  virtual function void write(trans t);
    realtime lat = t.done_time - t.start_time;
    int bin = lat / 10ns;
    latency_bins[bin]++;
  endfunction

  virtual function void report_phase(uvm_phase phase);
    foreach (latency_bins[i])
      `uvm_info(
        "PERF",
        $sformatf(
          "Bin %0dns-%0dns: %0d hits",
          i * 10,
          (i + 1) * 10,
          latency_bins[i]
        ),
        UVM_LOW
      )
  endfunction
endclass

Each transaction contributes done_time minus start_time to one 10 ns bucket. The report prints every occupied interval and its hit count.

Performance formulas and boundariestext

Source rendering:
BW=Window_TimeBits_Transferred

Intended dimensional form:
bandwidth = bits_transferred / window_time

effective_throughput =
  useful_data_beats / total_clock_cycles

backpressure observation =
  cycles from READY low to the last VALID beat

The source formula is preserved verbatim, then qualified with the dimensionally correct bandwidth relationship. Peak and sustained results must use named window boundaries.

Stress the design

Stress recipe

Drive representative congestion across 100 masters and timestamp start, accept, and done for every observed packet.
Run a full-capture subscriber and a 1-in-100 sampler on the same seeded workload.
Compare average and 99th-percentile latency to quantify sampling error.
Bin full-capture latency in 10 ns increments and export a final CSV or SQL-ready report.
Measure bits transferred in named sliding windows and report peak and sustained bandwidth.
Deassert READY under load, count cycles to the last VALID beat, and compare with the allowed skid depth.
Calculate useful data beats per total clock cycle and attribute lost efficiency to headers, stalls, and idle gaps.

Follow-up questions

How do you measure bandwidth over time?

Use a sliding window, count transferred bits inside it, and divide by the window duration. Track peak separately from sustained bandwidth.

How do you verify backpressure efficiency?

Measure the cycles between READY going low and the last VALID beat. Relate that delay to the permitted skid or buffering depth.

What is effective throughput?

It is the ratio of useful data beats to total clock cycles. It exposes cycles consumed by protocol headers and idle gaps.

Engineering qualifications