Day 1Why we verify, and the testbench without UVM
The same slides as the deck, with the instructor notes inside the text and the complete code. If you are taking the course on your own, start here and keep the deck alongside for the reviews.
Verify This!
An introduction to UVM
github.com/leandrotozzi/verifythis · SystemVerilog · Verilator · UVM 2020.3.1
How to use this course
Especially if you are taking it on your own
- What you need to know first: Verilog or VHDL, and having simulated
something. Object-oriented programming is not required — day 2 is exactly
that. The only thing the course takes for granted is SystemVerilog: if you
come from VHDL or Verilog-2001, half an hour with
docs/systemverilog-para-el-que-viene-de-vhdl.md(in Spanish) before starting - It is seven days of class, of 4 to 5 and a quarter hours — ≈ 31 h, and each agenda carries its own. Plus an optional day 8, after the wrap-up. On your own, count double: half of it goes into running the examples, and that is the half that teaches
- s opens the presenter notes in another window · n shows them down here, without leaving the page. That is where what the instructor would say out loud lives: the classic trap, the reason behind the number, the mistake everybody makes the first time. Do not skip them
- Every day closes with a review of clickable options and with a
self-checking exercise:
cd code/ejercicios/d1 && bash run.sh. The names of the other eighteen are incode/ejercicios/README.md
If you are teaching the course, these two slides are skipped with the 1 key. They are here because most people who open this will not have an instructor alongside, and without the presenter notes half the material gets lost — and that half is also the hardest one to reconstruct by reading the code.
How to use this course
Running it, moving around it, and where to look when something breaks
- Run the examples:
make u4/testsruns one,make u4the whole unit. A verification course that is only read is worth nothing. A UVM example takes a minute and a half the first time and fifteen seconds the second, if you haveccacheinstalled — and if you do not have Linux, the short road is GitHub Codespaces, which already ships with it - 0–8 jump to the cover and to each day · i opens the index · Esc shows the whole deck · and the corner ribbon unfolds the cheat sheet of the day
- If you are on your own, read the book:
en/libro/day1.htmlis this same course straight through, with the notes already inside the text. Keep the deck alongside for the reviews - When something does not work, go to the end: the appendices The debug toolbox —what to look at for each symptom— and The 21 silent traps —everything that compiles, runs and lies—. Those are the two slides to print
The last two are the ones that pay off most for someone studying alone, and the ones nobody reads until they are already stuck: the book carries these same notes inside the text, and the appendices are written as a table of symptoms, to look things up in rather than to read straight through.
Agenda
Day 1 · ≈ 4 h · units 1 and 2
Unit 1 · Why we verify
- Trends
- What UVM is
- The VTALU spec
- The verification plan
Unit 2 · The testbench without UVM
- The conventional testbench
- Functional coverage
- Interfaces and BFM · the
clocking block
Two exercises, and the second one is solved with the wave viewer
- And closing the day, going back to the first slide
By the end of the day you can:
- Write a five-column verification plan for a DUT you have not seen before
- Prove that your scoreboard checks something: put a bug into the DUT on purpose and watch it fail
- Read the number a
covergroupgives, say which row of the plan is missing, and what to look at first when it reads 0 % - Explain which race a
clocking blockavoids — and why they are optional anyway
The four verbs below are the contract of the day, and it is worth reading them
out loud before starting: on video they are the ten seconds in which someone
decides whether this is the one they were looking for. Three are rows 2, 4 and
5 of the closing self-assessment, brought to the front; the one about the bug
on purpose is the practice that runs through the whole course, from make mutante today to +BUG=1 in the capstone. Whoever can answer them at the end
of the day does not need to watch anything else.
And an honest warning for whoever comes from VHDL or from Verilog-2001, which is
the declared audience: day 2 levels the OOP, but nobody levels SystemVerilog,
and today interface, logic, enum, package, covergroup, clocking and an
assert property all show up. It is all on one page in
docs/systemverilog-para-el-que-viene-de-vhdl.md (in Spanish): half an hour
before starting, and day 1 stops having two steps at once.
Six out of seven
And the bug that reached silicon passed a green regression
- 14 % of projects get first silicon right. Six out of seven go back to the fab, and it is the worst figure in twenty years of the survey
- Every functional bug that reached silicon went through a testbench first. That testbench ran, finished, and said everything was fine
- Nobody tapes out with a
FAILEDin the log. You tape out with aPASS - Today has a single question: how do you know you verified? And "I ran a lot of tests" is not an answer
It is the first content slide of the course on purpose: before explaining what
UVM is and before any other chart, a question the student cannot answer yet. The
number is from the Wilson Research Group 2024 survey and it is not an
impression: only 14 % come out right on first silicon, so six out of seven need
at least one respin, and it is the worst figure in twenty years of the survey.
Asking who has done a tape-out and how it went hooks better than the chart. If
they ask where the data comes from: res/trends/data.json, and the figures are
regenerated with make figs.
The honesty worth saying out loud: not every respin is functional —there are
timing ones, analog ones, a spec that changed late—. But the one that is went
through a regression that said PASS, because had it said FAILED it would
not have been fabricated. That is the enemy of the day, and it is not the
verification engineer nor their testbench: it is a PASS nobody knew the
meaning of.
The question in the last bullet is thrown to the group and not answered.
The answers that will come are "100 % coverage", "I ran a thousand
operations" and "nothing failed", and all three come back today: the first
is the coverage section, the second is the plan, and the third is the one that
hurts most, because nothing failed is exactly what the respin's regression
said. The short answer arrives in the conventional testbench, when we put a bug
into the DUT on purpose; the long one is the whole day, and the last slide of
the day comes back to this question with what the student wrote.
Trends
Why they come back: what is inside a chip today
It is the first half of the answer to why they come back: inside there are two asynchronous clocks in almost all of them, a processor in five out of six and an AI accelerator in more than half. It is not that verification is done worse than twenty years ago; it is that there is much more to verify, and the question on the first slide is harder to answer than ever.
Trends
Verification is not a stage, it is half the work
Here is the argument for why this course exists: the verification effort has
already caught up with design. Verifying is not the final step before tape-out: it is
half the project, with its own team and its own language.
The lower 47 % is planted today and collected twice: in the second
exercise this afternoon, when the log says FAILED and nothing else, and in
the reporting section on day 3, which is where the log learns to say why.
Trends
And it is verified, mostly, with UVM
This one answers with what: eight out of ten ASIC projects verify with UVM, and half of the FPGA ones. What follows —the COBOL joke and the introduction— is what that UVM is and why it is not scary. And one thing worth leaving said now: UVM does not answer the question on the first slide. The testbench does; UVM is the way to write it so that the person next to you recognizes it.
Is SystemVerilog the COBOL of electronics?
| Language | Reserved Keywords |
|---|---|
| ANSI COBOL 85 | 357 |
| SystemVerilog | 248 |
| VHDL 2008 | 115 |
| Verilog 95 | 102 |
| C# | 102 |
| C++ 20 | 92 |
| Python 3 | 35 |
" No academic computer scientists participated in the design of COBOL; all of those on the committee came from commerce or government"* Sound familiar?
It is a joke with a moral: SystemVerilog has 248 keywords —the ones in Annex B of IEEE 1800-2017— and nobody uses them all. In this course we are going to use about thirty. The same thing happens with UVM: the library is enormous and with six or seven classes you put together a complete testbench. It is useful for lowering the anxiety of whoever arrives scared by the size of UVM. And it is the bridge to the next section: eight out of ten verify with UVM, UVM is written in this language, and on days 1 and 2 everything UVM will later hand you ready-made gets written by hand. Thirty keywords are enough.
Introduction
What is UVM?
- Universal Verification Methodology: a class library written in SystemVerilog, plus a pile of conventions on how to use it
- It is not a language and it is not a simulator. Everything we are going to see in the course can be written by hand — and on days 1 and 2 we are going to write it by hand
- It is made by Accellera, the same consortium that standardized SystemVerilog, and it is
IEEE 1800.2. The course uses the reference implementation,
uvm-core 2020.3.1 - It comes from OVM (Cadence + Mentor), with ideas from VMM (Synopsys) and from eRM (Verisity): the methodologies of four vendors that competed, until the industry got tired of translating testbenches between tools
- And there is the goal, and it is a single one: that the testbench of the person next to you looks like yours
The link with what came before is one line: eight out of ten projects verify with this. And the clarification that keeps UVM from looking like the answer to the question of the day: it is not. The answer is a testbench with a plan, a check and a measure; UVM is the way to write it so it scales and so someone else recognizes it. The sentence that orders the section is the one in the second bullet, and it is worth saying it loudly because it lowers the anxiety of whoever arrives scared: UVM does nothing you could not do yourself. There is no magic inside. There is a factory, some ports, some phases and a lot of agreement about names. If somebody asks why bother then: because of the last bullet. A verification engineer who changes project —or company— opens the testbench and recognizes the structure on day one. That is what you are buying. It is not simulation speed and it is not fewer lines: it is that the structure is the same everywhere. The historical note is useful for whoever comes from the industry: until 2011 every vendor had its own, and porting a testbench from VMM to OVM meant rewriting it. UVM exists because that was unsustainable, not because somebody had a better idea. And one clarification that avoids a classic misunderstanding: Open Source here means that the library is free (Apache 2.0). The simulator that runs it almost always is not — except in this course, which is what all of this is about.
Introduction
Where it comes from
The diagram in one sentence: six acronyms from four vendors that converge. eRM was
Verisity's (the e language), RVM and later VMM Synopsys', AVM Mentor's and URM
Cadence's; OVM merged AVM with URM, and it is the base UVM came out of.
What has to be pointed out is the date at the bottom and the jump it means: UVM 1.0 is
from 2011, which means this is young. Somebody with fifteen years of career today
started before it existed.
And the moral that gets used for the rest of the course: when they find code with
ovm_component instead of uvm_component, it is not a typo — it is a
testbench from before 2011 that nobody migrated. It happens more than it seems.
Introduction
A SystemVerilog testbench, from the inside
This is the testbench we are going to write, by hand, on days 1 and 2. It is worth saying it like that: the boxes in this diagram are not theory, they are the files that are going to be open between today and day 4. Point at the tester, the driver, the monitor and the scoreboard, and make the point that none of those names is UVM's: they are the names anybody gives to the pieces of a testbench, in any language. UVM did not invent them; it standardized them.
Introduction
The same testbench, with UVM structure
It is worth staying here a while: this diagram is the map of the rest of the course. Point at driver, monitor, scoreboard and sequencer, and say that each one is going to be a section. By the end of day 5 the student is going to have everything except the agent and the sequencer, which are day 6. The comparison worth making live is against the previous diagram: they are the same boxes. What UVM adds is the tree that contains them, the ports they talk through and the phases that build them in order. Nothing else, and that is not little.
Introduction
The class tree
This diagram is scary and that is why it is worth defusing it right away: out of all of
this we are going to extend ten: uvm_component, uvm_test, uvm_env, uvm_agent,
uvm_driver, uvm_monitor, uvm_subscriber, uvm_sequence_item,
uvm_sequence and uvm_callback. They are the amber ones in the drawing minus the scoreboard
—which in the course is a uvm_subscriber—, plus the sequence and the callback.
What does have to be marked is the split at the top, because it explains
half the course: hanging off uvm_object is the data —the transactions, the
sequences, the configs— and hanging off uvm_component is the structure —everything
that lives in the tree and has phases—. An object is created and thrown away; a component
is built once and lasts the whole simulation.
The colours say what you do with each one: the amber ones you extend yourself, the
grey ones are never touched —they are only there for the others to inherit from—
and uvm_sequencer is the only green one because it is the only one instantiated
without extending it, with a parameterized typedef. That comes on day 6; the only
time the course extends it is for the virtual sequencer on day 7.
The question to throw at the group when we get to agents: is a
uvm_sequence an object or a component? An object. And that is why it does not show up in
print_topology().
It is the same tree that is in the printed cheat sheet, top left, as text, to
look at again when needed.
Introduction
Run more tests writing less code
- That is the promise, and it has one condition: the env and the components barely change from one test to another. What changes is the stimulus
- For that to work, the testbench has to leave hooks: virtual methods, the factory and callbacks. A test hooks in there and does not touch the structure
- A new test is then one of these three things, and none of them touches the env:
- adding constraints to reach a corner case
- doing a factory override to swap one class for another
- injecting errors or delays with callbacks
- And on top of all that, the lever that pays off the most: the same test with hundreds of different seeds
This is the slide you have to be able to repeat at the end of the course, because it is the
whole thesis. And it is worth being honest: "run more tests, write less code" is a vendor
slogan, but the verifiable part is true and you can point at it with your finger on
day 6 — the add_test of the sequences is two lines, and it changes the whole
stimulus without touching a single class of the env.
The order of the three hooks is not accidental, it goes from smallest to largest: a constraint
changes values, an override changes a class, a callback changes behaviour
at one point of the class that already exists. All three are a section of the course —Constrained
random on day 5, The factory pattern on day 2 and Callbacks on day 6—, so
this slide can be put up three more times.
And the last line is the most underestimated one. A test with a thousand seeds is not
"the same test a thousand times": it is a thousand different scenarios for the price of writing
one. It is literally the d7-semillas exercise, the one day 7 starts with.
And the close that leads to the DUT: nothing on this slide answers how do you
know you verified? yet. Answering it is what comes next —the spec, the plan
and a testbench that can be put to the test—, and only with that written by
hand does what UVM adds become visible.
The VTALU spec
- start has to stay at 1 and the operands stable until
the operation finishes - done goes up when the operation finishes: on the multiplication
it is a one-cycle pulse; on add/sub/and/xor it is a level, and it stays
up while start stays up - reset_n is asynchronous and active low; the diagram starts with the DUT
already out of reset - ovf comes out together with done, and it is the borrow of the subtraction: it is 1
whenA < B. For the other operations it is always 0
The protocol is everything you have to respect: start up and operands stable
until done. Careful with the fine print: done is a pulse on the multiplication (it
turns itself off) and a level on add/sub/and/xor, where it is start && (op != no_op). It is
the kind of detail that only shows up when the testbench hangs and nobody
knows why.
The VTALU spec
| Operation | Opcode | Cycles | ovf |
|---|---|---|---|
| no_op | 3'b000 | — | 0 |
| add_op | 3'b001 | 1 | 0 |
| sub_op | 3'b010 | 1 | A < B |
| and_op | 3'b011 | 1 | 0 |
| xor_op | 3'b100 | 1 | 0 |
| mul_op | 3'b101 | 4 | 0 |
| free | 3'b110 | 1 | 0 |
| rst_op | 3'b111 | — | — |
- Cycles counts
clkedges withstartup, up to and including the one that raisesdone: one on the single-cycle ops, four on the multiplication. The plan and the day 7 assertions count the same way rst_opis not decoded by the RTL — the DUT sees it as unused. It is the testbench convention for pulsingreset_n, and that is why it is in theoperation_tof the testbench3'b110is free on purpose: it is today's exercise
rst_op belongs to the testbench, not to the RTL: the DUT sees 3'b111 as unused. It is in the enum because the tester asks for the reset as if it were one more operation, and that way the reset enters the functional coverage. A good moment to preview that the coverage plan includes "any operation after a reset". The ovf column is the one that gets the most discussion, and that is good: with 8-bit operands and a 16-bit result, neither the addition nor the multiplication can overflow — the subtraction can, and only when A < B. Which means there is an output of the DUT that for five of the six operations is always 0. Ask out loud: how do you verify a signal that almost never moves? The answer is today's, in functional coverage, and it is a bin. And the free opcode: 3'b110 is not there by chance and not by oversight. It is this afternoon's exercise, and it is worth saying now so that nobody reads it as a design slip.
The VTALU spec
Single Cycle: Add - Sub - AND - XOR
code/vtalu_dut/vtalu_1c.sv · #the-two-resets // The two resets are deliberately different: the one on the result is
// SYNCHRONOUS and the one on done is ASYNCHRONOUS. Not an oversight: the spec.
always_ff @(posedge clk) begin
if (!reset_n) begin
result_1c <= 16'h0000;
ovf_1c <= 1'b0;
end else if (start) begin
ovf_1c <= (op == 3'b010) && (A < B);
case (op)
3'b001: result_1c <= {8'h00, A} + {8'h00, B};
3'b010: result_1c <= {8'h00, A} - {8'h00, B};
3'b011: result_1c <= {8'h00, A} & {8'h00, B};
3'b100: result_1c <= {8'h00, A} ^ {8'h00, B};
default: ; // 3'b110 is free: it is the day 1 exercise
endcase
end
end
// done goes high on the edge if start is high and the op is not no_op
always_ff @(posedge clk or negedge reset_n) begin
if (!reset_n) done_1c <= 1'b0;
else done_1c <= start && (op != 3'b000);
endel archivo entero, 44 líneas
// The single-cycle operations: add, sub, and, xor. Registers A op B on the edge.
//
// Subtraction is the only one that can overflow: with 8-bit operands and a
// 16-bit result, neither the addition nor the multiplication can. A - B with
// A < B can, and that is why ovf_1c exists. The result in that case is the
// two's complement truncated to 16 bits -- it wraps to 0xFFxx -- which is what
// the hardware does and what the scoreboard has to predict correctly.
module vtalu_1c (
input logic [ 7:0] A,
input logic [ 7:0] B,
input logic clk,
input logic [ 2:0] op,
input logic reset_n,
input logic start,
output logic done_1c,
output logic ovf_1c,
output logic [15:0] result_1c
);
// The two resets are deliberately different: the one on the result is
// SYNCHRONOUS and the one on done is ASYNCHRONOUS. Not an oversight: the spec.
always_ff @(posedge clk) begin
if (!reset_n) begin
result_1c <= 16'h0000;
ovf_1c <= 1'b0;
end else if (start) begin
ovf_1c <= (op == 3'b010) && (A < B);
case (op)
3'b001: result_1c <= {8'h00, A} + {8'h00, B};
3'b010: result_1c <= {8'h00, A} - {8'h00, B};
3'b011: result_1c <= {8'h00, A} & {8'h00, B};
3'b100: result_1c <= {8'h00, A} ^ {8'h00, B};
default: ; // 3'b110 is free: it is the day 1 exercise
endcase
end
end
// done goes high on the edge if start is high and the op is not no_op
always_ff @(posedge clk or negedge reset_n) begin
if (!reset_n) done_1c <= 1'b0;
else done_1c <= start && (op != 3'b000);
end
endmodule : vtalu_1c- Two
always_ffand nothing else: one registersA op B, the other raisesdone - The resets are not the same: the one on the result is synchronous —only
clkin the sensitivity list—, the one ondoneis asynchronous done <= start && (op != no_op): that is why on these four operationsdoneis a level and stays up whilestartis
The asymmetry of the two resets is not an oversight: it comes from the original VHDL and is
copied as is on purpose. It is exactly the kind of detail a testbench
has to expose, and that is why the DUT was not "cleaned up" when it was translated.
Say it out loud: this contradicts the bare asynchronous of the spec bullet,
and no row of the plan looks at what the reset leaves on the outputs: the reset
row looks at the operation that comes after. It is a gap in the plan, and for
now it stays that way.
That done is a level here and a pulse on the multiplication is the fine print that
is going to hang the first testbench of the course. It is worth going back to the waveform
of the first slide and pointing at it again.
The default: ; of the case is what makes no_op and the unused opcodes not
change the result: the ALU keeps whatever it computed last.
The VTALU spec
Multi Cycle: Multiplication
code/vtalu_dut/vtalu_mult.sv · #the-pipeline // Four register stages. done is pipelined alongside the data, and each
// stage clears itself with (& ~done_mult): that is what makes done a
// one-cycle pulse instead of staying high for as long as start does. It is
// subtle -- it is copied straight from the VHDL.
always_ff @(posedge clk or negedge reset_n) begin
if (!reset_n) begin
done_mult <= 1'b0;
done3 <= 1'b0;
done2 <= 1'b0;
done1 <= 1'b0;
a_int <= 8'h00;
b_int <= 8'h00;
mult1 <= 16'h0000;
mult2 <= 16'h0000;
result_mult <= 16'h0000;
end else begin
a_int <= A;
b_int <= B;
mult1 <= a_int * b_int;
mult2 <= mult1;
result_mult <= mult2;
done3 <= start & ~done_mult;
done2 <= done3 & ~done_mult;
done1 <= done2 & ~done_mult;
done_mult <= done1 & ~done_mult;
end
endel archivo entero, 46 líneas
// Multi-cycle multiplication: done rises on the FOURTH clock edge after start
// (the one-cycle ops raise it on the first). The latency is deliberate: it is
// what forces the testbench to wait for 'done' instead of counting cycles.
module vtalu_mult (
input logic [ 7:0] A,
input logic [ 7:0] B,
input logic clk,
input logic reset_n,
input logic start,
output logic done_mult,
output logic [15:0] result_mult
);
logic [7:0] a_int, b_int; // input registers
logic [15:0] mult1, mult2; // pipeline registers
logic done3, done2, done1; // done travels down the same pipeline
// Four register stages. done is pipelined alongside the data, and each
// stage clears itself with (& ~done_mult): that is what makes done a
// one-cycle pulse instead of staying high for as long as start does. It is
// subtle -- it is copied straight from the VHDL.
always_ff @(posedge clk or negedge reset_n) begin
if (!reset_n) begin
done_mult <= 1'b0;
done3 <= 1'b0;
done2 <= 1'b0;
done1 <= 1'b0;
a_int <= 8'h00;
b_int <= 8'h00;
mult1 <= 16'h0000;
mult2 <= 16'h0000;
result_mult <= 16'h0000;
end else begin
a_int <= A;
b_int <= B;
mult1 <= a_int * b_int;
mult2 <= mult1;
result_mult <= mult2;
done3 <= start & ~done_mult;
done2 <= done3 & ~done_mult;
done1 <= done2 & ~done_mult;
done_mult <= done1 & ~done_mult;
end
end
endmodule : vtalu_mult- It is a pipeline: the operands get registered, they get multiplied, and the product
goes through two more registers before coming out on
result_mult: four edges - The
donetravels down the same chain —done3,done2,done1— so it arrives exactly with the data and not before - The
& ~done_multof each stage is what turns the chain off on its own: that is why heredoneis a one-cycle pulse, even ifstartstays up
The four-edge latency is on purpose, and it is the reason the conventional testbench exists:
it forces the testbench to wait for done instead of reading the result on the next
cycle. A combinational DUT would teach nothing.
The & ~done_mult is subtle and worth reading slowly: without it, while start
was up the chain would reload itself and done would stay at 1. It is
copied from the original VHDL as is.
The question that orders the slide: what happens if the testbench sends a
multiplication and reads the result on the next cycle? It reads the previous result,
and the scoreboard reports an error that is not in the DUT.
The VTALU spec
Top Level
code/vtalu_dut/vtalu.sv · #the-mux assign es_mult = (op == 3'b101);
assign start_1c = es_mult ? 1'b0 : start;
assign start_mult = es_mult ? start : 1'b0;
assign result_mux = es_mult ? result_mult : result_1c;
assign done = es_mult ? done_mult : done_1c;el archivo entero, 75 líneas
// VTALU top: A, B, clk, op[2:0], reset_n, start -> done, ovf, result[15:0].
//
// The op is NOT decided by a single loose bit any more: there is a decode, which
// is what a real design does, and what lets the operations avoid having to sit
// in tidy halves of the opcode space.
//
// no_op 3'b000 and_op 3'b011
// add_op 3'b001 xor_op 3'b100
// sub_op 3'b010 mul_op 3'b101
// 3'b110 free -- the day 1 exercise
// rst_op 3'b111 never reaches the DUT: the BFM turns it into lowering reset_n
module vtalu (
input logic [ 7:0] A,
input logic [ 7:0] B,
input logic clk,
input logic [ 2:0] op,
input logic reset_n,
input logic start,
output logic done,
output logic ovf,
output logic [15:0] result
);
logic done_1c, done_mult;
logic ovf_1c;
logic [15:0] result_1c, result_mult, result_mux;
logic start_1c, start_mult;
logic es_mult;
assign es_mult = (op == 3'b101);
assign start_1c = es_mult ? 1'b0 : start;
assign start_mult = es_mult ? start : 1'b0;
assign result_mux = es_mult ? result_mult : result_1c;
assign done = es_mult ? done_mult : done_1c;
// The multiplication cannot overflow: 8 x 8 fits exactly in 16 bits.
assign ovf = es_mult ? 1'b0 : ovf_1c;
// The injectable mutation -- the only liberty taken in this DUT, and the same
// one apb_regs.sv takes with its bug_en. With +VTALU_BUG, bit 0 of the result
// comes out flipped for every operation. Nothing turns it on in a normal run:
// it is 'make mutante' that runs the examples with it and DEMANDS that they
// fail. A scoreboard that never caught anything is not tested, it is
// decoration -- and until this existed there was no way of telling one from
// the other.
// It hangs off the decode above and not inside it so that the four lines the
// day 1 slide quotes stay the plain mux they claim to be.
bit bug;
initial bug = $test$plusargs("VTALU_BUG");
assign result = result_mux ^ {15'h0000, bug};
vtalu_1c u_1c (
.A (A),
.B (B),
.clk (clk),
.op (op),
.reset_n (reset_n),
.start (start_1c),
.done_1c (done_1c),
.ovf_1c (ovf_1c),
.result_1c(result_1c)
);
vtalu_mult u_mult (
.A (A),
.B (B),
.clk (clk),
.reset_n (reset_n),
.start (start_mult),
.done_mult (done_mult),
.result_mult(result_mult)
);
endmodule : vtalu- The top computes nothing: it instantiates the two blocks and decodes the opcode
es_mult = (op == mul_op). Thestartis routed to only one of them, andresult_muxanddonecome out of the same side.ovffollows the same pattern right below the excerpt —assign ovf = es_mult ? 1'b0 : ovf_1c;— andresult_muxonly comes out throughresultafter the+VTALU_BUGXOR- That is why the bus protocol is a single one even though inside there are two different
latencies: what the testbench sees is
start→done
That both blocks share A, B and clk and only split on the start
is what makes the DUT have a single interface. Worth pointing out because it is the
shape the vtalu_bfm of interfaces and BFM is going to have.
The complete file, with both instances and their connections, is in
code/vtalu_dut/vtalu.sv. Here there are only the decode and the four lines that decide.
Worth pointing out why it is a decode and not a loose bit: with an op[2] the
opcodes would have to sit in tidy halves of the space, and the design
would lose the freedom to assign them as convenient. A decode costs one gate
and does not tie the spec down. And along the way, the free opcode 3'b110 falls on the
single-cycle side without anybody having to do anything. The DUT does not validate the
opcode; the one that has to catch it is the verification plan.
The VTALU spec
How to run it
- The DUT lives in
code/vtalu_dut/, in SystemVerilog - The simulator of the course is Verilator (free, no licence):
make u4/tests # one UVM example: ~1 min 30 the first time
make u4/tests # the second, with ccache: ~15 seconds
make # the 38 examples
- The ones that use UVM compile the whole library the first time. Install
ccachebefore starting: the course detects it on its own and does not recompile twice - Every example has its
run.sh; the common flags are incode/verilator/common.sh
Here it is worth leaving the deck and running it live. The numbers on the slide are
measured on a 12-core laptop. On a 2-core machine —a free Codespaces— the
first one is 4 minutes; the second is still 15 seconds, because a ccache hit
is copying a file and that does not depend on how many cores you have. Which means
the expensive part is paid once.
What has to be explained is where the difference between 1 min 30 and 15
seconds comes from, because it is not magic: Verilator compiles every simulation to a native
binary, and for that it generates about 2300 C++ files —almost all of them from the UVM library—.
ccache stores the result of each one; the second time it does not compile again,
it copies. It is in the Dockerfile and in the devcontainer, and outside it is enough to have it
installed.
And the one to stress anyway: Verilator is free — the student takes it
home and keeps practising without asking anybody for a licence.
The VTALU spec
Looking at the real waves
- The diagram on the first slide is drawn. The real one you generate yourself, and it is the tool 47 % of the work gets done with: debugging
cd code/u2/convencional
VLT_TRACE=1 bash run.sh # compiles with --trace and dumps vtalu.vcd
gtkwave vtalu.vcd # or surfer, or whatever viewer you use
VLT_TRACE=1turns on two things at once: Verilator's--traceand the top's$dumpfile/$dumpvars, which without the flag would not compile- It costs nothing and it is free, same as the simulator. When the testbench hangs
on
while (done == 0), this is what is going to tell you why
It is worth opening it live once, even for thirty seconds: load
op_set, start and done, and show the one-cycle pulse of the multiplication
next to the level of the single-cycle ones. It is the fine print of the protocol of
slide 1, but seen.
And say it explicitly: whoever takes the course alone has their safety net here.
When an exercise does not work out, the answer is almost always in the waves before
it is in the code.
The VTALU spec
Summary of the unit
- The protocol is a single sentence, and it has to be respected:
startat 1 and the operands stable untildonegoes up - The fine print that is going to hang the first testbench:
doneis a one-cycle pulse on the multiplication and a level on the single-cycle ones - Inside there are two blocks with different latencies —one and four cycles— and the top decodes the opcode. Outside, the bus is a single one
ovfis one more output, and it belongs tosub_opand to nobody elserst_opdoes not exist for the RTL: it is a testbench convention for pulsingreset_n, and that is why it enters the coverage- The DUT does not validate the opcode: the free one raises
doneall the same, with theresultof the previous operation. Catching it is the verification plan's job, not the design's - And the tool used 47 % of the time is already installed:
VLT_TRACE=1and GTKWave
Closing of the section that looks like RTL and is really about verification. The
question to close with: which of these seven facts is the most expensive one to
forget? The one about done, and it will show today, in the waves exercise.
Worth saying why the DUT was not "cleaned up" when it was translated from VHDL: the
asymmetry of the resets and the unvalidated opcode are exactly the kind of
thing a testbench has to expose. A tidy DUT teaches nothing.
The verification plan
The most professional deliverable of the discipline, and the cheapest
- It is a table. It gets written before the testbench — if it gets written after, it describes what the testbench already does instead of what the spec asks for
- It comes out of the spec you have just read, not out of the testbench that does not exist yet
- And it is what gets delivered: the day 7 capstone is handed in with the plan filled, which is exactly what gets handed in on a project
This is the slide that orders the whole course, and that is why it comes before the first
line of testbench. Everything that comes after —the stimulus, the scoreboard, the
covergroups, the assertions— are columns of this table.
It is worth saying it head on: the plan is not bureaucracy. It is the only serious
answer to "are you done verifying yet?", and it is the first thing a technical
lead asks for when they ask how the block is going.
The empty template is in docs/en/verification-plan.md, and it gets used twice in
the course: here to read it, and on day 7 to fill it from scratch.
The verification plan
The five columns
| Column | The question it answers | What does not go in |
|---|---|---|
| Feature | which part of the spec does this row come from? | the name of a testbench file |
| Scenario | which concrete situation has to be provoked? | "test the ALU" |
| Stimulus | who provokes it: the random or a directed case? | "by hand" |
| Check | who says it went right? | "you look at the wave" |
| Measure | which bin gets filled when it happens? | "you see it in the log" |
- One row per scenario, not per feature: a feature with three corner cases is three rows, and they get closed one at a time
It is worth reading the right-hand column out loud: they are the five mistakes people make the first time, and all five sound reasonable when you write them. The hardest column is the measure one, because it forces you to decide beforehand what you are going to count. It is the one that on day 1 still cannot be filled —the bins do not exist until the next unit— and that is why the plan gets filled as the testbench grows.
The verification plan
Two rules that come out of the columns
- *If the check column says "by eye", the scenario is not verified.* It is simulated, which is another thing
- If the measure column is empty, nobody is going to find out that the scenario never happened. A case the random never touched and that has no bin is indistinguishable from one that happened a thousand times
- And one distinction that orders the check column: the scoreboard checks what the DUT computes; the assertions check how it is talked to
The difference between verified and simulated is who finds out when it fails at three in the morning in the regression. If the check is a pair of eyes, nobody finds out: the regression passes green with the bug inside. The third bullet is a promise the course keeps on day 7: until then every check is a scoreboard check, and half the protocol goes unchecked. Worth saying now so that day 7 does not look like an add-on. A serious plan has both columns, and the VTALU table —which is seen whole in the next unit, already with the bins— has eight scoreboard rows and four assertion ones.
The conventional testbench
The VTALU plan, with three columns to fill
| Feature | Scenario | Stimulus | Check | Measure |
|---|---|---|---|---|
| ALU | the six operations | |||
| ALU | operands at 00 and at FF |
|||
| reset | operate after a reset | |||
| mult | a mult after a single-cycle one | |||
| ALU | the same operation twice in a row | |||
| sub | subtract too much: A < B, and ovf goes up |
- Six rows that come out of the spec, and none of the right-hand columns yet. This section fills two: the stimulus provokes them and the scoreboard says whether the result was right
- The third one —which bin gets filled when it happens— is the next section
- The testbench has three parts because the plan has three columns to fill
The order matters and it is the one from the previous unit: the plan first,
the testbench after. If you write the testbench first, you end up measuring
what the testbench does instead of what the spec asks for.
Worth reading the six rows one at a time and asking which one is missing. The
answer that usually comes up —"test intermediate values"— is good for
discussing why it is not there: 256 × 256 × 5 combinations do not get
simulated, and that is where random comes in.
The last one is the one that brings today's topic: the DUT has TWO outputs, and
a row of the plan that only closes by looking at both. A scoreboard that
compares result and nothing else passes green with ovf stuck at zero.
The table comes back twice: in functional coverage with the measure column
filled in, row by row, and at the end of that whole section, with the twelve
rows and the three columns. Worth announcing now so nobody reads it as
decoration: it is the same table, filling up.
The conventional testbench
The stimulus: a thousand operations, and the protocol by hand
code/u2/convencional/vtalu_tb.sv · #stimulus-loop repeat (1000) begin
@(negedge clk);
op_set = get_op();
A = get_data();
B = get_data();
start = 1'b1;
case (op_set) // handle the start signal
no_op: begin
@(posedge clk);
start = 1'b0;
end
rst_op: begin
reset_n = 1'b0;
start = 1'b0;
@(negedge clk);
reset_n = 1'b1;
end
default: begin
// Wait for an EDGE with done up, not for done itself: on the
// one-cycle ops done is a level still up from the previous op,
// and a plain wait(done) drops this one without a sound.
enviadas++;
do @(negedge clk); while (done == 0);
start = 1'b0;
end
endcase // case (op_set)
end
$finish;el archivo entero, 256 líneas
// Typical TB:
// ----------
// * Functional Coverage
// * Self Checking
// * Constrained Random Stimulus
// Why is this bad?
// * No modularity (everything in a single file)
// * Almost nothing here can be reused
module top;
typedef enum bit[2:0] {no_op = 3'b000,
add_op = 3'b001,
sub_op = 3'b010,
and_op = 3'b011,
xor_op = 3'b100,
mul_op = 3'b101,
rst_op = 3'b111} operation_t;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);
`ifdef VLT_TRACE
// Waves for GTKWave: VLT_TRACE=1 bash run.sh && gtkwave vtalu.vcd
// Behind an `ifdef because $dumpvars does not compile without --trace, and
// the define comes from the same VLT_TRACE that adds the flag (common.sh).
initial begin
$dumpfile("vtalu.vcd");
$dumpvars;
end
`endif
//op_cov: make sure every operation, and every interaction between them, is covered
covergroup op_cov;
coverpoint op_set {
bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
bins multi_cycle = {mul_op};
// => are transitional coverage. From => To
// Transition bins. Verilator 5.052 does NOT implement them yet: any form
// with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
// code/verilator/repro-cg-transition.sv. They stay because they are part
// of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
bins opn_rst[] = ([add_op:mul_op] => rst_op);
bins rst_opn[] = (rst_op => [add_op:mul_op]);
bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);
bins twoops[] = ([add_op:mul_op] [* 2]);
bins manymult = (mul_op [* 3:5]);
`endif
}
endgroup
covergroup zeros_or_ones_on_ops;
// Check the corners where the inputs are all 0s or all 1s
all_ops : coverpoint op_set {
ignore_bins null_ops = {rst_op, no_op};}
a_leg: coverpoint A {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
b_leg: coverpoint B {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
// VTALU rev2: the borrow of the subtraction, the row of the plan that the
// scoreboard can only close by looking at TWO outputs. Without this bin, a
// subtraction that never went negative looks exactly like one that did.
borrow: coverpoint ((op_set == sub_op) && (A < B)) {
bins hubo_borrow = {1};
ignore_bins resto = {0};
}
op_00_FF: cross a_leg, b_leg, all_ops {
bins add_00 = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins add_FF = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins sub_00 = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins sub_FF = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins and_00 = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins and_FF = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins xor_00 = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins xor_FF = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_00 = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins mul_FF = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_max = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) && binsof (b_leg.ones));
ignore_bins others_only =
binsof(a_leg.others) && binsof(b_leg.others);
}
endgroup
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
end
op_cov oc;
zeros_or_ones_on_ops c_00_FF;
initial begin : coverage
oc = new();
c_00_FF = new();
forever begin @(posedge clk);
oc.sample();
c_00_FF.sample();
end
end : coverage
function operation_t get_op();
bit [2:0] op_choice;
op_choice = $random;
case (op_choice)
3'b000 : return no_op;
3'b001 : return add_op;
3'b010 : return sub_op;
3'b011 : return and_op;
3'b100 : return xor_op;
3'b101 : return mul_op;
3'b110 : return rst_op;
3'b111 : return rst_op;
endcase // case (op_choice)
endfunction : get_op
function byte get_data();
// This is how the random gets biased
bit [1:0] zero_ones;
zero_ones = $random;
if (zero_ones == 2'b00)
return 8'h00;
else if (zero_ones == 2'b11)
return 8'hFF;
else
return $random;
endfunction : get_data
// Sent by the tester, compared by the scoreboard: the final block below
// demands that the two match.
int enviadas, chequeadas;
// Scoreboard loop: checks that everything came out right
always @(posedge done) begin : scoreboard
shortint predicted_result;
bit predicted_ovf;
#1;
case (op_set)
add_op: predicted_result = A + B;
sub_op: predicted_result = A - B;
and_op: predicted_result = A & B;
xor_op: predicted_result = A ^ B;
mul_op: predicted_result = A * B;
endcase // case (op_set)
// ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
// output, neither the addition nor the multiplication can overflow.
predicted_ovf = (op_set == sub_op) && (A < B);
if ((op_set != no_op) && (op_set != rst_op)) begin
chequeadas++;
if (predicted_result != result || predicted_ovf != ovf)
$error ("FAILED: A: %0h B: %0h op: %s result: %0h ovf: %0b",
A, B, op_set.name(), result, ovf);
end
end : scoreboard
// Randomize the stimulus
// get_op and get_data are Constrained Random Data
// The point of CRD is to build values that are random but legal
initial begin : tester
reset_n = 1'b0;
@(negedge clk);
@(negedge clk);
reset_n = 1'b1;
start = 1'b0;
repeat (1000) begin
@(negedge clk);
op_set = get_op();
A = get_data();
B = get_data();
start = 1'b1;
case (op_set) // handle the start signal
no_op: begin
@(posedge clk);
start = 1'b0;
end
rst_op: begin
reset_n = 1'b0;
start = 1'b0;
@(negedge clk);
reset_n = 1'b1;
end
default: begin
// Wait for an EDGE with done up, not for done itself: on the
// one-cycle ops done is a level still up from the previous op,
// and a plain wait(done) drops this one without a sound.
enviadas++;
do @(negedge clk); while (done == 0);
start = 1'b0;
end
endcase // case (op_set)
end
$finish;
end : tester
// The testbench checks itself: every operation sent has to come back
// through the scoreboard. A tester that drops operations -- while the
// coverage counts them anyway -- passes green and has measured nothing.
final begin
if (enviadas != chequeadas)
$error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
else
$display ("%0d operations sent, %0d checked", enviadas, chequeadas);
end
endmodule : top- A thousand rounds: pick the operation, pick the operands, raise
start, wait for an edge withdoneup, lowerstart - The
caseis there because of the fine print of the spec:no_opdoes not raisedone,rst_oppulsesreset_n, and the rest wait get_op()andget_data()bias the random towards the edges —00 and FF— to reach the cases the plan asks for. It is constrained random written by handenviadasgets counted here andchequeadasin the scoreboard, and afinaldemands they match: it is the testbench checking itself- Notice who knows about the protocol here: the tester. In interfaces and BFM that moves
to the BFM and in the body of the
repeatshrinks to one line
This is the "before" slide of the whole course: the stimulus and the protocol mixed
in the same loop. Worth pointing at it, because the next five units
are successive separations of these thirty lines.
The do … while (done == 0) of the default is the one that hangs if the DUT does
not answer, and it is the place the student is going to end up in the first time they
break something. The safety net is VLT_TRACE=1 and the waves.
And why it is not a wait(done), which is the first thing anybody writes: on the
one-cycle operations done is a level, and on the negedge where the tester loads
the next operation it is still up from the previous one. The wait(done) does
not block, start drops at the same instant and the DUT never sees that operation.
There is no error: the scoreboard does not fire, and the coverage —which samples
op_set— counts it anyway. This testbench had it, and it dropped four in ten
one-cycle operations with the report in green: on d1, 77 shifts checked
against 130 with the fix. What catches it is the sent-versus-
checked counter, and it is the cheapest silent trap to guard against.
The bias of get_data() —a quarter at 00, a quarter at FF, half in the
middle— is exactly what the transactions are going to write in one line with dist
and :/. Worth naming it now so that the saving is visible later.
The conventional testbench
The self-checking: predict and compare
code/u2/convencional/vtalu_tb.sv · #scoreboard-block // Sent by the tester, compared by the scoreboard: the final block below
// demands that the two match.
int enviadas, chequeadas;
// Scoreboard loop: checks that everything came out right
always @(posedge done) begin : scoreboard
shortint predicted_result;
bit predicted_ovf;
#1;
case (op_set)
add_op: predicted_result = A + B;
sub_op: predicted_result = A - B;
and_op: predicted_result = A & B;
xor_op: predicted_result = A ^ B;
mul_op: predicted_result = A * B;
endcase // case (op_set)
// ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
// output, neither the addition nor the multiplication can overflow.
predicted_ovf = (op_set == sub_op) && (A < B);
if ((op_set != no_op) && (op_set != rst_op)) begin
chequeadas++;
if (predicted_result != result || predicted_ovf != ovf)
$error ("FAILED: A: %0h B: %0h op: %s result: %0h ovf: %0b",
A, B, op_set.name(), result, ovf);
end
end : scoreboardel archivo entero, 256 líneas
// Typical TB:
// ----------
// * Functional Coverage
// * Self Checking
// * Constrained Random Stimulus
// Why is this bad?
// * No modularity (everything in a single file)
// * Almost nothing here can be reused
module top;
typedef enum bit[2:0] {no_op = 3'b000,
add_op = 3'b001,
sub_op = 3'b010,
and_op = 3'b011,
xor_op = 3'b100,
mul_op = 3'b101,
rst_op = 3'b111} operation_t;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);
`ifdef VLT_TRACE
// Waves for GTKWave: VLT_TRACE=1 bash run.sh && gtkwave vtalu.vcd
// Behind an `ifdef because $dumpvars does not compile without --trace, and
// the define comes from the same VLT_TRACE that adds the flag (common.sh).
initial begin
$dumpfile("vtalu.vcd");
$dumpvars;
end
`endif
//op_cov: make sure every operation, and every interaction between them, is covered
covergroup op_cov;
coverpoint op_set {
bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
bins multi_cycle = {mul_op};
// => are transitional coverage. From => To
// Transition bins. Verilator 5.052 does NOT implement them yet: any form
// with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
// code/verilator/repro-cg-transition.sv. They stay because they are part
// of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
bins opn_rst[] = ([add_op:mul_op] => rst_op);
bins rst_opn[] = (rst_op => [add_op:mul_op]);
bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);
bins twoops[] = ([add_op:mul_op] [* 2]);
bins manymult = (mul_op [* 3:5]);
`endif
}
endgroup
covergroup zeros_or_ones_on_ops;
// Check the corners where the inputs are all 0s or all 1s
all_ops : coverpoint op_set {
ignore_bins null_ops = {rst_op, no_op};}
a_leg: coverpoint A {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
b_leg: coverpoint B {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
// VTALU rev2: the borrow of the subtraction, the row of the plan that the
// scoreboard can only close by looking at TWO outputs. Without this bin, a
// subtraction that never went negative looks exactly like one that did.
borrow: coverpoint ((op_set == sub_op) && (A < B)) {
bins hubo_borrow = {1};
ignore_bins resto = {0};
}
op_00_FF: cross a_leg, b_leg, all_ops {
bins add_00 = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins add_FF = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins sub_00 = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins sub_FF = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins and_00 = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins and_FF = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins xor_00 = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins xor_FF = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_00 = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins mul_FF = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_max = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) && binsof (b_leg.ones));
ignore_bins others_only =
binsof(a_leg.others) && binsof(b_leg.others);
}
endgroup
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
end
op_cov oc;
zeros_or_ones_on_ops c_00_FF;
initial begin : coverage
oc = new();
c_00_FF = new();
forever begin @(posedge clk);
oc.sample();
c_00_FF.sample();
end
end : coverage
function operation_t get_op();
bit [2:0] op_choice;
op_choice = $random;
case (op_choice)
3'b000 : return no_op;
3'b001 : return add_op;
3'b010 : return sub_op;
3'b011 : return and_op;
3'b100 : return xor_op;
3'b101 : return mul_op;
3'b110 : return rst_op;
3'b111 : return rst_op;
endcase // case (op_choice)
endfunction : get_op
function byte get_data();
// This is how the random gets biased
bit [1:0] zero_ones;
zero_ones = $random;
if (zero_ones == 2'b00)
return 8'h00;
else if (zero_ones == 2'b11)
return 8'hFF;
else
return $random;
endfunction : get_data
// Sent by the tester, compared by the scoreboard: the final block below
// demands that the two match.
int enviadas, chequeadas;
// Scoreboard loop: checks that everything came out right
always @(posedge done) begin : scoreboard
shortint predicted_result;
bit predicted_ovf;
#1;
case (op_set)
add_op: predicted_result = A + B;
sub_op: predicted_result = A - B;
and_op: predicted_result = A & B;
xor_op: predicted_result = A ^ B;
mul_op: predicted_result = A * B;
endcase // case (op_set)
// ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
// output, neither the addition nor the multiplication can overflow.
predicted_ovf = (op_set == sub_op) && (A < B);
if ((op_set != no_op) && (op_set != rst_op)) begin
chequeadas++;
if (predicted_result != result || predicted_ovf != ovf)
$error ("FAILED: A: %0h B: %0h op: %s result: %0h ovf: %0b",
A, B, op_set.name(), result, ovf);
end
end : scoreboard
// Randomize the stimulus
// get_op and get_data are Constrained Random Data
// The point of CRD is to build values that are random but legal
initial begin : tester
reset_n = 1'b0;
@(negedge clk);
@(negedge clk);
reset_n = 1'b1;
start = 1'b0;
repeat (1000) begin
@(negedge clk);
op_set = get_op();
A = get_data();
B = get_data();
start = 1'b1;
case (op_set) // handle the start signal
no_op: begin
@(posedge clk);
start = 1'b0;
end
rst_op: begin
reset_n = 1'b0;
start = 1'b0;
@(negedge clk);
reset_n = 1'b1;
end
default: begin
// Wait for an EDGE with done up, not for done itself: on the
// one-cycle ops done is a level still up from the previous op,
// and a plain wait(done) drops this one without a sound.
enviadas++;
do @(negedge clk); while (done == 0);
start = 1'b0;
end
endcase // case (op_set)
end
$finish;
end : tester
// The testbench checks itself: every operation sent has to come back
// through the scoreboard. A tester that drops operations -- while the
// coverage counts them anyway -- passes green and has measured nothing.
final begin
if (enviadas != chequeadas)
$error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
else
$display ("%0d operations sent, %0d checked", enviadas, chequeadas);
end
endmodule : top- An
always @(posedge done): every time the DUT says it finished, the scoreboard predicts the result and compares it - The
#1is not decoration: without it the signals get read at the same instantdonegoes up and a delta race can be swallowed no_opandrst_opdo not raisedone, so this block should not run with them. Theifis defensive —with no prediction for those two it would compare garbage— and inside goeschequeadas++, the other half of the counter- The covergroup —the third leg— is the section that follows
The reference model is four lines of case because the DUT is an ALU. On
a real project it can be a model in C or the RTL of the previous
generation, but the shape is always this one: predict with something that is not the DUT, and
compare.
That the scoreboard reads A, B and op_set off the wire at the moment of the result
works only because the protocol forces them to stay stable. It is fragile, and in
the analysis ports it will be replaced by a queue of commands the monitor sends.
And the $error here is going to be a `uvm_error from reporting on: same
concept, with a counter and a severity policy behind it.
The conventional testbench
How do you know the scoreboard checks anything?
code/u2/convencional/mutante.txtFAILED: A: 15 B: ce op: xor_op result: da ovf: 0- A thousand operations and not one error. Now the same testbench with the
DUT broken on purpose:
VTALU_BUG=1 bash run.shflips bit 0 of the result - It fails on the first comparison. That —and not the earlier
PASS— is what proves the scoreboard looks at the result - A scoreboard that never saw an error is not proven: it may have compared against itself, or compared nothing at all
make mutantedoes this with the three testbenches of days 1 and 2, and fails if any of them does not fail. It is the first answer to the question of the day
Run it live, it takes seconds because there is no UVM: first bash run.sh —a
thousand operations, silence, and the coverage summary—, then
VTALU_BUG=1 bash run.sh, which ends in the line on the slide and a $stop.
What has to be said in so many words is what that line proves and what it does
not. It proves the scoreboard compares the bit the bug touched. It does not
prove it compares ovf: a bug that only touched ovf is another run, and that
is why the plan has a row for each output.
This is the short answer to the first slide of the day: the regression that
said PASS and sent the bug to the fab had never seen a FAILED. It did not
know whether it was checking. A PASS says nothing until you know what
FAILED would have said.
And it is not a day-1 trick: it is the practice that runs through the course.
+BUG=1 in the day 7 capstone and +GOLDEN_BUG in the day 8 reference model
are this very thing, and the exercise checkers demand it.
The conventional testbench
Unit summary
- The testbench has three parts because the plan has three columns to fill: stimulus, self-checking and measure. Today they are loose in one file
- The stimulus biases the random towards the edges —
00andFF— because uniform randomness almost never visits them - The
#1before reading the signals is not decoration: without it you read in the very instant the DUT writes, and the result is undefined - A
PASSproves nothing until you have seen theFAILED:VTALU_BUG=1is the way to see it, andmake mutantethe way not to forget - And what to look at for interfaces and BFM: the tester knows the
protocol. It moves
startand waits fordoneby hand
The last bullet is the one that orders the rest of the day: this testbench
works, it is proven, and it is badly divided. The tester knows how start is
wiggled and the scoreboard knows when to read done: the protocol lives in two
places of the same file, and the day it changes both have to be touched.
That is the whole motivation for interfaces and BFM, and it is worth leaving as
an open question instead of answering it here. Before that, the next section
fills in the third column.
Functional coverage
When are you done verifying?
- Code coverage —lines, branches, toggle— the simulator gives you for free. It says which RTL got executed
- Functional coverage you write yourself. It says which scenarios of the spec went through the DUT
- They are not the same thing, and the first one misleads: a test that always sends
add_opwithA=B=8'h01executes almost every line of the adder and verified nothing - Functional coverage is the contract between the spec and the testbench: if a case of the spec has no bin, nobody is going to find out it is missing
The question on the slide is the one a verification engineer gets asked in the tape-out meeting, and it is not answered with "I ran a lot of tests". If somebody asks why code coverage is not enough: because it measures the DUT, not the spec. A DUT that is missing a whole feature can give 100 % of lines — the lines nobody wrote do not show up in the report.
Functional coverage
From the verification plan to the covergroup
The six rows of the plan from the previous section, translated:
| From the plan | How it gets measured |
|---|---|
| the six operations | coverpoint op_set with one bin per operation |
operands at 00 and at FF |
cross of A, B and the operation |
| operate after a reset | transition bin rst_op => op |
| a mult after a single-cycle one | transition bin |
| the same operation twice in a row | repetition bin [* 2] |
subtract too much: A < B, and ovf goes up |
coverpoint borrow with the hubo_borrow bin |
- The six rows stop being good intentions: each one is a line of code, and the simulator tells you which ones have not happened yet
It is worth going back to the plan of the previous section and reading it out loud before showing this table: the point is that there is not one item of the plan left untranslated. The order matters — the plan first, the covergroup after. The other way round you end up measuring what the testbench does, not what the spec asks for.
Functional coverage
Anatomy: it is three steps, not one
covergroup op_cov; // 1. WHAT gets measured
coverpoint op_set;
endgroup
op_cov oc; // 2. it gets INSTANTIATED: it is a type, like a class
initial oc = new();
always @(posedge clk) oc.sample(); // 3. WHEN it gets counted
- A
covergroupis a type: declaring it measures nothing, it has to benew()ed sample()is the one that counts. Withoutsample()the report reads 0 % and the code compiles just the same- Where to sample is a design decision. Here, the edge the DUT reads on, never the same one the tester writes on; when the TB has monitors it will be "every time the monitor sees a command"
Everybody's first-time mistake is declaring the covergroup and forgetting the
new() or the sample(). There is no warning: the report simply says 0 % and you
spend half an hour looking for the bug in the bins.
The third point is the one that holds for the rest of the course: the moment of the sample
defines what you are measuring. Sampling on the clock counts cycles; sampling on
transactions counts operations. In the analysis ports the coverage becomes a
subscriber and samples when a command arrives — which is the right thing.
And the edge is not a whim: sampling on the same negedge the tester writes
op_set on is a race the LRM does not define —Verilator always resolves it the
same way; another simulator may not—. It is the same rule as the day 7
assertions: you read on the opposite edge to the one that writes.
Functional coverage
bins: the buckets that have to be filled
coverpoint op_set; // automatic bins: one per value of the enum
coverpoint op_set {
bins single_cycle[] = {[add_op : xor_op], rst_op, no_op}; // [] -> one per value
bins multi_cycle = {mul_op}; // no [] -> a single one
}
- Without
{ }, SystemVerilog creates one bin per value. That works for anenum; for anintthe tool chops the range into 64 buckets —auto_bin_max— that correspond to no row of the plan - The brackets
[]hand out: one bucket per value of the range. Without them, every value falls into the same bucket - A coverpoint is covered when all of its bins have at least one hit. A bin that cannot be filled pins your number down forever
The brackets are the detail most often copied wrong, and it is worth showing it with
both cases side by side: bins x[] = {[0:3]} is four buckets, and
bins x = {[0:3]} is a single one that gets filled by any of the
four values landing. The syntax differs by two characters and the number the tool
reports changes completely.
The practical consequence has to be said loudly because it is a silent trap: a
bin without [] over a large range always gives 100 %. You covered one value out of
a thousand and the tool tells you it is done. Nobody warns you.
And the last bullet is the warning that orders what comes next: a bin that
cannot be filled is not a tool problem, it is a decision somebody did not write down.
The way to write it down is called ignore_bins, and it has its own slide in this
section.
Functional coverage
The VTALU covergroup
code/u2/convencional/vtalu_tb.sv · #op_cov //op_cov: make sure every operation, and every interaction between them, is covered
covergroup op_cov;
coverpoint op_set {
bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
bins multi_cycle = {mul_op};
// => are transitional coverage. From => To
// Transition bins. Verilator 5.052 does NOT implement them yet: any form
// with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
// code/verilator/repro-cg-transition.sv. They stay because they are part
// of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
bins opn_rst[] = ([add_op:mul_op] => rst_op);
bins rst_opn[] = (rst_op => [add_op:mul_op]);
bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);
bins twoops[] = ([add_op:mul_op] [* 2]);
bins manymult = (mul_op [* 3:5]);
`endif
}
endgroupel archivo entero, 256 líneas
// Typical TB:
// ----------
// * Functional Coverage
// * Self Checking
// * Constrained Random Stimulus
// Why is this bad?
// * No modularity (everything in a single file)
// * Almost nothing here can be reused
module top;
typedef enum bit[2:0] {no_op = 3'b000,
add_op = 3'b001,
sub_op = 3'b010,
and_op = 3'b011,
xor_op = 3'b100,
mul_op = 3'b101,
rst_op = 3'b111} operation_t;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);
`ifdef VLT_TRACE
// Waves for GTKWave: VLT_TRACE=1 bash run.sh && gtkwave vtalu.vcd
// Behind an `ifdef because $dumpvars does not compile without --trace, and
// the define comes from the same VLT_TRACE that adds the flag (common.sh).
initial begin
$dumpfile("vtalu.vcd");
$dumpvars;
end
`endif
//op_cov: make sure every operation, and every interaction between them, is covered
covergroup op_cov;
coverpoint op_set {
bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
bins multi_cycle = {mul_op};
// => are transitional coverage. From => To
// Transition bins. Verilator 5.052 does NOT implement them yet: any form
// with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
// code/verilator/repro-cg-transition.sv. They stay because they are part
// of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
bins opn_rst[] = ([add_op:mul_op] => rst_op);
bins rst_opn[] = (rst_op => [add_op:mul_op]);
bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);
bins twoops[] = ([add_op:mul_op] [* 2]);
bins manymult = (mul_op [* 3:5]);
`endif
}
endgroup
covergroup zeros_or_ones_on_ops;
// Check the corners where the inputs are all 0s or all 1s
all_ops : coverpoint op_set {
ignore_bins null_ops = {rst_op, no_op};}
a_leg: coverpoint A {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
b_leg: coverpoint B {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
// VTALU rev2: the borrow of the subtraction, the row of the plan that the
// scoreboard can only close by looking at TWO outputs. Without this bin, a
// subtraction that never went negative looks exactly like one that did.
borrow: coverpoint ((op_set == sub_op) && (A < B)) {
bins hubo_borrow = {1};
ignore_bins resto = {0};
}
op_00_FF: cross a_leg, b_leg, all_ops {
bins add_00 = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins add_FF = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins sub_00 = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins sub_FF = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins and_00 = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins and_FF = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins xor_00 = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins xor_FF = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_00 = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins mul_FF = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_max = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) && binsof (b_leg.ones));
ignore_bins others_only =
binsof(a_leg.others) && binsof(b_leg.others);
}
endgroup
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
end
op_cov oc;
zeros_or_ones_on_ops c_00_FF;
initial begin : coverage
oc = new();
c_00_FF = new();
forever begin @(posedge clk);
oc.sample();
c_00_FF.sample();
end
end : coverage
function operation_t get_op();
bit [2:0] op_choice;
op_choice = $random;
case (op_choice)
3'b000 : return no_op;
3'b001 : return add_op;
3'b010 : return sub_op;
3'b011 : return and_op;
3'b100 : return xor_op;
3'b101 : return mul_op;
3'b110 : return rst_op;
3'b111 : return rst_op;
endcase // case (op_choice)
endfunction : get_op
function byte get_data();
// This is how the random gets biased
bit [1:0] zero_ones;
zero_ones = $random;
if (zero_ones == 2'b00)
return 8'h00;
else if (zero_ones == 2'b11)
return 8'hFF;
else
return $random;
endfunction : get_data
// Sent by the tester, compared by the scoreboard: the final block below
// demands that the two match.
int enviadas, chequeadas;
// Scoreboard loop: checks that everything came out right
always @(posedge done) begin : scoreboard
shortint predicted_result;
bit predicted_ovf;
#1;
case (op_set)
add_op: predicted_result = A + B;
sub_op: predicted_result = A - B;
and_op: predicted_result = A & B;
xor_op: predicted_result = A ^ B;
mul_op: predicted_result = A * B;
endcase // case (op_set)
// ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
// output, neither the addition nor the multiplication can overflow.
predicted_ovf = (op_set == sub_op) && (A < B);
if ((op_set != no_op) && (op_set != rst_op)) begin
chequeadas++;
if (predicted_result != result || predicted_ovf != ovf)
$error ("FAILED: A: %0h B: %0h op: %s result: %0h ovf: %0b",
A, B, op_set.name(), result, ovf);
end
end : scoreboard
// Randomize the stimulus
// get_op and get_data are Constrained Random Data
// The point of CRD is to build values that are random but legal
initial begin : tester
reset_n = 1'b0;
@(negedge clk);
@(negedge clk);
reset_n = 1'b1;
start = 1'b0;
repeat (1000) begin
@(negedge clk);
op_set = get_op();
A = get_data();
B = get_data();
start = 1'b1;
case (op_set) // handle the start signal
no_op: begin
@(posedge clk);
start = 1'b0;
end
rst_op: begin
reset_n = 1'b0;
start = 1'b0;
@(negedge clk);
reset_n = 1'b1;
end
default: begin
// Wait for an EDGE with done up, not for done itself: on the
// one-cycle ops done is a level still up from the previous op,
// and a plain wait(done) drops this one without a sound.
enviadas++;
do @(negedge clk); while (done == 0);
start = 1'b0;
end
endcase // case (op_set)
end
$finish;
end : tester
// The testbench checks itself: every operation sent has to come back
// through the scoreboard. A tester that drops operations -- while the
// coverage counts them anyway -- passes green and has measured nothing.
final begin
if (enviadas != chequeadas)
$error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
else
$display ("%0d operations sent, %0d checked", enviadas, chequeadas);
end
endmodule : topsingle_cycle[]generates six bins —one per operation—,multi_cyclea single one- What is inside
`ifndef VERILATORare the transition bins, which come further on in this same section
This is the first real covergroup of the course, and it is worth reading it top to
bottom answering the three questions of the previous slide: what gets measured (the
coverpoints), into which buckets (the bins) and when it gets counted (the
sample(), which is further down in the same file).
That multi_cycle is a single bin and single_cycle[] is six is not broken
symmetry: it is the verification plan. The six that are not the multiplication matter
one by one; the multiplication matters as a separate case because it is the only one that takes
more than one cycle. The shape of the covergroup is the table of the plan, and that is why the
plan gets written first.
The `ifndef VERILATOR is worth naming now and not hiding: those bins are
a topic of the course, they are written, and today Verilator does not compile them. It is in
docs/en/verilator.md with the version and the date. A course that covers up what its
tool does not do is a course that lies.
Functional coverage
ignore_bins: saying out loud what does not get covered
code/u2/convencional/vtalu_tb.sv · #legs-and-ops // Check the corners where the inputs are all 0s or all 1s
all_ops : coverpoint op_set {
ignore_bins null_ops = {rst_op, no_op};}
a_leg: coverpoint A {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
b_leg: coverpoint B {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
// VTALU rev2: the borrow of the subtraction, the row of the plan that the
// scoreboard can only close by looking at TWO outputs. Without this bin, a
// subtraction that never went negative looks exactly like one that did.
borrow: coverpoint ((op_set == sub_op) && (A < B)) {
bins hubo_borrow = {1};
ignore_bins resto = {0};
}el archivo entero, 256 líneas
// Typical TB:
// ----------
// * Functional Coverage
// * Self Checking
// * Constrained Random Stimulus
// Why is this bad?
// * No modularity (everything in a single file)
// * Almost nothing here can be reused
module top;
typedef enum bit[2:0] {no_op = 3'b000,
add_op = 3'b001,
sub_op = 3'b010,
and_op = 3'b011,
xor_op = 3'b100,
mul_op = 3'b101,
rst_op = 3'b111} operation_t;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);
`ifdef VLT_TRACE
// Waves for GTKWave: VLT_TRACE=1 bash run.sh && gtkwave vtalu.vcd
// Behind an `ifdef because $dumpvars does not compile without --trace, and
// the define comes from the same VLT_TRACE that adds the flag (common.sh).
initial begin
$dumpfile("vtalu.vcd");
$dumpvars;
end
`endif
//op_cov: make sure every operation, and every interaction between them, is covered
covergroup op_cov;
coverpoint op_set {
bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
bins multi_cycle = {mul_op};
// => are transitional coverage. From => To
// Transition bins. Verilator 5.052 does NOT implement them yet: any form
// with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
// code/verilator/repro-cg-transition.sv. They stay because they are part
// of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
bins opn_rst[] = ([add_op:mul_op] => rst_op);
bins rst_opn[] = (rst_op => [add_op:mul_op]);
bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);
bins twoops[] = ([add_op:mul_op] [* 2]);
bins manymult = (mul_op [* 3:5]);
`endif
}
endgroup
covergroup zeros_or_ones_on_ops;
// Check the corners where the inputs are all 0s or all 1s
all_ops : coverpoint op_set {
ignore_bins null_ops = {rst_op, no_op};}
a_leg: coverpoint A {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
b_leg: coverpoint B {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
// VTALU rev2: the borrow of the subtraction, the row of the plan that the
// scoreboard can only close by looking at TWO outputs. Without this bin, a
// subtraction that never went negative looks exactly like one that did.
borrow: coverpoint ((op_set == sub_op) && (A < B)) {
bins hubo_borrow = {1};
ignore_bins resto = {0};
}
op_00_FF: cross a_leg, b_leg, all_ops {
bins add_00 = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins add_FF = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins sub_00 = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins sub_FF = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins and_00 = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins and_FF = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins xor_00 = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins xor_FF = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_00 = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins mul_FF = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_max = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) && binsof (b_leg.ones));
ignore_bins others_only =
binsof(a_leg.others) && binsof(b_leg.others);
}
endgroup
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
end
op_cov oc;
zeros_or_ones_on_ops c_00_FF;
initial begin : coverage
oc = new();
c_00_FF = new();
forever begin @(posedge clk);
oc.sample();
c_00_FF.sample();
end
end : coverage
function operation_t get_op();
bit [2:0] op_choice;
op_choice = $random;
case (op_choice)
3'b000 : return no_op;
3'b001 : return add_op;
3'b010 : return sub_op;
3'b011 : return and_op;
3'b100 : return xor_op;
3'b101 : return mul_op;
3'b110 : return rst_op;
3'b111 : return rst_op;
endcase // case (op_choice)
endfunction : get_op
function byte get_data();
// This is how the random gets biased
bit [1:0] zero_ones;
zero_ones = $random;
if (zero_ones == 2'b00)
return 8'h00;
else if (zero_ones == 2'b11)
return 8'hFF;
else
return $random;
endfunction : get_data
// Sent by the tester, compared by the scoreboard: the final block below
// demands that the two match.
int enviadas, chequeadas;
// Scoreboard loop: checks that everything came out right
always @(posedge done) begin : scoreboard
shortint predicted_result;
bit predicted_ovf;
#1;
case (op_set)
add_op: predicted_result = A + B;
sub_op: predicted_result = A - B;
and_op: predicted_result = A & B;
xor_op: predicted_result = A ^ B;
mul_op: predicted_result = A * B;
endcase // case (op_set)
// ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
// output, neither the addition nor the multiplication can overflow.
predicted_ovf = (op_set == sub_op) && (A < B);
if ((op_set != no_op) && (op_set != rst_op)) begin
chequeadas++;
if (predicted_result != result || predicted_ovf != ovf)
$error ("FAILED: A: %0h B: %0h op: %s result: %0h ovf: %0b",
A, B, op_set.name(), result, ovf);
end
end : scoreboard
// Randomize the stimulus
// get_op and get_data are Constrained Random Data
// The point of CRD is to build values that are random but legal
initial begin : tester
reset_n = 1'b0;
@(negedge clk);
@(negedge clk);
reset_n = 1'b1;
start = 1'b0;
repeat (1000) begin
@(negedge clk);
op_set = get_op();
A = get_data();
B = get_data();
start = 1'b1;
case (op_set) // handle the start signal
no_op: begin
@(posedge clk);
start = 1'b0;
end
rst_op: begin
reset_n = 1'b0;
start = 1'b0;
@(negedge clk);
reset_n = 1'b1;
end
default: begin
// Wait for an EDGE with done up, not for done itself: on the
// one-cycle ops done is a level still up from the previous op,
// and a plain wait(done) drops this one without a sound.
enviadas++;
do @(negedge clk); while (done == 0);
start = 1'b0;
end
endcase // case (op_set)
end
$finish;
end : tester
// The testbench checks itself: every operation sent has to come back
// through the scoreboard. A tester that drops operations -- while the
// coverage counts them anyway -- passes green and has measured nothing.
final begin
if (enviadas != chequeadas)
$error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
else
$display ("%0d operations sent, %0d checked", enviadas, chequeadas);
end
endmodule : topall_opsignoresrst_opandno_op: it makes no sense to ask for "an addition with the operands at 0x00" when the operation is a reseta_legandb_legbring 256 values down to the three buckets that matter: all zeros, the middle, all ones- An impossible bin you leave in place keeps your coverage pinned below 100 %
and nobody knows why.
ignore_binsturns it into a written decision
There is a relative worth naming: illegal_bins. ignore_bins takes the bin
out of the count; illegal_bins takes it out of the count and also reports an error if
it ever happens. It is for the cases the spec forbids — a reserved opcode,
an illegal handshake.
The rule of thumb: if it cannot be covered, ignore_bins; if it must not happen,
illegal_bins; if it simply has not happened yet, leave it and write the test.
Functional coverage
cross: the combinations, and how not to drown
op_00_FF : cross a_leg, b_leg, all_ops; // 3 x 3 x 5 = 45 bins, almost all meaningless
- A
crossis the cartesian product of its coverpoints: it grows fast and most of the combinations are in nobody's spec binsof(x) intersect {v}reads as "the bins ofxthat containv"- They combine with
&&and||, and what is left over gets thrown away withignore_bins - That way the 45 bins come down to the 11 the plan actually asked for
The arithmetic in the comment is what has to be done out loud, because the number
is scary and it has to be: three bins of A by three of B by five operations
is 45, and out of those 45 the plan asked for eleven. The other 34 are not wrong — they are
surplus, and a surplus bin is as expensive as a missing one: it pins your coverage
below 100 % and nobody knows whether it is a real hole.
binsof(x) intersect {v} is worth reading in plain words before
writing it, because the syntax does not help: "the bins of x that contain v".
Then they combine with && and || like any condition.
The scale mistake to anticipate: on a real DUT a cross of three
coverpoints with automatic bins is thousands of buckets, and a regression never
fills them. When somebody says "the coverage is not going up", half the time
the problem is an unfiltered cross, not a missing test.
And the number to keep at hand for the "Reading the number" slide: the 45 are the
LRM count, one bin per enum member. Verilator measures 54, because it hands out
the automatic bins by the base type (bit [2:0], 8 values minus the 2 of
ignore_bins, that is 3 x 3 x 6). The 9 extra are the 3'b110 crossed, and they
are exactly the 9 that stay at zero.
Functional coverage
The VTALU cross
code/u2/convencional/vtalu_tb.sv · #cross-mul-bins bins mul_00 = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins mul_FF = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_max = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) && binsof (b_leg.ones));
ignore_bins others_only =
binsof(a_leg.others) && binsof(b_leg.others);el archivo entero, 256 líneas
// Typical TB:
// ----------
// * Functional Coverage
// * Self Checking
// * Constrained Random Stimulus
// Why is this bad?
// * No modularity (everything in a single file)
// * Almost nothing here can be reused
module top;
typedef enum bit[2:0] {no_op = 3'b000,
add_op = 3'b001,
sub_op = 3'b010,
and_op = 3'b011,
xor_op = 3'b100,
mul_op = 3'b101,
rst_op = 3'b111} operation_t;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);
`ifdef VLT_TRACE
// Waves for GTKWave: VLT_TRACE=1 bash run.sh && gtkwave vtalu.vcd
// Behind an `ifdef because $dumpvars does not compile without --trace, and
// the define comes from the same VLT_TRACE that adds the flag (common.sh).
initial begin
$dumpfile("vtalu.vcd");
$dumpvars;
end
`endif
//op_cov: make sure every operation, and every interaction between them, is covered
covergroup op_cov;
coverpoint op_set {
bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
bins multi_cycle = {mul_op};
// => are transitional coverage. From => To
// Transition bins. Verilator 5.052 does NOT implement them yet: any form
// with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
// code/verilator/repro-cg-transition.sv. They stay because they are part
// of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
bins opn_rst[] = ([add_op:mul_op] => rst_op);
bins rst_opn[] = (rst_op => [add_op:mul_op]);
bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);
bins twoops[] = ([add_op:mul_op] [* 2]);
bins manymult = (mul_op [* 3:5]);
`endif
}
endgroup
covergroup zeros_or_ones_on_ops;
// Check the corners where the inputs are all 0s or all 1s
all_ops : coverpoint op_set {
ignore_bins null_ops = {rst_op, no_op};}
a_leg: coverpoint A {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
b_leg: coverpoint B {
bins zeros = {8'h00};
bins others= {[8'h01:8'hFE]};
bins ones = {8'hFF};
}
// VTALU rev2: the borrow of the subtraction, the row of the plan that the
// scoreboard can only close by looking at TWO outputs. Without this bin, a
// subtraction that never went negative looks exactly like one that did.
borrow: coverpoint ((op_set == sub_op) && (A < B)) {
bins hubo_borrow = {1};
ignore_bins resto = {0};
}
op_00_FF: cross a_leg, b_leg, all_ops {
bins add_00 = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins add_FF = binsof (all_ops) intersect {add_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins sub_00 = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins sub_FF = binsof (all_ops) intersect {sub_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins and_00 = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins and_FF = binsof (all_ops) intersect {and_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins xor_00 = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins xor_FF = binsof (all_ops) intersect {xor_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_00 = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.zeros) || binsof (b_leg.zeros));
bins mul_FF = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) || binsof (b_leg.ones));
bins mul_max = binsof (all_ops) intersect {mul_op} &&
(binsof (a_leg.ones) && binsof (b_leg.ones));
ignore_bins others_only =
binsof(a_leg.others) && binsof(b_leg.others);
}
endgroup
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
end
op_cov oc;
zeros_or_ones_on_ops c_00_FF;
initial begin : coverage
oc = new();
c_00_FF = new();
forever begin @(posedge clk);
oc.sample();
c_00_FF.sample();
end
end : coverage
function operation_t get_op();
bit [2:0] op_choice;
op_choice = $random;
case (op_choice)
3'b000 : return no_op;
3'b001 : return add_op;
3'b010 : return sub_op;
3'b011 : return and_op;
3'b100 : return xor_op;
3'b101 : return mul_op;
3'b110 : return rst_op;
3'b111 : return rst_op;
endcase // case (op_choice)
endfunction : get_op
function byte get_data();
// This is how the random gets biased
bit [1:0] zero_ones;
zero_ones = $random;
if (zero_ones == 2'b00)
return 8'h00;
else if (zero_ones == 2'b11)
return 8'hFF;
else
return $random;
endfunction : get_data
// Sent by the tester, compared by the scoreboard: the final block below
// demands that the two match.
int enviadas, chequeadas;
// Scoreboard loop: checks that everything came out right
always @(posedge done) begin : scoreboard
shortint predicted_result;
bit predicted_ovf;
#1;
case (op_set)
add_op: predicted_result = A + B;
sub_op: predicted_result = A - B;
and_op: predicted_result = A & B;
xor_op: predicted_result = A ^ B;
mul_op: predicted_result = A * B;
endcase // case (op_set)
// ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
// output, neither the addition nor the multiplication can overflow.
predicted_ovf = (op_set == sub_op) && (A < B);
if ((op_set != no_op) && (op_set != rst_op)) begin
chequeadas++;
if (predicted_result != result || predicted_ovf != ovf)
$error ("FAILED: A: %0h B: %0h op: %s result: %0h ovf: %0b",
A, B, op_set.name(), result, ovf);
end
end : scoreboard
// Randomize the stimulus
// get_op and get_data are Constrained Random Data
// The point of CRD is to build values that are random but legal
initial begin : tester
reset_n = 1'b0;
@(negedge clk);
@(negedge clk);
reset_n = 1'b1;
start = 1'b0;
repeat (1000) begin
@(negedge clk);
op_set = get_op();
A = get_data();
B = get_data();
start = 1'b1;
case (op_set) // handle the start signal
no_op: begin
@(posedge clk);
start = 1'b0;
end
rst_op: begin
reset_n = 1'b0;
start = 1'b0;
@(negedge clk);
reset_n = 1'b1;
end
default: begin
// Wait for an EDGE with done up, not for done itself: on the
// one-cycle ops done is a level still up from the previous op,
// and a plain wait(done) drops this one without a sound.
enviadas++;
do @(negedge clk); while (done == 0);
start = 1'b0;
end
endcase // case (op_set)
end
$finish;
end : tester
// The testbench checks itself: every operation sent has to come back
// through the scoreboard. A tester that drops operations -- while the
// coverage counts them anyway -- passes green and has measured nothing.
final begin
if (enviadas != chequeadas)
$error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
else
$display ("%0d operations sent, %0d checked", enviadas, chequeadas);
end
endmodule : topmul_00reads straight through: a multiplication where A or B is 0x00. The other eight bins of the cross are these two lines with anotheropmul_maxis the only one with&&: it asks for both legs at 0xFF: the maximum product,FF×FF=FE01. It does not overflow — 8 bits by 8 fit in 16- Verilator 5.052 ignores
binsof/intersect(%Warning-COVERIGN) and measures the whole cross: that is why the number is not a commercial tool's
Worth reading mul_00 and mul_max in plain words, one after the other, because the
difference between || and && is the one that gets copied wrong, and here both bins
belong to the same operation: "a multiplication where A or B is 0x00"
against "a multiplication with A and B at 0xFF". The first
is two cases, the second is a single one — the maximum product, the top corner of
the input space. And here it is worth killing the misunderstanding that comes on its own,
because it is expensive: FF × FF does not overflow anything. It gives FE01, which
fits exactly in the 16 bits of result —8 bits by 8 never go past 16— and that is why the
DUT forces ovf to 0 in every multiplication: assign ovf = es_mult ? 1'b0 : ovf_1c;.
The bin is called mul_max and not mul_ovf for precisely that reason: the case is worth
something for being the maximum of the input space, not for overflowing. The only operation
that overflows is the subtraction, and only when A < B.
The honesty of this slide is part of the course, so it is worth saying and not
rushing past: the 86.8 % the example reports is not the number Questa would give.
Verilator ignores the filter and measures the whole cross, and on top it adds a
value the enum does not have: where the LRM counts 45, Verilator counts 54 (the
9 extra are the 3'b110 crossed, and they are the 9 that stay at zero). That is why
the percentage comes out lower, and for a reason that is not the testbench's. It is
written, dated and with a version in docs/en/verilator.md.
The useful question for the classroom: does that invalidate the exercise? No. What gets learned
—writing the cross, filtering it, and knowing which bin belongs to which row of the plan—
is identical. The only thing that cannot be done with this tool is signing off on
coverage closure, and that is not the goal of the course.
Functional coverage
Transition bins: when the order matters
bins opn_rst[] = ([add_op:mul_op] => rst_op); // A and then B
bins sngl_mul[] = ([add_op:xor_op], no_op => mul_op);
bins twoops[] = ([add_op:mul_op] [* 2]); // twice in a row
bins manymult = (mul_op [* 3:5]); // between 3 and 5 times in a row
- Three rows of the plan are not values, they are sequences: "after a reset", "a mult after a single-cycle one"
- Verilator 5.052 still does not compile them (Internal Error). In the code
they are inside
`ifndef VERILATOR: they are part of the topic and read just the same - Minimal repro in
code/verilator/repro-cg-transition.sv. The day Verilator supports them, theifndefgets deleted and nothing else
It is the only place in the course where the free tool falls short, and it is worth
saying head on instead of hiding it: the concept belongs to the language, not to the
simulator, and on Questa or VCS these bins run.
There are two more cousins worth naming: [-> n] (goto, n times not necessarily
in a row) and [= n] (non-consecutive). They are in coverage.svh of the analysis ports.
Functional coverage
The knobs: when a bin counts as covered
coverpoint op_set {
option.at_least = 10; // a bin only counts at 10 hits
bins single[] = {[add_op:xor_op]};
}
coverpoint A { option.auto_bin_max = 8; } // 8 automatic buckets, not 64
option.at_leastis the one that matters most: by default it is 1, so a bin that happened once already shows up as covered. The corner case that came up once in a thousand runs is not verified, and the 100 % says it isoption.auto_bin_maxbounds the automatic bins. Without{ }and without this knob, an 8-bit coverpoint does not give 256 buckets: it gives 64, handed out by the tool and not by the specoption.weightchanges how much a coverpoint weighs in the total, andtype_option.merge_instancesadds every instance into a single number instead of reporting them separately- Verilator 5.052 honours
auto_bin_max, andat_leastonly written in the coverpoint: in the covergroup it ignores it without warning. It is indocs/en/verilator.md
This is the slide that puts an asterisk on every percentage of the unit, and
that is why it only arrives now: at_least = 1 means the tool tells you
covered with a single hit. For a bin that represents a value of an enum
that is perfect —either it happened or it did not—. For the bin that represents the maximum
product of the multiplier, one hit is an anecdote, not a verification.
The field rule, and it is worth giving because the question comes on its own: at_least
high on the bins that represent a rare case, default on the ones that represent a
value. Raising it for the whole covergroup is not rigour, it is a regression that never
closes and a number nobody looks at.
auto_bin_max is the flip side of the bracket trap, the one on the bins slide. There the problem was a surplus bin; here it is the other way round: if the coverpoint is
an int and you write it no bins, the tool invents 64 ranges that do not
correspond to any row of the plan. The number that comes out is real and means
nothing.
And the usual honesty: the four knobs are the LRM's, not Verilator's. What
this tool does with each one is measured, dated and with a repro in
code/verilator/repro-cg-options.sv. The at_least one is the worst of the four
because it is silent: in the covergroup you write it, it compiles, and it does nothing.
Functional coverage
Reading the number
$ make u2/convencional
Coverage Summary:
covergroup : 86.8% (66/76)
- 66 of 76 bins filled, with 1000 randomizations and 615 operations on the
bus —
no_opandrst_opget drawn and never sent, and the testbench prints that right before the Verilator report—. The other rows come out0/0:--coverage-userleaves code coverage out - The 10 that are missing are a single value:
all_ops.auto_5and its nine crosses. It is3'b110, which the enum does not have — Verilator hands out the automatic bins by the base type,bit [2:0], and not by enum member - Which means no row of the plan was left uncovered, and on Questa this gives 100 %.
The number is not the goal: the question that helps is which bin is missing,
and only the per-bin report says that,
obj_dir/top/coverage.dat - Run → look at what is missing → write the directed test → run again. That is called coverage closure, and it is what a verification engineer spends the day on
This is where it is worth running it live and opening the coverage.dat: the
student has to see that the ten zeros have the same name, and that the name is not
in the operation_t. The lesson is to read the report and not the number: the
number says 86.8 % and the report says "nothing is missing, the tool invented a
bucket". On a commercial tool the automatic bin of an enum is one per member
(IEEE 1800, 19.5) and this gives 100 % from the start; it is noted in
docs/en/verilator.md.
And the punchline: the exercise of the day uses precisely 3'b110 for the shift, so
the new operation fills the phantom bucket and the coverage goes from 86.8 % to
100 % — 77 out of 77, with the shift's own bin added. The one who raises it is the
same one who wrote the test, and the checker counts bins and not the percentage,
because we have just seen the percentage cannot read.
Functional coverage
The verification plan, whole and in one table
| # | Feature | Scenario | Stimulus | Check | Measure |
|---|---|---|---|---|---|
| 1 | ALU | the six operations | random | scoreboard | coverpoint op_set |
| 2 | ALU | operands at 00 and at FF |
dist biased to the edges |
scoreboard | cross op_00_FF |
| 3 | mult | maximum product: FF × FF |
directed case | scoreboard, 16 bits | bin mul_max |
| 4 | reset | operate after a reset | rst_op interleaved |
scoreboard | bin rst_op => op |
| 5 | mult | a mult after a single-cycle one | random | scoreboard | transition bin |
| 6 | ALU | the same operation twice in a row | random | scoreboard | bin [* 2] |
| 7 | sub | subtract too much: A < B, and ovf goes up |
random | scoreboard, two outputs | bin hubo_borrow |
| 8 | sub | A == B: the result is 0 and ovf does not go up |
random | scoreboard, two outputs | bin sub_00/sub_FF |
| 9 | ovf | ovf does not go up for any other operation |
random | assertion | c_ovf |
| 10 | protocol | the operands are not touched with start up |
random | assertion | cover property |
| 11 | protocol | done arrives, and before 5 cycles |
random | assertion | cover property |
| 12 | protocol | no_op is the only one that does not answer |
random | assertion | cover property |
- The three right-hand columns are the three parts of the testbench of this section. What ties them together is the plan, and until now you had not seen it whole
- The two subtraction rows are the ones that force the scoreboard to look at two
outputs: if it only compares
result, it passes green withovfstuck at zero - The last four are checked by no scoreboard: they are
ovfand protocol rules, and get checked where they happen —the assertions - The twelve rows are numbered the same as in
docs/en/verification-plan.md, which also says which file each one lives in and carries the capstone template
This is the slide that answers "and what is this for?" for the rest of day 1, and
it is worth saying head on why it only arrives now: the three right-hand columns
are stimulus, self-checking and coverage — the three parts the previous
unit showed loose. The plan is the table that makes them one thing.
The row that gets the most discussion is the maximum-product one, and that is good: FF × FF
is the case a short test does not fill, and that is why its stimulus column says
directed case: the plan is decided before knowing how long the random will run.
There you see that the plan does not only measure, it also decides which test has to be written.
It is the d5c exercise, the last one of the day 5 afternoon, with this same row.
The three protocol rows are worth naming and moving on: they are the half the
course does not touch until day 7, and they serve to plant the idea that a scoreboard does not
check everything. A serious verification plan has both columns.
And what the student takes home: docs/en/verification-plan.md is the same table
as an artifact, with an empty template. The day 7 capstone is handed in with the
plan filled — which is exactly what gets handed in on a project.
Functional coverage
Summary of the unit
- Code coverage ≠ functional coverage. The first one measures which RTL got executed; the second, which scenarios of the spec happened
- A
covergroupis a type: it has to be instantiated withnew()and somebody has to callsample(). Without that the report reads 0 % without a warning - The bins are the buckets.
[]hands out one per value; without brackets, the whole range goes into a single one ignore_binsis executable documentation: it says out loud what does not get covered and why. An impossible bin you leave in place pins your number down- A
crossis the cartesian product and grows extremely fast: without bounding it, the coverage never closes option.at_leastis 1 by default: a bin that happened once already shows up as covered, and a rare case with one hit is not verified- The number is not the goal. The goal is the plan; the coverage only says how much of the plan was met
Closing of the most misunderstood concept of the course. The sentence to take home:
coverage does not tell you the design is right, it tells you that you tested it.
A DUT that is missing a whole feature gives 100 % of code coverage and 0 %
of the functional coverage nobody wrote.
And the operational mistake that repeats the most, to leave it burned in: the missing
sample(). It compiles, it runs, and the report reads zero.
Interfaces and BFM
First: the signals stop being loose
code/u2/interfaces-bfm/vtalu_bfm.sv · #signals-and-clockinterface vtalu_bfm;
// brings in operation_t. e.g. (no_op = 3'b000)
import vtalu_pkg::*;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [ 2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
// The BFM is a first step towards modularity
// It takes care of driving every low-level signal
// and keeps the protocol in one single place
// DUT clock
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
endel archivo entero, 71 líneas
interface vtalu_bfm;
// brings in operation_t. e.g. (no_op = 3'b000)
import vtalu_pkg::*;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [ 2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
// The BFM is a first step towards modularity
// It takes care of driving every low-level signal
// and keeps the protocol in one single place
// DUT clock
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
end
// DUT reset
task reset_alu();
reset_n = 1'b0;
@(negedge clk);
@(negedge clk);
reset_n = 1'b1;
start = 1'b0;
endtask : reset_alu
// The protocol handling can now be changed in one single place.
// Fix it here once, and the fix propagates through the rest of the code
task send_op(input byte iA, input byte iB, input operation_t iop,
output shortint alu_result);
op_set = iop;
if (iop == rst_op) begin
@(posedge clk);
reset_n = 1'b0;
start = 1'b0;
@(posedge clk);
#1;
reset_n = 1'b1;
end else begin
@(negedge clk);
A = iA;
B = iB;
start = 1'b1;
if (iop == no_op) begin
@(posedge clk);
#1;
start = 1'b0;
end else begin
do @(negedge clk); while (done == 0);
start = 1'b0;
end
end
endtask : send_op
endinterface : vtalu_bfm- An
interfaceis a bundle of signals with a name of its own. Nine wires that used to be declared in thetopnow live together - The clock gets generated inside: the interface is not a wire, it is a model of the bus
- Connecting the DUT becomes
.A(bfm.A),.clk(bfm.clk)… and a new signal gets declared once: the modules that use it see it appear without touching their ports - What is not needed here and you will see in somebody else's RTL: the
modport, which is the same interface with the directions set for one side of the wire
First step towards UVM and it does not have a single line of UVM.
The immediate gain is a maintenance one and it is worth measuring: with loose
signals, splitting the testbench into modules forces every one of them through every
port list, and a new signal gets wired in all of them. Here it gets declared in the
interface, and the only port that changes is the DUT's.
One detail of types that costs you: op is a wire [2:0] and op_set is the
operation_t. The assign op = op_set is the bridge between the testbench enum and
the DUT wires. It is the only place in the course where the seam between the
two worlds is visible.
And the modport, which does not show up in the rest of the course and does in anybody
else's RTL: it is a view of the interface with the directions declared —modport dut (input A, input B, output result)—, and the module asks for it in its port list
(vtalu_bfm.dut bus). It is good for two things, and both are worth it: it lets the compiler
stop the DUT if it writes a signal that was an input as far as it is concerned, and it lets whoever reads the
interface know who drives what without opening the RTL. It is not here because the BFM belongs
to the testbench and drives everything; at work the interface almost always comes with
two or three modports and what you need to know is which one to ask for.
Interfaces and BFM
Then: the protocol hides inside a task
code/u2/interfaces-bfm/vtalu_bfm.sv · #send_op // The protocol handling can now be changed in one single place.
// Fix it here once, and the fix propagates through the rest of the code
task send_op(input byte iA, input byte iB, input operation_t iop,
output shortint alu_result);
op_set = iop;
if (iop == rst_op) begin
@(posedge clk);
reset_n = 1'b0;
start = 1'b0;
@(posedge clk);
#1;
reset_n = 1'b1;
end else begin
@(negedge clk);
A = iA;
B = iB;
start = 1'b1;
if (iop == no_op) begin
@(posedge clk);
#1;
start = 1'b0;
end else begin
do @(negedge clk); while (done == 0);
start = 1'b0;
end
end
endtask : send_opel archivo entero, 71 líneas
interface vtalu_bfm;
// brings in operation_t. e.g. (no_op = 3'b000)
import vtalu_pkg::*;
byte unsigned A;
byte unsigned B;
bit clk;
bit reset_n;
wire [ 2:0] op;
bit start;
wire done;
wire [15:0] result;
wire ovf;
operation_t op_set;
assign op = op_set;
// The BFM is a first step towards modularity
// It takes care of driving every low-level signal
// and keeps the protocol in one single place
// DUT clock
initial begin
clk = 0;
forever begin
#10;
clk = ~clk;
end
end
// DUT reset
task reset_alu();
reset_n = 1'b0;
@(negedge clk);
@(negedge clk);
reset_n = 1'b1;
start = 1'b0;
endtask : reset_alu
// The protocol handling can now be changed in one single place.
// Fix it here once, and the fix propagates through the rest of the code
task send_op(input byte iA, input byte iB, input operation_t iop,
output shortint alu_result);
op_set = iop;
if (iop == rst_op) begin
@(posedge clk);
reset_n = 1'b0;
start = 1'b0;
@(posedge clk);
#1;
reset_n = 1'b1;
end else begin
@(negedge clk);
A = iA;
B = iB;
start = 1'b1;
if (iop == no_op) begin
@(posedge clk);
#1;
start = 1'b0;
end else begin
do @(negedge clk); while (done == 0);
start = 1'b0;
end
end
endtask : send_op
endinterface : vtalu_bfmsend_op(A, B, op, result)translates "do an addition" into the signal wiggling the DUT expects. That is a Bus Functional Model- Inside it is all the fine print of the spec: the
reset_npulse, theno_opthat does not wait fordone, and thedo ... while (done == 0)of the rest - From here on nobody moves
bfm.startby hand. Whoever wants an operation asks for it
The sentence that orders the section: the protocol gets written once. If tomorrow
the DUT asks for two cycles of start, this task gets touched and nothing else.
The #1 after the @(posedge clk) is the same precaution as the scoreboard of the conventional testbench: moving the signals one delta after the edge, not on the edge.
And the hook for day 4: the driver of the put and get ports is exactly this task,
but inside a class and fed by a port. The operation —hiding the
wire— is the same; what changes is where the data comes from.
Interfaces and BFM
The top, with the BFM in the middle
- The testbench is still split into the same three pieces —tester, scoreboard and coverage—, and not one of them changed its name
- What changed is how they connect to the DUT: there are no loose signals travelling any more, there is a single interface the three of them share
code/u2/interfaces-bfm/top.svmodule top;
vtalu_bfm bfm ();
// Tester: generates the stimulus
tester tester_i (bfm);
// Coverage: provides functional coverage
coverage coverage_i (bfm);
// ScoreBoard: checks the results
scoreboard scoreboard_i (bfm);
vtalu DUT (
.A(bfm.A),
.B(bfm.B),
.op(bfm.op),
.clk(bfm.clk),
.reset_n(bfm.reset_n),
.start(bfm.start),
.done(bfm.done),
.ovf(bfm.ovf),
.result(bfm.result)
);
endmodule : topIt is worth reading this top.sv next to the previous section's, because the diff is
the whole section: where there was a list of logic declared in the top and
wired by hand to the DUT, now there is one line that instantiates the BFM and one that
plugs it in.
What has to be pointed out, because it is what repeats for the rest of the course: the DUT
connects to bfm.A, bfm.B, bfm.op… The interface is not an abstract object,
it is the same old wires grouped under a name. Nothing became slower
or stranger — it became harder to connect wrong.
And the detail that pays for itself the day the DUT grows: adding a signal to the
protocol means touching the interface, and everyone who uses it sees it appear. In the
previous version you had to remember to wire it in every instance.
Interfaces and BFM
Scoreboard: the same check, now talking to the BFM
code/u2/interfaces-bfm/scoreboard.svmodule scoreboard (
vtalu_bfm bfm
);
import vtalu_pkg::*;
always @(posedge bfm.done) begin
shortint predicted_result;
bit predicted_ovf;
#1;
case (bfm.op_set)
add_op: predicted_result = bfm.A + bfm.B;
sub_op: predicted_result = bfm.A - bfm.B;
and_op: predicted_result = bfm.A & bfm.B;
xor_op: predicted_result = bfm.A ^ bfm.B;
mul_op: predicted_result = bfm.A * bfm.B;
endcase // case (op_set)
// ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
// output, neither the addition nor the multiplication can overflow.
predicted_ovf = (bfm.op_set == sub_op) && (bfm.A < bfm.B);
if ((bfm.op_set != no_op) && (bfm.op_set != rst_op))
if (predicted_result != bfm.result || predicted_ovf != bfm.ovf)
$error(
"FAILED: A: %0h B: %0h op: %s result: %0h ovf: %0b",
bfm.A,
bfm.B,
bfm.op_set.name(),
bfm.result,
bfm.ovf
);
end
endmodule : scoreboard- It is the conventional testbench's
scoreboard: it predicts and compares with the same logic - The only thing that changed is where it gets the signals from: they used to be variables of the
same module, now they are
bfm.A,bfm.B,bfm.op_set - And that is why it is a separate module now: it can live in its own file because it shares nothing with the tester except the interface
What has to be pointed out is what did not change. The verification logic is
identical; what got modularized is the wiring. That is the whole section.
The bfm. in front of every signal is the first time the student sees an interface
hierarchy, and it is worth writing it on the board: top.bfm.A. In the tests that
same reference is going to arrive through the config_db into a class, and there the prefix will
be a handle instead of an instance name.
The debt that stays open: the scoreboard still watches the signals
directly. Only in the analysis ports is it going to receive transactions from a monitor and
stop knowing that a clk exists.
Interfaces and BFM
Tester: it asks for operations, it does not move wires
code/u2/interfaces-bfm/tester.sv · #stimulus-loop initial begin
byte unsigned iA;
byte unsigned iB;
operation_t op_set;
shortint result;
bfm.reset_alu();
repeat (1000) begin : random_loop
op_set = get_op();
iA = get_data();
iB = get_data();
bfm.send_op(iA, iB, op_set, result);
end : random_loop
$finish;
end // initial beginel archivo entero, 46 líneas
// The Tester module is simpler now, because it no
// longer has to deal with low-level signals
module tester (
vtalu_bfm bfm
);
import vtalu_pkg::*;
function operation_t get_op();
bit [2:0] op_choice;
op_choice = $random;
case (op_choice)
3'b000: return no_op;
3'b001: return add_op;
3'b010: return sub_op;
3'b011: return and_op;
3'b100: return xor_op;
3'b101: return mul_op;
3'b110: return rst_op;
3'b111: return rst_op;
endcase // case (op_choice)
endfunction : get_op
function byte get_data();
bit [1:0] zero_ones;
zero_ones = $random;
if (zero_ones == 2'b00) return 8'h00;
else if (zero_ones == 2'b11) return 8'hFF;
else return $random;
endfunction : get_data
initial begin
byte unsigned iA;
byte unsigned iB;
operation_t op_set;
shortint result;
bfm.reset_alu();
repeat (1000) begin : random_loop
op_set = get_op();
iA = get_data();
iB = get_data();
bfm.send_op(iA, iB, op_set, result);
end : random_loop
$finish;
end // initial begin
endmodule : tester- The tester no longer touches
startor waits fordone: it callsbfm.send_op(...)and forgets about the protocol - The handshake ended up written once, inside the BFM. If tomorrow the
DUT asks for two cycles of
start, one file gets touched - That is the definition of Bus Functional Model: the task translates "do an addition" into the signal wiggling the DUT expects
The comparison with the conventional testbench is the slide: over there the tester raised start,
waited for done and lowered start, mixed in with the stimulus generation. Here
it only generates stimulus.
Worth counting how many places of the conventional testbench knew about the protocol
—the tester and the scoreboard— and how many move start now: one, the send_op.
The scoreboard still waits for done on its own; that debt gets paid by the monitor
of the analysis ports.
And the hook for day 4: in the put and get ports this same tester is going to be a class with
a put_port, and the BFM is going to end up on the other side of a driver. The operation is
the same one we did here —hiding the wire— repeated one level up.
Interfaces and BFM
The protocol rule, written in the simulator's language
// IMMEDIATE: a statement. It checks the present, where the thread goes by.
assert (op_set != no_op || done == 0)
else $error("no_op should not raise done");
// CONCURRENT: a declaration with a clock. It plugs itself in once and
// checks EVERY edge of the simulation, on its own, forever.
a_operandos_estables :
assert property (@(negedge clk) start |=> $stable(A) && $stable(B));
- The rule —while
startis up, the operands are not touched— is already written twice: in prose in the spec, and wrapped insidesend_op()here - An assertion is that same sentence a third time, but executable: it does not merely comply with the rule, it checks it
- And it checks it where it happens —in the interface, on the exact edge— and not at the end, comparing results
This is a hook, not the section: the syntax must not be explained here, and least of all
the |=> or the edge. The only thing that has to stick is that the protocol rule
can be written in a form the simulator understands.
Worth pointing out the asymmetry that opens up: the BFM complies with the protocol, and that
is enough while all the stimulus goes through the BFM. As soon as a legacy module,
a VIP from another team or a driver written in a hurry shows up, complying stops
being the same as checking.
If somebody asks why one says negedge and the other does not, the short answer
is "because the BFM writes on the negedge, and that is only fully visible in the assertions". That is the question we want to plant.
Interfaces and BFM
Where each thing gets checked
- The conventional testbench's scoreboard checks the result:
A op Bagainst what the DUT returned. It is the half this course develops to the end - An assertion checks the protocol: who moves, when, and for how many cycles. It is the other half
- A DUT that returns
resultcorrectly but dropsdoneone cycle early passes every test of the next six days. Nobody is watching it - It is seen whole in the assertions section, with the same VTALU and the same
run.sh
The sentence worth leaving on the board for the whole course: the scoreboard checks what the DUT computes; the assertions, how it is talked to. What is left of day 1 and the six days that follow are the first half; the second arrives at the end, and it arrives on this same example. There is no need to promise more than that. The assertions come back with the property of this slide, write it for real, and show why the edge chosen is not a detail.
Interfaces and BFM
The hole the BFM leaves: the timing
code/u2/clocking/sin_clocking.sv · #three-samples // 1. Sample AT the edge. The DUT always_ff updates d_out with a
// nonblocking assignment, which lands after this initial already ran:
// you read the OLD value.
@(posedge clk);
en_el_flanco = d_out;
// 2. The same edge, one delta later. The nonblocking assignment landed
// already and you read the NEW value. That "#1" is visible nowhere and
// changes the result.
@(posedge clk);
#1;
un_delta_despues = d_out;
// 3. The opposite edge: the trick the whole course BFM uses. It works,
// but it asks whoever reads it to know why.
@(negedge clk);
en_el_flanco_opuesto = d_out;el archivo entero, 50 líneas
// Sampling by hand: without a clocking block you have to pick the edge AND the
// delta, in every task, over and over. This top runs the three variants people
// write in their first week: one reads the OLD value, the other two the NEW.
//
// bash run.sh
module dut_reg (input bit clk, input byte unsigned d_in, output byte unsigned d_out);
// The DUT counts: on every edge, the output becomes the input + 1.
always_ff @(posedge clk) d_out <= d_in + 8'd1;
endmodule
module top_sin;
bit clk = 0;
byte unsigned d_in, d_out;
always #5 clk = ~clk;
dut_reg u_dut (.*);
byte unsigned en_el_flanco, un_delta_despues, en_el_flanco_opuesto;
initial begin
d_in = 8'd10;
// 1. Sample AT the edge. The DUT always_ff updates d_out with a
// nonblocking assignment, which lands after this initial already ran:
// you read the OLD value.
@(posedge clk);
en_el_flanco = d_out;
// 2. The same edge, one delta later. The nonblocking assignment landed
// already and you read the NEW value. That "#1" is visible nowhere and
// changes the result.
@(posedge clk);
#1;
un_delta_despues = d_out;
// 3. The opposite edge: the trick the whole course BFM uses. It works,
// but it asks whoever reads it to know why.
@(negedge clk);
en_el_flanco_opuesto = d_out;
$display("d_in = %0d, and the DUT computes d_in + 1 = %0d", 8'd10, 8'd11);
$display(" sampled AT the posedge : %0d", en_el_flanco);
$display(" sampled one delta later : %0d", un_delta_despues);
$display(" sampled at the negedge : %0d", en_el_flanco_opuesto);
$display("");
$display("Three lines that look the same and are not. And that decision");
$display("is repeated in every task of the interface.");
$finish;
end
endmodule : top_sin- The BFM wrapped the protocol. What it did not wrap is when it drives and when it samples: that is still decided task by task
- Three ways of reading the same data:
0,11and11. Two values, but three mechanisms, and the middle one depends on a#1that is visible nowhere - The
@(negedge clk)of our BFM is the third one. It works — and it asks whoever reads it to know why
Run this live, it takes two seconds: make u2/clocking. The three lines
it prints are the whole slide.
The first one is the surprising one and it is worth explaining slowly: sampling on
the posedge gives the old value, because the DUT's always_ff updates with
a nonblocking assignment and that one lands after this initial already
ran. It is not a Verilator bug and not an oddity: it is the LRM region order,
and it is the same on Questa.
The second one is the worst of the three, and it is the one to mark as a silent trap:
it works, and it works by accident. The #1 does not say which problem it solves,
it is not documented, and the day somebody deletes it because "it did nothing" the
testbench keeps compiling and starts lying. It is the same #1 as the scoreboard of
the conventional testbench, and the difference is a single one: there the slide says
which race it avoids. An explained #1 is a patch that knows it is a patch; a loose
#1 is the trap.
And the third is ours. Worth being honest with the group: the course drives on
negedge and samples on posedge because it is the trick you can understand without having
seen clocking blocks, not because it is what gets written on a project.
Interfaces and BFM
Why the edge alone is not enough: the regions
- An edge is not an indivisible instant: inside it there are regions, in a fixed order
- The
<=of thealways_fflands in NBA: whoever reads in Active reads the old value - The
#1crosses the NBA, the opposite edge crosses it with room to spare, and SVA needs neither of the two: it samples in preponed, before everything
This figure is the one to leave on screen while reading the three samples of the
previous slide. Without it the explanation is "the old value", which sounds like
a quirk of the simulator; with it, it is a consequence of the order, and the order
is in the LRM.
The scheduler has more regions than the four in the drawing —docs/clocking-blocks.md
(in Spanish) lists them all— but these are enough to explain the three cases, and adding the
others changes no answer.
The question worth asking the group before showing the answer: if the DUT writes in
NBA and your initial wakes up in Active, which one runs first? That is where it
shows on its own why sampling on the edge gives the value from before.
And the hook with day 7: the amber bar on the left is the region where SVA samples.
It is the same figure that explains why an assertion does not check what happened
but what it saw, and why the @() you give it changes the result.
Interfaces and BFM
The clocking block: the edge and the delta, declared once
code/u2/clocking/con_clocking.sv · #reg_bfminterface reg_bfm (input bit clk);
byte unsigned d_in, d_out;
// The only two timing lines in the testbench:
// input #1step samples the STABLE value right before the edge, which is
// what the hardware sees. Never what the nonblocking
// assignment just wrote on that same edge.
// output #0 drives ON the edge, in the nonblocking region, so the DUT
// does not see it until the next edge.
clocking cb @(posedge clk);
default input #1step output #0;
output d_in;
input d_out;
endclocking
modport tb (clocking cb);
endinterface : reg_bfmel archivo entero, 73 líneas
// The same DUT, with a clocking block: the edge and the delta are declared ONCE,
// inside the interface, and no task ever picks them again.
//
// bash run.sh
module dut_reg (input bit clk, input byte unsigned d_in, output byte unsigned d_out);
always_ff @(posedge clk) d_out <= d_in + 8'd1;
endmodule
interface reg_bfm (input bit clk);
byte unsigned d_in, d_out;
// The only two timing lines in the testbench:
// input #1step samples the STABLE value right before the edge, which is
// what the hardware sees. Never what the nonblocking
// assignment just wrote on that same edge.
// output #0 drives ON the edge, in the nonblocking region, so the DUT
// does not see it until the next edge.
clocking cb @(posedge clk);
default input #1step output #0;
output d_in;
input d_out;
endclocking
modport tb (clocking cb);
endinterface : reg_bfm
module top_con;
bit clk = 0;
always #5 clk = ~clk;
reg_bfm bfm (clk);
dut_reg u_dut (.clk(clk), .d_in(bfm.d_in), .d_out(bfm.d_out));
byte unsigned primera, segunda, tercera;
initial begin
// Driving is "<=" against the clocking block. There is no edge to pick.
bfm.cb.d_in <= 8'd10;
// @(bfm.cb) is "wait for this clocking block's edge". There are THREE, and
// the reason is the whole lesson:
// 1st edge the output #0 only now puts d_in at 10, in the nonblocking
// region, so the DUT has not seen it yet.
// 2nd edge now the DUT takes d_in=10 and computes d_out <= 11.
// 3rd edge the input #1step samples the STABLE value before this edge,
// which is already 11.
// A testbench without a clocking block hides this behind a #1 and works by
// accident. Here the arithmetic is visible and does not depend on the tool.
@(bfm.cb);
@(bfm.cb);
@(bfm.cb);
// And sampling is reading cb.d_out. It is read three times, at three
// different points of the same cycle, and gives the same value all three.
primera = bfm.cb.d_out;
#2;
segunda = bfm.cb.d_out;
#3;
tercera = bfm.cb.d_out;
$display("d_in = %0d, and the DUT computes d_in + 1 = %0d", 8'd10, 8'd11);
$display(" cb.d_out, right after the edge : %0d", primera);
$display(" cb.d_out, two units later : %0d", segunda);
$display(" cb.d_out, five later : %0d", tercera);
$display("");
$display("The same number all three times, and not a single #1 in the testbench.");
$display("The edge and the delta were declared once, in the clocking block.");
if (primera != 8'd11 || segunda != 8'd11 || tercera != 8'd11)
$fatal(1, "the clocking block should give 11 all three times");
$finish;
end
endmodule : top_coninput #1step— sample the value stable just before the edge, which is the one the hardware sees. Never the one the nonblocking assignment just wroteoutput #0— drive on the edge, with the nonblocking assignments, so the DUT does not see it until the next edge- Driving becomes
cb.d_in <= value; sampling, readingcb.d_out; waiting,@(cb). No#1in any task - And the value read is the same at any point of the cycle: the race got closed by declaration, not by habit
This is the slide you have to be able to repeat in an interview, because
clocking block is one of the first things people get asked and the good answer
is not "it synchronizes": it is it declares the sampling and driving instant once,
in the interface, so that no task picks it again.
The detail that confuses and is worth previewing: in the example you need three
@(cb) to see the 11, and not two. One for the output #0 to put the data,
another for the DUT to take it, and the third because the input #1step samples what is
stable before the edge. The arithmetic is in plain sight in the code and does not
depend on the simulator — which is exactly the difference with the #1.
If somebody asks why the course does not use it from day 1: because to
understand which problem it solves you have to have had the problem. Only now,
with the BFM written and the three lines running, does the question make sense.
Verilator supports it since 5.x and the example ends in $fatal if the
sampling stops giving 11: it is the safety net that warns if a release changes the semantics.
Interfaces and BFM
Are they mandatory? No — and it is worth knowing why
| The question | The answer |
|---|---|
| Can a correct TB be written without clocking blocks? | Yes. With NBA in the driver and scheduler discipline, there is no race |
| What should a team use? | Use them on synchronous interfaces — it is what the lowRISC/OpenTitan style guide says |
- They are not UVM's: they are SystemVerilog's. UVM neither asks for them nor knows them
- Dave Rich —the reference at Verification Academy— has held since 2014 that they are optional: what avoids the race is understanding the scheduler, not the clocking block
- The two positions answer different questions: one is language correctness, the other is team policy. Both are right
This slide is the one that keeps the course from lying by simplifying, and it is worth
spelling out: "always use clocking blocks" is a style rule,
not a theorem. Whoever repeats it as if it were the latter is in for an
uncomfortable argument with the first senior they run into.
Dave Rich's sentence that orders the topic —and it is worth quoting verbatim— is from 2014:
"If you are already familiar with Verilog testbenches, then you probably don't
need them." He held it in 2022 and in 2024. It does not say they are wrong: it says they are not
a necessary condition.
And on the other side is lowRISC, which in OpenTitan's DV Coding Style Guide makes them
mandatory for synchronous signals. They do not contradict each other: Dave answers
"can I?" and lowRISC answers "what do we all do here?". It is exactly the
difference between well-used malloc/free and a style guide that mandates
smart pointers — an expert being able to manage memory by hand does not make
RAII useless.
Where they really move the needle, and this is what to retain: reusable agents and VIP
—whoever writes them is far from whoever uses them—, gate-level
with SDF, and protocols where the spec defines setup/hold. There they stop being style.
The long material, with the sources, is in docs/clocking-blocks.md (in Spanish).
Interfaces and BFM
The price: two names for the same wire
code/u2/clocking/mezcla.sv · #two-reads // The two reads, at the SAME instant, on the SAME wire.
por_cb = bfm.cb.d_out;
crudo = bfm.d_out;
$display("at the same instant, on the same wire:");
$display(" bfm.cb.d_out (what the clocking block sampled) : %0d", por_cb);
$display(" bfm.d_out (the live wire) : %0d", crudo);
$display("");
$display("They differ by one cycle, and no warning says so. A monitor that");
$display("reads through the clocking block and a scoreboard that reads the");
$display("raw wire do not fail every time: they fail when the data changes.");el archivo entero, 53 líneas
// The trap that comes with the clocking block: mixing the two domains.
//
// Once a signal belongs to a clocking block, there are TWO names for the same
// wire -- bfm.d_out and bfm.cb.d_out -- and they are NOT the same value. One is
// the live wire; the other is what the clocking block sampled #1step before the
// edge. On a signal that changes every cycle they differ by exactly one cycle.
//
// Nothing warns about this: it compiles, it runs, and a scoreboard that reads
// one of them while the monitor reads the other fails intermittently.
//
// bash run.sh
module contador (input bit clk, output byte unsigned d_out);
// Counts on every edge, so the live wire and the sampled one never agree.
always_ff @(posedge clk) d_out <= d_out + 8'd1;
endmodule
interface cnt_bfm (input bit clk);
byte unsigned d_out;
clocking cb @(posedge clk);
default input #1step;
input d_out;
endclocking
endinterface : cnt_bfm
module top_mezcla;
bit clk = 0;
always #5 clk = ~clk;
cnt_bfm bfm (clk);
contador u_cnt (.clk(clk), .d_out(bfm.d_out));
byte unsigned por_cb, crudo;
initial begin
repeat (4) @(bfm.cb);
// The two reads, at the SAME instant, on the SAME wire.
por_cb = bfm.cb.d_out;
crudo = bfm.d_out;
$display("at the same instant, on the same wire:");
$display(" bfm.cb.d_out (what the clocking block sampled) : %0d", por_cb);
$display(" bfm.d_out (the live wire) : %0d", crudo);
$display("");
$display("They differ by one cycle, and no warning says so. A monitor that");
$display("reads through the clocking block and a scoreboard that reads the");
$display("raw wire do not fail every time: they fail when the data changes.");
if (por_cb == crudo)
$fatal(1, "they were expected to differ: the example stopped showing the trap");
$finish;
end
endmodule : top_mezcla- With the signal inside a clocking block there are two ways of reading it:
bfm.d_outis the live wire,bfm.cb.d_outis what got sampled#1stepbefore the edge - On a signal that changes every cycle they differ by one cycle: the example
prints
3and4, at the same instant and on the same wire - Nobody warns you. It compiles, it runs, and the scoreboard fails only when the data changes
- The rule: if a signal went into the clocking block, the whole protocol reads it
through there — and you wait on
@(bfm.cb), not on@(posedge bfm.clk)
This is the flip side of the previous slide and the reason the course does not use
them in its BFM: used badly they are worse than not using them, because
the failure mode is intermittent and the waveform looks fine.
The real mistake, the one that shows up in the forums, has two shapes and it is worth
naming both. One is mixing the event: @(posedge vif.clk) and then reading
vif.cb.foo. The other is mixing the access: the monitor reads vif.cb.data and the
scoreboard —or an assertion, or a debug $display— reads vif.data. Both
end the same way: two temporal views of the same wire, and a mismatch that
shows up once every twenty runs.
Worth running the example and leaving the number in plain sight: 3 and 4. Nobody argues
with a 3 and a 4 on the same wire.
And the moral that hooks into the silent traps appendix, where this one
is written down: in verification the expensive error is not the one that breaks, it is the one that
lies. A half-done clocking block lies.
Interfaces and BFM
Summary of the unit
- An
interfaceis a bundle of named signals: adding a wire stops being touching six port lists - A BFM is the interface plus the protocol:
send_op(A, B, op, result)translates "do an addition" into the signal wiggling - All the fine print of the spec —the reset pulse, waiting for
done, theno_opthat does not answer— ends up written once - From here on nobody moves
bfm.startby hand. Whoever wants an operation asks for it - The tester, the scoreboard and the coverage become separate modules, each one in its file, connected by the BFM
- And this does not get abandoned: the agent's driver keeps calling this same task
- The rule the BFM complies with can also be checked, with an assertion that lives in this same interface. That is the assertions section
- And the
clocking blockcloses the last hole: the edge and the delta stop being decided task by task and get declared once
This is the most important section of day 1 and the one that looks least like it. Everything that comes after —classes, factory, agents, sequences— rearranges pieces above this line. The line itself never gets touched again. The short way to say it: from here on the testbench stops talking in wires and starts talking in operations. That is the first step towards UVM, and it is already taken, without having written a single class. If the group comes from RTL, the analogy closes on its own: the BFM is to the testbench what a device driver is to the operating system.
Review · Day 1
1 of 7 · The question of the day
A thousand random operations, the scoreboard reported no error and the log ends in PASS. What are you missing to say the DUT is verified?
- Nothing: a thousand operations without an error is a verified DUT
- Running more seeds until code coverage reaches 100 %
- Knowing what the log would have said with a bug inside: run it with the mutated DUT
- Replacing the scoreboard with assertions, which check the protocol on the exact edge and not at the end
Watch it fail — the
PASSof a scoreboard that never saw an error says nothing: it may not have compared.VTALU_BUG=1flips one bit of the result and the testbench has to fail;make mutantedemands it for the three testbenches of days 1 and 2.
Review · Day 1
2 of 7 · The ALU spec
While the VTALU is running an operation, what do start and the operands have to do?
-
startdrops right away: it is a start pulse, and the DUT already latched everything - It makes no difference: the DUT registers them on the first edge
- The operands have to change on every cycle
-
startat 1 and the operands still untildonegoes up
Stable until
done— it is the DUT protocol, and it is exactly the reason the BFM exists: to wrap that rule in a single place so that no test forgets it. The first distractor describes a real protocol —pulse start, latched operands— that this DUT does not have.
Review · Day 1
3 of 7 · Functional coverage
The test runs 1000 random operations and the code coverage reads 100 %. What does that tell you about the verification?
- That the DUT is verified and the verification plan can be closed
- That the testbench has no bugs
- That few scenarios are missing: 100 % already walked the whole design
- Very little: it measures the RTL that ran, not the spec
Very little — code coverage measures the DUT; functional coverage measures the spec. A feature the designer never wrote gives 100 % of lines and 0 % of what matters, and the report is not going to tell you.
Review · Day 1
4 of 7 · covergroup
You declare a covergroup, you new() it, you run a thousand operations and the report reads 0 %. What is the first thing to look at?
- That nobody is calling
sample() - That the bins are badly defined and match no value at all
- That the DUT is not answering
- That
ignore_binsare missing
The
sample()— a covergroup does not sample itself: somebody has to call it, on the edge or when a transaction arrives. Without that call the code compiles, runs, and the report reads 0 without a single warning.
Review · Day 1
5 of 7 · Interfaces and BFM
What does the testbench gain when the protocol moves into a BFM?
- It stops talking in signals and starts talking in operations
- It simulates faster: one BFM task costs the simulator less than moving wires
- The testbench becomes synthesizable
- You save having to declare a
clk
It stops talking in signals — the BFM translates one operation into a handshake of signals. The tester never touches a wire again; the scoreboard and the coverage keep reading them, through the interface, until a monitor sends them transactions. It is the first step towards UVM.
Review · Day 1
6 of 7 · The verification plan
In the verification plan, what goes in the Measure column?
- How long the scenario takes to run, so the regression can be estimated
- The name of the testbench file that covers that row
- Which bin gets filled when the scenario happens
- How many times the test has to be run to call it covered
Which bin gets filled — the five columns are Feature, Scenario, Stimulus, Check and Measure, and this is the one that costs the most: it forces you to decide beforehand what is going to be counted. If it is left empty, nobody is going to find out that the scenario never happened — a case the random never touched and that has no bin is indistinguishable from one that happened a thousand times.
Review · Day 1
7 of 7 · clocking block
Are clocking blocks needed to write a UVM testbench without races?
- Yes: without a clocking block, driver and DUT always compete on the same edge
- Yes, and they are also part of the UVM library
- No: NBA in the driver plus scheduler discipline is enough
- No: the
uvm_driverreplaces them, since it already samples in the right region
They are not needed, and they are worth using anyway — they belong to SystemVerilog, not to UVM, and no
uvm_driversamples for you. What avoids the race is understanding the scheduler: a driver that drives with<=against a DUT that registers with<=is already deterministic. The clocking block does not replace that understanding, it wraps it — and that is where it pays off: reusable agents, VIP, gate-level and protocols with setup/hold in the spec.
Exercise · Day 1 · 1 of 2
A new operation, end to end
cd code/ejercicios/d1 && bash run.sh
- The VTALU has a free opcode,
3'b110. Teach it to shift, and teach the TB that it has to verify it too - It gets touched in three places: the RTL, the stimulus with its check, and the measure — the bin that does not exist today
- The
run.shfails until it printsEXERCISE OKand the report closes with 77 covered bins; the solution is right next to it
Half an hour, and it is worth leaving them alone: the statement and the README are enough.
The guaranteed stumble is the width: the hardware shifts by B[2:0], not by
the whole B, so with B = 8'h20 it shifts zero places and not thirty-two. The
scoreboard model has to do the same or the TB they wrote themselves
is going to accuse them. It is warned about in the README, and it still happens.
The third step is the most skipped one and the only one that fails silently:
bins single_cycle[] covers the range [add_op : xor_op], which reaches up to
3'b100. shr_op falls outside every bin, the simulation passes green and
— here is the worst part — the report still says 100 %: 76 out of 76. The
missing bin does not lower the percentage because it never made it into the
denominator. There is the lesson that holds for the rest of the course: an opcode
nobody measures is an opcode nobody verified, and the percentage is precisely the
metric that cannot see it.
With the solution it is 77 out of 77, starting from 86.8 % (66 out of 76). The ten
that were missing were a single one: the bucket Verilator invents for 3'b110, the
value the enum did not have — and the shift is precisely 3'b110, so it fills it
along the way. Worth showing both numbers and not only the percentage: that is why
the run.sh demands, on top of the 100 %, that single_cycle has one more bin: a
half-done job reaches the 100 % too.
Exercise · Day 1 · 2 of 2
The waves: when the log is not enough
cd code/ejercicios/d1b && bash run.sh
- The run aborts with a single line:
FAILED: A: e5 B: 0 op: mul_op result: fe01 ovf: 0.e5 * 00is0, notfe01. The scoreboard is right, and so is the DUT: what got it wrong is the BFM's handshake - That is where what the log gives you ends. The
run.shleavesondas.vcdnext to it:gtkwave ondas.vcd - Stage 1: two times that only exist in the viewer, in
respuesta.txt. Stage 2: the line of the BFM that explains them - It is the only exercise of the course that is not solved by reading the log
Half an hour, and it is the exercise that justifies the debug appendix: there it says that
the waves are the tool 47 % of the work gets done with, and up to here
the course had not made them use one even once.
The bug is a protocol one and it is the same one that comes back in the capstone: send_op
counts edges instead of waiting for the handshake. For single-cycle operations it
makes no difference; the multiplication takes four, so send_op returns too
early, the next stimulus overwrites A and B, and when done finally goes up the
scoreboard compares the new operands against the old result.
That is why the two times stage 1 asks for are not bureaucracy: the second is the
done of the first multiplication, and whoever looks for it in the viewer sees that
A and B have already changed. That is the answer to the why, and there is no way to
read it in the log.
The question to throw at the group when they finish: if the multiplier went
from four edges to five, which of the two versions of send_op finds out?
Neither counts right; only the one that waits for done keeps working. It is literally the
clue of the APB wait state of day 7.
And this is the 47 % from the trends slide, seen up close: the log says
failed and for the why you have to go somewhere else. Today that somewhere
else is the waves; on day 3, in reporting, it is the log that learns to say it.
Six out of seven, again
Closing the day
- The regression that sent the bug to the fab said
PASS. Today you have three ways of knowing whether thatPASSwas worth anything, and all three are columns of one table - What had to be tested? A plan: one row per scenario, written before the testbench, from the spec
- Who said it was right? A scoreboard that predicts and compares — and that you watched fail with a bug inside
- How much of the plan passed? A
covergroupwith one bin per row, and the per-bin report, not the percentage - The tester already asks for operations instead of moving wires: that was the BFM. What you still cannot do is change the stimulus without editing the file. Tomorrow is that
Thirty seconds, and they are the ones that close the arc of the day: the first
slide asked a question and this one answers it with what the student wrote,
not with theory. If the group is tired, put the 14 % slide back up for a second
before this one: the question was "how do you know you verified?", and the
short answer is "I have the plan, I watched it fail and I know which bin is
missing".
What not to over-promise: this does not prevent the respin. What it prevents
is sending to the fab a PASS nobody knew the meaning of.
The last bullet is the one day 2 opens with "Yesterday we left…": a tester
that only multiplies is copying the file and deleting five lines. That is the
problem classes solve, and it is worth leaving as a problem and not as a
promise.