Verify This! · the course to read

Day 1Why we verify, and the testbench without UVM

The same slides as the deck, with the instructor notes inside the text and the complete code. If you are taking the course on your own, start here and keep the deck alongside for the reviews.

Verify This!

An introduction to UVM

github.com/leandrotozzi/verifythis · SystemVerilog · Verilator · UVM 2020.3.1

see this slide in the deck ›

How to use this course

Especially if you are taking it on your own

If you are teaching the course, these two slides are skipped with the 1 key. They are here because most people who open this will not have an instructor alongside, and without the presenter notes half the material gets lost — and that half is also the hardest one to reconstruct by reading the code.

see this slide in the deck ›

How to use this course

Running it, moving around it, and where to look when something breaks

The last two are the ones that pay off most for someone studying alone, and the ones nobody reads until they are already stuck: the book carries these same notes inside the text, and the appendices are written as a table of symptoms, to look things up in rather than to read straight through.

see this slide in the deck ›

Agenda

Day 1 · ≈ 4 h · units 1 and 2

Unit 1 · Why we verify

Unit 2 · The testbench without UVM

Two exercises, and the second one is solved with the wave viewer

By the end of the day you can:

The four verbs below are the contract of the day, and it is worth reading them out loud before starting: on video they are the ten seconds in which someone decides whether this is the one they were looking for. Three are rows 2, 4 and 5 of the closing self-assessment, brought to the front; the one about the bug on purpose is the practice that runs through the whole course, from make mutante today to +BUG=1 in the capstone. Whoever can answer them at the end of the day does not need to watch anything else. And an honest warning for whoever comes from VHDL or from Verilog-2001, which is the declared audience: day 2 levels the OOP, but nobody levels SystemVerilog, and today interface, logic, enum, package, covergroup, clocking and an assert property all show up. It is all on one page in docs/systemverilog-para-el-que-viene-de-vhdl.md (in Spanish): half an hour before starting, and day 1 stops having two steps at once.

see this slide in the deck ›

Six out of seven

And the bug that reached silicon passed a green regression

How IC/ASIC projects end in 2024: 14 % get first silicon right and 75 % run late

It is the first content slide of the course on purpose: before explaining what UVM is and before any other chart, a question the student cannot answer yet. The number is from the Wilson Research Group 2024 survey and it is not an impression: only 14 % come out right on first silicon, so six out of seven need at least one respin, and it is the worst figure in twenty years of the survey. Asking who has done a tape-out and how it went hooks better than the chart. If they ask where the data comes from: res/trends/data.json, and the figures are regenerated with make figs. The honesty worth saying out loud: not every respin is functional —there are timing ones, analog ones, a spec that changed late—. But the one that is went through a regression that said PASS, because had it said FAILED it would not have been fabricated. That is the enemy of the day, and it is not the verification engineer nor their testbench: it is a PASS nobody knew the meaning of. The question in the last bullet is thrown to the group and not answered. The answers that will come are "100 % coverage", "I ran a thousand operations" and "nothing failed", and all three come back today: the first is the coverage section, the second is the plan, and the third is the one that hurts most, because nothing failed is exactly what the respin's regression said. The short answer arrives in the conventional testbench, when we put a bug into the DUT on purpose; the long one is the whole day, and the last slide of the day comes back to this question with what the student wrote.

see this slide in the deck ›

Why they come back: what is inside a chip today

What is inside 2024 IC/ASICs: asynchronous clocks, embedded processor, security, AI, RISC-V and safety

It is the first half of the answer to why they come back: inside there are two asynchronous clocks in almost all of them, a processor in five out of six and an AI accelerator in more than half. It is not that verification is done worse than twenty years ago; it is that there is much more to verify, and the question on the first slide is harder to answer than ever.

see this slide in the deck ›

Verification is not a stage, it is half the work

49 % of the designer's time goes to verification; 47 % of the verification engineer's time goes to debug

Here is the argument for why this course exists: the verification effort has already caught up with design. Verifying is not the final step before tape-out: it is half the project, with its own team and its own language. The lower 47 % is planted today and collected twice: in the second exercise this afternoon, when the log says FAILED and nothing else, and in the reporting section on day 3, which is where the log learns to say why.

see this slide in the deck ›

And it is verified, mostly, with UVM

What verification uses in 2024: UVM at 80 % in IC/ASIC and 50 % in FPGA

This one answers with what: eight out of ten ASIC projects verify with UVM, and half of the FPGA ones. What follows —the COBOL joke and the introduction— is what that UVM is and why it is not scary. And one thing worth leaving said now: UVM does not answer the question on the first slide. The testbench does; UVM is the way to write it so that the person next to you recognizes it.

see this slide in the deck ›

Is SystemVerilog the COBOL of electronics?

Language Reserved Keywords
ANSI COBOL 85 357
SystemVerilog 248
VHDL 2008 115
Verilog 95 102
C# 102
C++ 20 92
Python 3 35

" No academic computer scientists participated in the design of COBOL; all of those on the committee came from commerce or government"* Sound familiar?

It is a joke with a moral: SystemVerilog has 248 keywords —the ones in Annex B of IEEE 1800-2017— and nobody uses them all. In this course we are going to use about thirty. The same thing happens with UVM: the library is enormous and with six or seven classes you put together a complete testbench. It is useful for lowering the anxiety of whoever arrives scared by the size of UVM. And it is the bridge to the next section: eight out of ten verify with UVM, UVM is written in this language, and on days 1 and 2 everything UVM will later hand you ready-made gets written by hand. Thirty keywords are enough.

see this slide in the deck ›

Introduction

What is UVM?

The link with what came before is one line: eight out of ten projects verify with this. And the clarification that keeps UVM from looking like the answer to the question of the day: it is not. The answer is a testbench with a plan, a check and a measure; UVM is the way to write it so it scales and so someone else recognizes it. The sentence that orders the section is the one in the second bullet, and it is worth saying it loudly because it lowers the anxiety of whoever arrives scared: UVM does nothing you could not do yourself. There is no magic inside. There is a factory, some ports, some phases and a lot of agreement about names. If somebody asks why bother then: because of the last bullet. A verification engineer who changes project —or company— opens the testbench and recognizes the structure on day one. That is what you are buying. It is not simulation speed and it is not fewer lines: it is that the structure is the same everywhere. The historical note is useful for whoever comes from the industry: until 2011 every vendor had its own, and porting a testbench from VMM to OVM meant rewriting it. UVM exists because that was unsustainable, not because somebody had a better idea. And one clarification that avoids a classic misunderstanding: Open Source here means that the library is free (Apache 2.0). The simulator that runs it almost always is not — except in this course, which is what all of this is about.

see this slide in the deck ›

Introduction

Where it comes from

From eRM, VMM, AVM and OVM to UVM

The diagram in one sentence: six acronyms from four vendors that converge. eRM was Verisity's (the e language), RVM and later VMM Synopsys', AVM Mentor's and URM Cadence's; OVM merged AVM with URM, and it is the base UVM came out of. What has to be pointed out is the date at the bottom and the jump it means: UVM 1.0 is from 2011, which means this is young. Somebody with fifteen years of career today started before it existed. And the moral that gets used for the rest of the course: when they find code with ovm_component instead of uvm_component, it is not a typo — it is a testbench from before 2011 that nobody migrated. It happens more than it seems.

see this slide in the deck ›

Introduction

A SystemVerilog testbench, from the inside

Anatomy of a SystemVerilog testbench

This is the testbench we are going to write, by hand, on days 1 and 2. It is worth saying it like that: the boxes in this diagram are not theory, they are the files that are going to be open between today and day 4. Point at the tester, the driver, the monitor and the scoreboard, and make the point that none of those names is UVM's: they are the names anybody gives to the pieces of a testbench, in any language. UVM did not invent them; it standardized them.

see this slide in the deck ›

Introduction

The same testbench, with UVM structure

The same testbench with UVM structure

It is worth staying here a while: this diagram is the map of the rest of the course. Point at driver, monitor, scoreboard and sequencer, and say that each one is going to be a section. By the end of day 5 the student is going to have everything except the agent and the sequencer, which are day 6. The comparison worth making live is against the previous diagram: they are the same boxes. What UVM adds is the tree that contains them, the ports they talk through and the phases that build them in order. Nothing else, and that is not little.

see this slide in the deck ›

Introduction

The class tree

The UVM base class hierarchy

This diagram is scary and that is why it is worth defusing it right away: out of all of this we are going to extend ten: uvm_component, uvm_test, uvm_env, uvm_agent, uvm_driver, uvm_monitor, uvm_subscriber, uvm_sequence_item, uvm_sequence and uvm_callback. They are the amber ones in the drawing minus the scoreboard —which in the course is a uvm_subscriber—, plus the sequence and the callback. What does have to be marked is the split at the top, because it explains half the course: hanging off uvm_object is the data —the transactions, the sequences, the configs— and hanging off uvm_component is the structure —everything that lives in the tree and has phases—. An object is created and thrown away; a component is built once and lasts the whole simulation. The colours say what you do with each one: the amber ones you extend yourself, the grey ones are never touched —they are only there for the others to inherit from— and uvm_sequencer is the only green one because it is the only one instantiated without extending it, with a parameterized typedef. That comes on day 6; the only time the course extends it is for the virtual sequencer on day 7. The question to throw at the group when we get to agents: is a uvm_sequence an object or a component? An object. And that is why it does not show up in print_topology(). It is the same tree that is in the printed cheat sheet, top left, as text, to look at again when needed.

see this slide in the deck ›

Introduction

Run more tests writing less code

This is the slide you have to be able to repeat at the end of the course, because it is the whole thesis. And it is worth being honest: "run more tests, write less code" is a vendor slogan, but the verifiable part is true and you can point at it with your finger on day 6 — the add_test of the sequences is two lines, and it changes the whole stimulus without touching a single class of the env. The order of the three hooks is not accidental, it goes from smallest to largest: a constraint changes values, an override changes a class, a callback changes behaviour at one point of the class that already exists. All three are a section of the course —Constrained random on day 5, The factory pattern on day 2 and Callbacks on day 6—, so this slide can be put up three more times. And the last line is the most underestimated one. A test with a thousand seeds is not "the same test a thousand times": it is a thousand different scenarios for the price of writing one. It is literally the d7-semillas exercise, the one day 7 starts with. And the close that leads to the DUT: nothing on this slide answers how do you know you verified? yet. Answering it is what comes next —the spec, the plan and a testbench that can be put to the test—, and only with that written by hand does what UVM adds become visible.

see this slide in the deck ›

The VTALU spec

VTALU protocol: start and operands stable until done

The protocol is everything you have to respect: start up and operands stable until done. Careful with the fine print: done is a pulse on the multiplication (it turns itself off) and a level on add/sub/and/xor, where it is start && (op != no_op). It is the kind of detail that only shows up when the testbench hangs and nobody knows why.

see this slide in the deck ›

The VTALU spec

Operation Opcode Cycles ovf
no_op 3'b000 — 0
add_op 3'b001 1 0
sub_op 3'b010 1 A < B
and_op 3'b011 1 0
xor_op 3'b100 1 0
mul_op 3'b101 4 0
free 3'b110 1 0
rst_op 3'b111 — —

rst_op belongs to the testbench, not to the RTL: the DUT sees 3'b111 as unused. It is in the enum because the tester asks for the reset as if it were one more operation, and that way the reset enters the functional coverage. A good moment to preview that the coverage plan includes "any operation after a reset". The ovf column is the one that gets the most discussion, and that is good: with 8-bit operands and a 16-bit result, neither the addition nor the multiplication can overflow — the subtraction can, and only when A < B. Which means there is an output of the DUT that for five of the six operations is always 0. Ask out loud: how do you verify a signal that almost never moves? The answer is today's, in functional coverage, and it is a bin. And the free opcode: 3'b110 is not there by chance and not by oversight. It is this afternoon's exercise, and it is worth saying now so that nobody reads it as a design slip.

see this slide in the deck ›

The VTALU spec

Single Cycle: Add - Sub - AND - XOR

code/vtalu_dut/vtalu_1c.sv · #the-two-resets
   // The two resets are deliberately different: the one on the result is
   // SYNCHRONOUS and the one on done is ASYNCHRONOUS. Not an oversight: the spec.
   always_ff @(posedge clk) begin
      if (!reset_n) begin
         result_1c <= 16'h0000;
         ovf_1c    <= 1'b0;
      end else if (start) begin
         ovf_1c <= (op == 3'b010) && (A < B);
         case (op)
            3'b001:  result_1c <= {8'h00, A} + {8'h00, B};
            3'b010:  result_1c <= {8'h00, A} - {8'h00, B};
            3'b011:  result_1c <= {8'h00, A} & {8'h00, B};
            3'b100:  result_1c <= {8'h00, A} ^ {8'h00, B};
            default: ;  // 3'b110 is free: it is the day 1 exercise
         endcase
      end
   end

   // done goes high on the edge if start is high and the op is not no_op
   always_ff @(posedge clk or negedge reset_n) begin
      if (!reset_n) done_1c <= 1'b0;
      else done_1c <= start && (op != 3'b000);
   end
el archivo entero, 44 líneas
// The single-cycle operations: add, sub, and, xor. Registers A op B on the edge.
//
// Subtraction is the only one that can overflow: with 8-bit operands and a
// 16-bit result, neither the addition nor the multiplication can. A - B with
// A < B can, and that is why ovf_1c exists. The result in that case is the
// two's complement truncated to 16 bits -- it wraps to 0xFFxx -- which is what
// the hardware does and what the scoreboard has to predict correctly.
module vtalu_1c (
    input  logic [ 7:0] A,
    input  logic [ 7:0] B,
    input  logic        clk,
    input  logic [ 2:0] op,
    input  logic        reset_n,
    input  logic        start,
    output logic        done_1c,
    output logic        ovf_1c,
    output logic [15:0] result_1c
);

   // The two resets are deliberately different: the one on the result is
   // SYNCHRONOUS and the one on done is ASYNCHRONOUS. Not an oversight: the spec.
   always_ff @(posedge clk) begin
      if (!reset_n) begin
         result_1c <= 16'h0000;
         ovf_1c    <= 1'b0;
      end else if (start) begin
         ovf_1c <= (op == 3'b010) && (A < B);
         case (op)
            3'b001:  result_1c <= {8'h00, A} + {8'h00, B};
            3'b010:  result_1c <= {8'h00, A} - {8'h00, B};
            3'b011:  result_1c <= {8'h00, A} & {8'h00, B};
            3'b100:  result_1c <= {8'h00, A} ^ {8'h00, B};
            default: ;  // 3'b110 is free: it is the day 1 exercise
         endcase
      end
   end

   // done goes high on the edge if start is high and the op is not no_op
   always_ff @(posedge clk or negedge reset_n) begin
      if (!reset_n) done_1c <= 1'b0;
      else done_1c <= start && (op != 3'b000);
   end

endmodule : vtalu_1c

The asymmetry of the two resets is not an oversight: it comes from the original VHDL and is copied as is on purpose. It is exactly the kind of detail a testbench has to expose, and that is why the DUT was not "cleaned up" when it was translated. Say it out loud: this contradicts the bare asynchronous of the spec bullet, and no row of the plan looks at what the reset leaves on the outputs: the reset row looks at the operation that comes after. It is a gap in the plan, and for now it stays that way. That done is a level here and a pulse on the multiplication is the fine print that is going to hang the first testbench of the course. It is worth going back to the waveform of the first slide and pointing at it again. The default: ; of the case is what makes no_op and the unused opcodes not change the result: the ALU keeps whatever it computed last.

see this slide in the deck ›

The VTALU spec

Multi Cycle: Multiplication

code/vtalu_dut/vtalu_mult.sv · #the-pipeline
   // Four register stages. done is pipelined alongside the data, and each
   // stage clears itself with (& ~done_mult): that is what makes done a
   // one-cycle pulse instead of staying high for as long as start does. It is
   // subtle -- it is copied straight from the VHDL.
   always_ff @(posedge clk or negedge reset_n) begin
      if (!reset_n) begin
         done_mult <= 1'b0;
         done3 <= 1'b0;
         done2 <= 1'b0;
         done1 <= 1'b0;
         a_int <= 8'h00;
         b_int <= 8'h00;
         mult1 <= 16'h0000;
         mult2 <= 16'h0000;
         result_mult <= 16'h0000;
      end else begin
         a_int <= A;
         b_int <= B;
         mult1 <= a_int * b_int;
         mult2 <= mult1;
         result_mult <= mult2;
         done3 <= start & ~done_mult;
         done2 <= done3 & ~done_mult;
         done1 <= done2 & ~done_mult;
         done_mult <= done1 & ~done_mult;
      end
   end
el archivo entero, 46 líneas
// Multi-cycle multiplication: done rises on the FOURTH clock edge after start
// (the one-cycle ops raise it on the first). The latency is deliberate: it is
// what forces the testbench to wait for 'done' instead of counting cycles.
module vtalu_mult (
    input  logic [ 7:0] A,
    input  logic [ 7:0] B,
    input  logic        clk,
    input  logic        reset_n,
    input  logic        start,
    output logic        done_mult,
    output logic [15:0] result_mult
);

   logic [7:0] a_int, b_int;  // input registers
   logic [15:0] mult1, mult2;  // pipeline registers
   logic done3, done2, done1;  // done travels down the same pipeline

   // Four register stages. done is pipelined alongside the data, and each
   // stage clears itself with (& ~done_mult): that is what makes done a
   // one-cycle pulse instead of staying high for as long as start does. It is
   // subtle -- it is copied straight from the VHDL.
   always_ff @(posedge clk or negedge reset_n) begin
      if (!reset_n) begin
         done_mult <= 1'b0;
         done3 <= 1'b0;
         done2 <= 1'b0;
         done1 <= 1'b0;
         a_int <= 8'h00;
         b_int <= 8'h00;
         mult1 <= 16'h0000;
         mult2 <= 16'h0000;
         result_mult <= 16'h0000;
      end else begin
         a_int <= A;
         b_int <= B;
         mult1 <= a_int * b_int;
         mult2 <= mult1;
         result_mult <= mult2;
         done3 <= start & ~done_mult;
         done2 <= done3 & ~done_mult;
         done1 <= done2 & ~done_mult;
         done_mult <= done1 & ~done_mult;
      end
   end

endmodule : vtalu_mult

The four-edge latency is on purpose, and it is the reason the conventional testbench exists: it forces the testbench to wait for done instead of reading the result on the next cycle. A combinational DUT would teach nothing. The & ~done_mult is subtle and worth reading slowly: without it, while start was up the chain would reload itself and done would stay at 1. It is copied from the original VHDL as is. The question that orders the slide: what happens if the testbench sends a multiplication and reads the result on the next cycle? It reads the previous result, and the scoreboard reports an error that is not in the DUT.

see this slide in the deck ›

The VTALU spec

Top Level

code/vtalu_dut/vtalu.sv · #the-mux
   assign es_mult = (op == 3'b101);

   assign start_1c = es_mult ? 1'b0 : start;
   assign start_mult = es_mult ? start : 1'b0;
   assign result_mux = es_mult ? result_mult : result_1c;
   assign done = es_mult ? done_mult : done_1c;
el archivo entero, 75 líneas
// VTALU top: A, B, clk, op[2:0], reset_n, start -> done, ovf, result[15:0].
//
// The op is NOT decided by a single loose bit any more: there is a decode, which
// is what a real design does, and what lets the operations avoid having to sit
// in tidy halves of the opcode space.
//
//   no_op  3'b000      and_op 3'b011
//   add_op 3'b001      xor_op 3'b100
//   sub_op 3'b010      mul_op 3'b101
//                      3'b110 free -- the day 1 exercise
//   rst_op 3'b111 never reaches the DUT: the BFM turns it into lowering reset_n
module vtalu (
    input  logic [ 7:0] A,
    input  logic [ 7:0] B,
    input  logic        clk,
    input  logic [ 2:0] op,
    input  logic        reset_n,
    input  logic        start,
    output logic        done,
    output logic        ovf,
    output logic [15:0] result
);

   logic done_1c, done_mult;
   logic ovf_1c;
   logic [15:0] result_1c, result_mult, result_mux;
   logic start_1c, start_mult;
   logic es_mult;

   assign es_mult = (op == 3'b101);

   assign start_1c = es_mult ? 1'b0 : start;
   assign start_mult = es_mult ? start : 1'b0;
   assign result_mux = es_mult ? result_mult : result_1c;
   assign done = es_mult ? done_mult : done_1c;

   // The multiplication cannot overflow: 8 x 8 fits exactly in 16 bits.
   assign ovf = es_mult ? 1'b0 : ovf_1c;

   // The injectable mutation -- the only liberty taken in this DUT, and the same
   // one apb_regs.sv takes with its bug_en. With +VTALU_BUG, bit 0 of the result
   // comes out flipped for every operation. Nothing turns it on in a normal run:
   // it is 'make mutante' that runs the examples with it and DEMANDS that they
   // fail. A scoreboard that never caught anything is not tested, it is
   // decoration -- and until this existed there was no way of telling one from
   // the other.
   // It hangs off the decode above and not inside it so that the four lines the
   // day 1 slide quotes stay the plain mux they claim to be.
   bit bug;
   initial bug = $test$plusargs("VTALU_BUG");
   assign result = result_mux ^ {15'h0000, bug};

   vtalu_1c u_1c (
       .A        (A),
       .B        (B),
       .clk      (clk),
       .op       (op),
       .reset_n  (reset_n),
       .start    (start_1c),
       .done_1c  (done_1c),
       .ovf_1c   (ovf_1c),
       .result_1c(result_1c)
   );

   vtalu_mult u_mult (
       .A          (A),
       .B          (B),
       .clk        (clk),
       .reset_n    (reset_n),
       .start      (start_mult),
       .done_mult  (done_mult),
       .result_mult(result_mult)
   );

endmodule : vtalu

That both blocks share A, B and clk and only split on the start is what makes the DUT have a single interface. Worth pointing out because it is the shape the vtalu_bfm of interfaces and BFM is going to have. The complete file, with both instances and their connections, is in code/vtalu_dut/vtalu.sv. Here there are only the decode and the four lines that decide. Worth pointing out why it is a decode and not a loose bit: with an op[2] the opcodes would have to sit in tidy halves of the space, and the design would lose the freedom to assign them as convenient. A decode costs one gate and does not tie the spec down. And along the way, the free opcode 3'b110 falls on the single-cycle side without anybody having to do anything. The DUT does not validate the opcode; the one that has to catch it is the verification plan.

see this slide in the deck ›

The VTALU spec

How to run it

make u4/tests          # one UVM example:  ~1 min 30 the first time
make u4/tests          # the second, with ccache:      ~15 seconds
make               # the 38 examples

Here it is worth leaving the deck and running it live. The numbers on the slide are measured on a 12-core laptop. On a 2-core machine —a free Codespaces— the first one is 4 minutes; the second is still 15 seconds, because a ccache hit is copying a file and that does not depend on how many cores you have. Which means the expensive part is paid once. What has to be explained is where the difference between 1 min 30 and 15 seconds comes from, because it is not magic: Verilator compiles every simulation to a native binary, and for that it generates about 2300 C++ files —almost all of them from the UVM library—. ccache stores the result of each one; the second time it does not compile again, it copies. It is in the Dockerfile and in the devcontainer, and outside it is enough to have it installed. And the one to stress anyway: Verilator is free — the student takes it home and keeps practising without asking anybody for a licence.

see this slide in the deck ›

The VTALU spec

Looking at the real waves

cd code/u2/convencional
VLT_TRACE=1 bash run.sh      # compiles with --trace and dumps vtalu.vcd
gtkwave vtalu.vcd            # or surfer, or whatever viewer you use

It is worth opening it live once, even for thirty seconds: load op_set, start and done, and show the one-cycle pulse of the multiplication next to the level of the single-cycle ones. It is the fine print of the protocol of slide 1, but seen. And say it explicitly: whoever takes the course alone has their safety net here. When an exercise does not work out, the answer is almost always in the waves before it is in the code.

see this slide in the deck ›

The VTALU spec

Summary of the unit

Closing of the section that looks like RTL and is really about verification. The question to close with: which of these seven facts is the most expensive one to forget? The one about done, and it will show today, in the waves exercise. Worth saying why the DUT was not "cleaned up" when it was translated from VHDL: the asymmetry of the resets and the unvalidated opcode are exactly the kind of thing a testbench has to expose. A tidy DUT teaches nothing.

see this slide in the deck ›

The verification plan

The most professional deliverable of the discipline, and the cheapest

This is the slide that orders the whole course, and that is why it comes before the first line of testbench. Everything that comes after —the stimulus, the scoreboard, the covergroups, the assertions— are columns of this table. It is worth saying it head on: the plan is not bureaucracy. It is the only serious answer to "are you done verifying yet?", and it is the first thing a technical lead asks for when they ask how the block is going. The empty template is in docs/en/verification-plan.md, and it gets used twice in the course: here to read it, and on day 7 to fill it from scratch.

see this slide in the deck ›

The verification plan

The five columns

Column The question it answers What does not go in
Feature which part of the spec does this row come from? the name of a testbench file
Scenario which concrete situation has to be provoked? "test the ALU"
Stimulus who provokes it: the random or a directed case? "by hand"
Check who says it went right? "you look at the wave"
Measure which bin gets filled when it happens? "you see it in the log"

It is worth reading the right-hand column out loud: they are the five mistakes people make the first time, and all five sound reasonable when you write them. The hardest column is the measure one, because it forces you to decide beforehand what you are going to count. It is the one that on day 1 still cannot be filled —the bins do not exist until the next unit— and that is why the plan gets filled as the testbench grows.

see this slide in the deck ›

The verification plan

Two rules that come out of the columns

The difference between verified and simulated is who finds out when it fails at three in the morning in the regression. If the check is a pair of eyes, nobody finds out: the regression passes green with the bug inside. The third bullet is a promise the course keeps on day 7: until then every check is a scoreboard check, and half the protocol goes unchecked. Worth saying now so that day 7 does not look like an add-on. A serious plan has both columns, and the VTALU table —which is seen whole in the next unit, already with the bins— has eight scoreboard rows and four assertion ones.

see this slide in the deck ›

The conventional testbench

The VTALU plan, with three columns to fill

Feature Scenario Stimulus Check Measure
ALU the six operations
ALU operands at 00 and at FF
reset operate after a reset
mult a mult after a single-cycle one
ALU the same operation twice in a row
sub subtract too much: A < B, and ovf goes up

The order matters and it is the one from the previous unit: the plan first, the testbench after. If you write the testbench first, you end up measuring what the testbench does instead of what the spec asks for. Worth reading the six rows one at a time and asking which one is missing. The answer that usually comes up —"test intermediate values"— is good for discussing why it is not there: 256 × 256 × 5 combinations do not get simulated, and that is where random comes in. The last one is the one that brings today's topic: the DUT has TWO outputs, and a row of the plan that only closes by looking at both. A scoreboard that compares result and nothing else passes green with ovf stuck at zero. The table comes back twice: in functional coverage with the measure column filled in, row by row, and at the end of that whole section, with the twelve rows and the three columns. Worth announcing now so nobody reads it as decoration: it is the same table, filling up.

see this slide in the deck ›

The conventional testbench

The stimulus: a thousand operations, and the protocol by hand

code/u2/convencional/vtalu_tb.sv · #stimulus-loop
      repeat (1000) begin
         @(negedge clk);
         op_set = get_op();
         A = get_data();
         B = get_data();
         start = 1'b1;
         case (op_set) // handle the start signal
           no_op: begin
              @(posedge clk);
              start = 1'b0;
           end
           rst_op: begin
              reset_n = 1'b0;
              start = 1'b0;
              @(negedge clk);
              reset_n = 1'b1;
           end
           default: begin
              // Wait for an EDGE with done up, not for done itself: on the
              // one-cycle ops done is a level still up from the previous op,
              // and a plain wait(done) drops this one without a sound.
              enviadas++;
              do @(negedge clk); while (done == 0);
              start = 1'b0;
           end
         endcase // case (op_set)
      end
      $finish;
el archivo entero, 256 líneas
// Typical TB:
// ----------
//  * Functional Coverage
//  * Self Checking
//  * Constrained Random Stimulus

// Why is this bad?
//  * No modularity (everything in a single file)
//  * Almost nothing here can be reused
module top;

   typedef enum bit[2:0] {no_op  = 3'b000,
                          add_op = 3'b001,
                          sub_op = 3'b010,
                          and_op = 3'b011,
                          xor_op = 3'b100,
                          mul_op = 3'b101,
                          rst_op = 3'b111} operation_t;
   byte         unsigned        A;
   byte         unsigned        B;
   bit          clk;
   bit          reset_n;
   wire [2:0]   op;
   bit          start;
   wire         done;
   wire [15:0]  result;
   wire         ovf;
   operation_t  op_set;

   assign op = op_set;

   vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);

`ifdef VLT_TRACE
   // Waves for GTKWave:  VLT_TRACE=1 bash run.sh  &&  gtkwave vtalu.vcd
   // Behind an `ifdef because $dumpvars does not compile without --trace, and
   // the define comes from the same VLT_TRACE that adds the flag (common.sh).
   initial begin
      $dumpfile("vtalu.vcd");
      $dumpvars;
   end
`endif

   //op_cov: make sure every operation, and every interaction between them, is covered
   covergroup op_cov;
      coverpoint op_set {
         bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
         bins multi_cycle = {mul_op};

         // => are transitional coverage. From => To
         // Transition bins. Verilator 5.052 does NOT implement them yet: any form
         // with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
         // code/verilator/repro-cg-transition.sv. They stay because they are part
         // of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
         bins opn_rst[] = ([add_op:mul_op] => rst_op);
         bins rst_opn[] = (rst_op => [add_op:mul_op]);

         bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
         bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);

         bins twoops[] = ([add_op:mul_op] [* 2]);
         bins manymult = (mul_op [* 3:5]);
`endif
      }
   endgroup

   covergroup zeros_or_ones_on_ops;
      // Check the corners where the inputs are all 0s or all 1s
      all_ops : coverpoint op_set {
         ignore_bins null_ops = {rst_op, no_op};}

      a_leg: coverpoint A {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }

      b_leg: coverpoint B {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }
      // VTALU rev2: the borrow of the subtraction, the row of the plan that the
      // scoreboard can only close by looking at TWO outputs. Without this bin, a
      // subtraction that never went negative looks exactly like one that did.
      borrow: coverpoint ((op_set == sub_op) && (A < B)) {
         bins hubo_borrow = {1};
         ignore_bins resto = {0};
      }

      op_00_FF:  cross a_leg, b_leg, all_ops {
         bins add_00 = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins add_FF = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins sub_00 = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins sub_FF = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins and_00 = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins and_FF = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins xor_00 = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins xor_FF = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_00 = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins mul_FF = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_max = binsof (all_ops) intersect {mul_op} &&
                        (binsof (a_leg.ones) && binsof (b_leg.ones));

         ignore_bins others_only =
                         binsof(a_leg.others) && binsof(b_leg.others);
      }
   endgroup

   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end

   op_cov oc;
   zeros_or_ones_on_ops c_00_FF;

   initial begin : coverage
      oc = new();
      c_00_FF = new();

      forever begin @(posedge clk);
         oc.sample();
         c_00_FF.sample();
      end
   end : coverage

   function operation_t get_op();
      bit [2:0] op_choice;
      op_choice = $random;
      case (op_choice)
        3'b000 : return no_op;
        3'b001 : return add_op;
        3'b010 : return sub_op;
        3'b011 : return and_op;
        3'b100 : return xor_op;
        3'b101 : return mul_op;
        3'b110 : return rst_op;
        3'b111 : return rst_op;
      endcase // case (op_choice)
   endfunction : get_op

   function byte get_data();
      // This is how the random gets biased
      bit [1:0] zero_ones;
      zero_ones = $random;
      if (zero_ones == 2'b00)
        return 8'h00;
      else if (zero_ones == 2'b11)
        return 8'hFF;
      else
        return $random;
   endfunction : get_data

   // Sent by the tester, compared by the scoreboard: the final block below
   // demands that the two match.
   int enviadas, chequeadas;

   // Scoreboard loop: checks that everything came out right
   always @(posedge done) begin : scoreboard
      shortint predicted_result;
      bit      predicted_ovf;
      #1;
      case (op_set)
        add_op: predicted_result = A + B;
        sub_op: predicted_result = A - B;
        and_op: predicted_result = A & B;
        xor_op: predicted_result = A ^ B;
        mul_op: predicted_result = A * B;
      endcase // case (op_set)

      // ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
      // output, neither the addition nor the multiplication can overflow.
      predicted_ovf = (op_set == sub_op) && (A < B);

      if ((op_set != no_op) && (op_set != rst_op)) begin
        chequeadas++;
        if (predicted_result != result || predicted_ovf != ovf)
          $error ("FAILED: A: %0h  B: %0h  op: %s result: %0h ovf: %0b",
                  A, B, op_set.name(), result, ovf);
      end
   end : scoreboard

   // Randomize the stimulus
   // get_op and get_data are Constrained Random Data
   // The point of CRD is to build values that are random but legal
   initial begin : tester
      reset_n = 1'b0;
      @(negedge clk);
      @(negedge clk);
      reset_n = 1'b1;
      start = 1'b0;
      repeat (1000) begin
         @(negedge clk);
         op_set = get_op();
         A = get_data();
         B = get_data();
         start = 1'b1;
         case (op_set) // handle the start signal
           no_op: begin
              @(posedge clk);
              start = 1'b0;
           end
           rst_op: begin
              reset_n = 1'b0;
              start = 1'b0;
              @(negedge clk);
              reset_n = 1'b1;
           end
           default: begin
              // Wait for an EDGE with done up, not for done itself: on the
              // one-cycle ops done is a level still up from the previous op,
              // and a plain wait(done) drops this one without a sound.
              enviadas++;
              do @(negedge clk); while (done == 0);
              start = 1'b0;
           end
         endcase // case (op_set)
      end
      $finish;
   end : tester

   // The testbench checks itself: every operation sent has to come back
   // through the scoreboard. A tester that drops operations -- while the
   // coverage counts them anyway -- passes green and has measured nothing.
   final begin
      if (enviadas != chequeadas)
        $error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
      else
        $display ("%0d operations sent, %0d checked", enviadas, chequeadas);
   end
endmodule : top

This is the "before" slide of the whole course: the stimulus and the protocol mixed in the same loop. Worth pointing at it, because the next five units are successive separations of these thirty lines. The do … while (done == 0) of the default is the one that hangs if the DUT does not answer, and it is the place the student is going to end up in the first time they break something. The safety net is VLT_TRACE=1 and the waves. And why it is not a wait(done), which is the first thing anybody writes: on the one-cycle operations done is a level, and on the negedge where the tester loads the next operation it is still up from the previous one. The wait(done) does not block, start drops at the same instant and the DUT never sees that operation. There is no error: the scoreboard does not fire, and the coverage —which samples op_set— counts it anyway. This testbench had it, and it dropped four in ten one-cycle operations with the report in green: on d1, 77 shifts checked against 130 with the fix. What catches it is the sent-versus- checked counter, and it is the cheapest silent trap to guard against. The bias of get_data() —a quarter at 00, a quarter at FF, half in the middle— is exactly what the transactions are going to write in one line with dist and :/. Worth naming it now so that the saving is visible later.

see this slide in the deck ›

The conventional testbench

The self-checking: predict and compare

code/u2/convencional/vtalu_tb.sv · #scoreboard-block
   // Sent by the tester, compared by the scoreboard: the final block below
   // demands that the two match.
   int enviadas, chequeadas;

   // Scoreboard loop: checks that everything came out right
   always @(posedge done) begin : scoreboard
      shortint predicted_result;
      bit      predicted_ovf;
      #1;
      case (op_set)
        add_op: predicted_result = A + B;
        sub_op: predicted_result = A - B;
        and_op: predicted_result = A & B;
        xor_op: predicted_result = A ^ B;
        mul_op: predicted_result = A * B;
      endcase // case (op_set)

      // ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
      // output, neither the addition nor the multiplication can overflow.
      predicted_ovf = (op_set == sub_op) && (A < B);

      if ((op_set != no_op) && (op_set != rst_op)) begin
        chequeadas++;
        if (predicted_result != result || predicted_ovf != ovf)
          $error ("FAILED: A: %0h  B: %0h  op: %s result: %0h ovf: %0b",
                  A, B, op_set.name(), result, ovf);
      end
   end : scoreboard
el archivo entero, 256 líneas
// Typical TB:
// ----------
//  * Functional Coverage
//  * Self Checking
//  * Constrained Random Stimulus

// Why is this bad?
//  * No modularity (everything in a single file)
//  * Almost nothing here can be reused
module top;

   typedef enum bit[2:0] {no_op  = 3'b000,
                          add_op = 3'b001,
                          sub_op = 3'b010,
                          and_op = 3'b011,
                          xor_op = 3'b100,
                          mul_op = 3'b101,
                          rst_op = 3'b111} operation_t;
   byte         unsigned        A;
   byte         unsigned        B;
   bit          clk;
   bit          reset_n;
   wire [2:0]   op;
   bit          start;
   wire         done;
   wire [15:0]  result;
   wire         ovf;
   operation_t  op_set;

   assign op = op_set;

   vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);

`ifdef VLT_TRACE
   // Waves for GTKWave:  VLT_TRACE=1 bash run.sh  &&  gtkwave vtalu.vcd
   // Behind an `ifdef because $dumpvars does not compile without --trace, and
   // the define comes from the same VLT_TRACE that adds the flag (common.sh).
   initial begin
      $dumpfile("vtalu.vcd");
      $dumpvars;
   end
`endif

   //op_cov: make sure every operation, and every interaction between them, is covered
   covergroup op_cov;
      coverpoint op_set {
         bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
         bins multi_cycle = {mul_op};

         // => are transitional coverage. From => To
         // Transition bins. Verilator 5.052 does NOT implement them yet: any form
         // with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
         // code/verilator/repro-cg-transition.sv. They stay because they are part
         // of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
         bins opn_rst[] = ([add_op:mul_op] => rst_op);
         bins rst_opn[] = (rst_op => [add_op:mul_op]);

         bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
         bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);

         bins twoops[] = ([add_op:mul_op] [* 2]);
         bins manymult = (mul_op [* 3:5]);
`endif
      }
   endgroup

   covergroup zeros_or_ones_on_ops;
      // Check the corners where the inputs are all 0s or all 1s
      all_ops : coverpoint op_set {
         ignore_bins null_ops = {rst_op, no_op};}

      a_leg: coverpoint A {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }

      b_leg: coverpoint B {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }
      // VTALU rev2: the borrow of the subtraction, the row of the plan that the
      // scoreboard can only close by looking at TWO outputs. Without this bin, a
      // subtraction that never went negative looks exactly like one that did.
      borrow: coverpoint ((op_set == sub_op) && (A < B)) {
         bins hubo_borrow = {1};
         ignore_bins resto = {0};
      }

      op_00_FF:  cross a_leg, b_leg, all_ops {
         bins add_00 = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins add_FF = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins sub_00 = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins sub_FF = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins and_00 = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins and_FF = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins xor_00 = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins xor_FF = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_00 = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins mul_FF = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_max = binsof (all_ops) intersect {mul_op} &&
                        (binsof (a_leg.ones) && binsof (b_leg.ones));

         ignore_bins others_only =
                         binsof(a_leg.others) && binsof(b_leg.others);
      }
   endgroup

   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end

   op_cov oc;
   zeros_or_ones_on_ops c_00_FF;

   initial begin : coverage
      oc = new();
      c_00_FF = new();

      forever begin @(posedge clk);
         oc.sample();
         c_00_FF.sample();
      end
   end : coverage

   function operation_t get_op();
      bit [2:0] op_choice;
      op_choice = $random;
      case (op_choice)
        3'b000 : return no_op;
        3'b001 : return add_op;
        3'b010 : return sub_op;
        3'b011 : return and_op;
        3'b100 : return xor_op;
        3'b101 : return mul_op;
        3'b110 : return rst_op;
        3'b111 : return rst_op;
      endcase // case (op_choice)
   endfunction : get_op

   function byte get_data();
      // This is how the random gets biased
      bit [1:0] zero_ones;
      zero_ones = $random;
      if (zero_ones == 2'b00)
        return 8'h00;
      else if (zero_ones == 2'b11)
        return 8'hFF;
      else
        return $random;
   endfunction : get_data

   // Sent by the tester, compared by the scoreboard: the final block below
   // demands that the two match.
   int enviadas, chequeadas;

   // Scoreboard loop: checks that everything came out right
   always @(posedge done) begin : scoreboard
      shortint predicted_result;
      bit      predicted_ovf;
      #1;
      case (op_set)
        add_op: predicted_result = A + B;
        sub_op: predicted_result = A - B;
        and_op: predicted_result = A & B;
        xor_op: predicted_result = A ^ B;
        mul_op: predicted_result = A * B;
      endcase // case (op_set)

      // ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
      // output, neither the addition nor the multiplication can overflow.
      predicted_ovf = (op_set == sub_op) && (A < B);

      if ((op_set != no_op) && (op_set != rst_op)) begin
        chequeadas++;
        if (predicted_result != result || predicted_ovf != ovf)
          $error ("FAILED: A: %0h  B: %0h  op: %s result: %0h ovf: %0b",
                  A, B, op_set.name(), result, ovf);
      end
   end : scoreboard

   // Randomize the stimulus
   // get_op and get_data are Constrained Random Data
   // The point of CRD is to build values that are random but legal
   initial begin : tester
      reset_n = 1'b0;
      @(negedge clk);
      @(negedge clk);
      reset_n = 1'b1;
      start = 1'b0;
      repeat (1000) begin
         @(negedge clk);
         op_set = get_op();
         A = get_data();
         B = get_data();
         start = 1'b1;
         case (op_set) // handle the start signal
           no_op: begin
              @(posedge clk);
              start = 1'b0;
           end
           rst_op: begin
              reset_n = 1'b0;
              start = 1'b0;
              @(negedge clk);
              reset_n = 1'b1;
           end
           default: begin
              // Wait for an EDGE with done up, not for done itself: on the
              // one-cycle ops done is a level still up from the previous op,
              // and a plain wait(done) drops this one without a sound.
              enviadas++;
              do @(negedge clk); while (done == 0);
              start = 1'b0;
           end
         endcase // case (op_set)
      end
      $finish;
   end : tester

   // The testbench checks itself: every operation sent has to come back
   // through the scoreboard. A tester that drops operations -- while the
   // coverage counts them anyway -- passes green and has measured nothing.
   final begin
      if (enviadas != chequeadas)
        $error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
      else
        $display ("%0d operations sent, %0d checked", enviadas, chequeadas);
   end
endmodule : top

The reference model is four lines of case because the DUT is an ALU. On a real project it can be a model in C or the RTL of the previous generation, but the shape is always this one: predict with something that is not the DUT, and compare. That the scoreboard reads A, B and op_set off the wire at the moment of the result works only because the protocol forces them to stay stable. It is fragile, and in the analysis ports it will be replaced by a queue of commands the monitor sends. And the $error here is going to be a `uvm_error from reporting on: same concept, with a counter and a severity policy behind it.

see this slide in the deck ›

The conventional testbench

How do you know the scoreboard checks anything?

code/u2/convencional/mutante.txt
FAILED: A: 15  B: ce  op: xor_op result: da ovf: 0

Run it live, it takes seconds because there is no UVM: first bash run.sh —a thousand operations, silence, and the coverage summary—, then VTALU_BUG=1 bash run.sh, which ends in the line on the slide and a $stop. What has to be said in so many words is what that line proves and what it does not. It proves the scoreboard compares the bit the bug touched. It does not prove it compares ovf: a bug that only touched ovf is another run, and that is why the plan has a row for each output. This is the short answer to the first slide of the day: the regression that said PASS and sent the bug to the fab had never seen a FAILED. It did not know whether it was checking. A PASS says nothing until you know what FAILED would have said. And it is not a day-1 trick: it is the practice that runs through the course. +BUG=1 in the day 7 capstone and +GOLDEN_BUG in the day 8 reference model are this very thing, and the exercise checkers demand it.

see this slide in the deck ›

The conventional testbench

Unit summary

The last bullet is the one that orders the rest of the day: this testbench works, it is proven, and it is badly divided. The tester knows how start is wiggled and the scoreboard knows when to read done: the protocol lives in two places of the same file, and the day it changes both have to be touched. That is the whole motivation for interfaces and BFM, and it is worth leaving as an open question instead of answering it here. Before that, the next section fills in the third column.

see this slide in the deck ›

Functional coverage

When are you done verifying?

The question on the slide is the one a verification engineer gets asked in the tape-out meeting, and it is not answered with "I ran a lot of tests". If somebody asks why code coverage is not enough: because it measures the DUT, not the spec. A DUT that is missing a whole feature can give 100 % of lines — the lines nobody wrote do not show up in the report.

see this slide in the deck ›

Functional coverage

From the verification plan to the covergroup

The six rows of the plan from the previous section, translated:

From the plan How it gets measured
the six operations coverpoint op_set with one bin per operation
operands at 00 and at FF cross of A, B and the operation
operate after a reset transition bin rst_op => op
a mult after a single-cycle one transition bin
the same operation twice in a row repetition bin [* 2]
subtract too much: A < B, and ovf goes up coverpoint borrow with the hubo_borrow bin

It is worth going back to the plan of the previous section and reading it out loud before showing this table: the point is that there is not one item of the plan left untranslated. The order matters — the plan first, the covergroup after. The other way round you end up measuring what the testbench does, not what the spec asks for.

see this slide in the deck ›

Functional coverage

Anatomy: it is three steps, not one

covergroup op_cov;                     // 1. WHAT gets measured
   coverpoint op_set;
endgroup

op_cov oc;                             // 2. it gets INSTANTIATED: it is a type, like a class
initial oc = new();

always @(posedge clk) oc.sample();     // 3. WHEN it gets counted

Everybody's first-time mistake is declaring the covergroup and forgetting the new() or the sample(). There is no warning: the report simply says 0 % and you spend half an hour looking for the bug in the bins. The third point is the one that holds for the rest of the course: the moment of the sample defines what you are measuring. Sampling on the clock counts cycles; sampling on transactions counts operations. In the analysis ports the coverage becomes a subscriber and samples when a command arrives — which is the right thing. And the edge is not a whim: sampling on the same negedge the tester writes op_set on is a race the LRM does not define —Verilator always resolves it the same way; another simulator may not—. It is the same rule as the day 7 assertions: you read on the opposite edge to the one that writes.

see this slide in the deck ›

Functional coverage

bins: the buckets that have to be filled

coverpoint op_set;                   // automatic bins: one per value of the enum

coverpoint op_set {
   bins single_cycle[] = {[add_op : xor_op], rst_op, no_op};  // [] -> one per value
   bins multi_cycle    = {mul_op};                            // no [] -> a single one
}

The brackets are the detail most often copied wrong, and it is worth showing it with both cases side by side: bins x[] = {[0:3]} is four buckets, and bins x = {[0:3]} is a single one that gets filled by any of the four values landing. The syntax differs by two characters and the number the tool reports changes completely. The practical consequence has to be said loudly because it is a silent trap: a bin without [] over a large range always gives 100 %. You covered one value out of a thousand and the tool tells you it is done. Nobody warns you. And the last bullet is the warning that orders what comes next: a bin that cannot be filled is not a tool problem, it is a decision somebody did not write down. The way to write it down is called ignore_bins, and it has its own slide in this section.

see this slide in the deck ›

Functional coverage

The VTALU covergroup

code/u2/convencional/vtalu_tb.sv · #op_cov
   //op_cov: make sure every operation, and every interaction between them, is covered
   covergroup op_cov;
      coverpoint op_set {
         bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
         bins multi_cycle = {mul_op};

         // => are transitional coverage. From => To
         // Transition bins. Verilator 5.052 does NOT implement them yet: any form
         // with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
         // code/verilator/repro-cg-transition.sv. They stay because they are part
         // of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
         bins opn_rst[] = ([add_op:mul_op] => rst_op);
         bins rst_opn[] = (rst_op => [add_op:mul_op]);

         bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
         bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);

         bins twoops[] = ([add_op:mul_op] [* 2]);
         bins manymult = (mul_op [* 3:5]);
`endif
      }
   endgroup
el archivo entero, 256 líneas
// Typical TB:
// ----------
//  * Functional Coverage
//  * Self Checking
//  * Constrained Random Stimulus

// Why is this bad?
//  * No modularity (everything in a single file)
//  * Almost nothing here can be reused
module top;

   typedef enum bit[2:0] {no_op  = 3'b000,
                          add_op = 3'b001,
                          sub_op = 3'b010,
                          and_op = 3'b011,
                          xor_op = 3'b100,
                          mul_op = 3'b101,
                          rst_op = 3'b111} operation_t;
   byte         unsigned        A;
   byte         unsigned        B;
   bit          clk;
   bit          reset_n;
   wire [2:0]   op;
   bit          start;
   wire         done;
   wire [15:0]  result;
   wire         ovf;
   operation_t  op_set;

   assign op = op_set;

   vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);

`ifdef VLT_TRACE
   // Waves for GTKWave:  VLT_TRACE=1 bash run.sh  &&  gtkwave vtalu.vcd
   // Behind an `ifdef because $dumpvars does not compile without --trace, and
   // the define comes from the same VLT_TRACE that adds the flag (common.sh).
   initial begin
      $dumpfile("vtalu.vcd");
      $dumpvars;
   end
`endif

   //op_cov: make sure every operation, and every interaction between them, is covered
   covergroup op_cov;
      coverpoint op_set {
         bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
         bins multi_cycle = {mul_op};

         // => are transitional coverage. From => To
         // Transition bins. Verilator 5.052 does NOT implement them yet: any form
         // with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
         // code/verilator/repro-cg-transition.sv. They stay because they are part
         // of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
         bins opn_rst[] = ([add_op:mul_op] => rst_op);
         bins rst_opn[] = (rst_op => [add_op:mul_op]);

         bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
         bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);

         bins twoops[] = ([add_op:mul_op] [* 2]);
         bins manymult = (mul_op [* 3:5]);
`endif
      }
   endgroup

   covergroup zeros_or_ones_on_ops;
      // Check the corners where the inputs are all 0s or all 1s
      all_ops : coverpoint op_set {
         ignore_bins null_ops = {rst_op, no_op};}

      a_leg: coverpoint A {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }

      b_leg: coverpoint B {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }
      // VTALU rev2: the borrow of the subtraction, the row of the plan that the
      // scoreboard can only close by looking at TWO outputs. Without this bin, a
      // subtraction that never went negative looks exactly like one that did.
      borrow: coverpoint ((op_set == sub_op) && (A < B)) {
         bins hubo_borrow = {1};
         ignore_bins resto = {0};
      }

      op_00_FF:  cross a_leg, b_leg, all_ops {
         bins add_00 = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins add_FF = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins sub_00 = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins sub_FF = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins and_00 = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins and_FF = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins xor_00 = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins xor_FF = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_00 = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins mul_FF = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_max = binsof (all_ops) intersect {mul_op} &&
                        (binsof (a_leg.ones) && binsof (b_leg.ones));

         ignore_bins others_only =
                         binsof(a_leg.others) && binsof(b_leg.others);
      }
   endgroup

   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end

   op_cov oc;
   zeros_or_ones_on_ops c_00_FF;

   initial begin : coverage
      oc = new();
      c_00_FF = new();

      forever begin @(posedge clk);
         oc.sample();
         c_00_FF.sample();
      end
   end : coverage

   function operation_t get_op();
      bit [2:0] op_choice;
      op_choice = $random;
      case (op_choice)
        3'b000 : return no_op;
        3'b001 : return add_op;
        3'b010 : return sub_op;
        3'b011 : return and_op;
        3'b100 : return xor_op;
        3'b101 : return mul_op;
        3'b110 : return rst_op;
        3'b111 : return rst_op;
      endcase // case (op_choice)
   endfunction : get_op

   function byte get_data();
      // This is how the random gets biased
      bit [1:0] zero_ones;
      zero_ones = $random;
      if (zero_ones == 2'b00)
        return 8'h00;
      else if (zero_ones == 2'b11)
        return 8'hFF;
      else
        return $random;
   endfunction : get_data

   // Sent by the tester, compared by the scoreboard: the final block below
   // demands that the two match.
   int enviadas, chequeadas;

   // Scoreboard loop: checks that everything came out right
   always @(posedge done) begin : scoreboard
      shortint predicted_result;
      bit      predicted_ovf;
      #1;
      case (op_set)
        add_op: predicted_result = A + B;
        sub_op: predicted_result = A - B;
        and_op: predicted_result = A & B;
        xor_op: predicted_result = A ^ B;
        mul_op: predicted_result = A * B;
      endcase // case (op_set)

      // ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
      // output, neither the addition nor the multiplication can overflow.
      predicted_ovf = (op_set == sub_op) && (A < B);

      if ((op_set != no_op) && (op_set != rst_op)) begin
        chequeadas++;
        if (predicted_result != result || predicted_ovf != ovf)
          $error ("FAILED: A: %0h  B: %0h  op: %s result: %0h ovf: %0b",
                  A, B, op_set.name(), result, ovf);
      end
   end : scoreboard

   // Randomize the stimulus
   // get_op and get_data are Constrained Random Data
   // The point of CRD is to build values that are random but legal
   initial begin : tester
      reset_n = 1'b0;
      @(negedge clk);
      @(negedge clk);
      reset_n = 1'b1;
      start = 1'b0;
      repeat (1000) begin
         @(negedge clk);
         op_set = get_op();
         A = get_data();
         B = get_data();
         start = 1'b1;
         case (op_set) // handle the start signal
           no_op: begin
              @(posedge clk);
              start = 1'b0;
           end
           rst_op: begin
              reset_n = 1'b0;
              start = 1'b0;
              @(negedge clk);
              reset_n = 1'b1;
           end
           default: begin
              // Wait for an EDGE with done up, not for done itself: on the
              // one-cycle ops done is a level still up from the previous op,
              // and a plain wait(done) drops this one without a sound.
              enviadas++;
              do @(negedge clk); while (done == 0);
              start = 1'b0;
           end
         endcase // case (op_set)
      end
      $finish;
   end : tester

   // The testbench checks itself: every operation sent has to come back
   // through the scoreboard. A tester that drops operations -- while the
   // coverage counts them anyway -- passes green and has measured nothing.
   final begin
      if (enviadas != chequeadas)
        $error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
      else
        $display ("%0d operations sent, %0d checked", enviadas, chequeadas);
   end
endmodule : top

This is the first real covergroup of the course, and it is worth reading it top to bottom answering the three questions of the previous slide: what gets measured (the coverpoints), into which buckets (the bins) and when it gets counted (the sample(), which is further down in the same file). That multi_cycle is a single bin and single_cycle[] is six is not broken symmetry: it is the verification plan. The six that are not the multiplication matter one by one; the multiplication matters as a separate case because it is the only one that takes more than one cycle. The shape of the covergroup is the table of the plan, and that is why the plan gets written first. The `ifndef VERILATOR is worth naming now and not hiding: those bins are a topic of the course, they are written, and today Verilator does not compile them. It is in docs/en/verilator.md with the version and the date. A course that covers up what its tool does not do is a course that lies.

see this slide in the deck ›

Functional coverage

ignore_bins: saying out loud what does not get covered

code/u2/convencional/vtalu_tb.sv · #legs-and-ops
      // Check the corners where the inputs are all 0s or all 1s
      all_ops : coverpoint op_set {
         ignore_bins null_ops = {rst_op, no_op};}

      a_leg: coverpoint A {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }

      b_leg: coverpoint B {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }
      // VTALU rev2: the borrow of the subtraction, the row of the plan that the
      // scoreboard can only close by looking at TWO outputs. Without this bin, a
      // subtraction that never went negative looks exactly like one that did.
      borrow: coverpoint ((op_set == sub_op) && (A < B)) {
         bins hubo_borrow = {1};
         ignore_bins resto = {0};
      }
el archivo entero, 256 líneas
// Typical TB:
// ----------
//  * Functional Coverage
//  * Self Checking
//  * Constrained Random Stimulus

// Why is this bad?
//  * No modularity (everything in a single file)
//  * Almost nothing here can be reused
module top;

   typedef enum bit[2:0] {no_op  = 3'b000,
                          add_op = 3'b001,
                          sub_op = 3'b010,
                          and_op = 3'b011,
                          xor_op = 3'b100,
                          mul_op = 3'b101,
                          rst_op = 3'b111} operation_t;
   byte         unsigned        A;
   byte         unsigned        B;
   bit          clk;
   bit          reset_n;
   wire [2:0]   op;
   bit          start;
   wire         done;
   wire [15:0]  result;
   wire         ovf;
   operation_t  op_set;

   assign op = op_set;

   vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);

`ifdef VLT_TRACE
   // Waves for GTKWave:  VLT_TRACE=1 bash run.sh  &&  gtkwave vtalu.vcd
   // Behind an `ifdef because $dumpvars does not compile without --trace, and
   // the define comes from the same VLT_TRACE that adds the flag (common.sh).
   initial begin
      $dumpfile("vtalu.vcd");
      $dumpvars;
   end
`endif

   //op_cov: make sure every operation, and every interaction between them, is covered
   covergroup op_cov;
      coverpoint op_set {
         bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
         bins multi_cycle = {mul_op};

         // => are transitional coverage. From => To
         // Transition bins. Verilator 5.052 does NOT implement them yet: any form
         // with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
         // code/verilator/repro-cg-transition.sv. They stay because they are part
         // of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
         bins opn_rst[] = ([add_op:mul_op] => rst_op);
         bins rst_opn[] = (rst_op => [add_op:mul_op]);

         bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
         bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);

         bins twoops[] = ([add_op:mul_op] [* 2]);
         bins manymult = (mul_op [* 3:5]);
`endif
      }
   endgroup

   covergroup zeros_or_ones_on_ops;
      // Check the corners where the inputs are all 0s or all 1s
      all_ops : coverpoint op_set {
         ignore_bins null_ops = {rst_op, no_op};}

      a_leg: coverpoint A {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }

      b_leg: coverpoint B {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }
      // VTALU rev2: the borrow of the subtraction, the row of the plan that the
      // scoreboard can only close by looking at TWO outputs. Without this bin, a
      // subtraction that never went negative looks exactly like one that did.
      borrow: coverpoint ((op_set == sub_op) && (A < B)) {
         bins hubo_borrow = {1};
         ignore_bins resto = {0};
      }

      op_00_FF:  cross a_leg, b_leg, all_ops {
         bins add_00 = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins add_FF = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins sub_00 = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins sub_FF = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins and_00 = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins and_FF = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins xor_00 = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins xor_FF = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_00 = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins mul_FF = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_max = binsof (all_ops) intersect {mul_op} &&
                        (binsof (a_leg.ones) && binsof (b_leg.ones));

         ignore_bins others_only =
                         binsof(a_leg.others) && binsof(b_leg.others);
      }
   endgroup

   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end

   op_cov oc;
   zeros_or_ones_on_ops c_00_FF;

   initial begin : coverage
      oc = new();
      c_00_FF = new();

      forever begin @(posedge clk);
         oc.sample();
         c_00_FF.sample();
      end
   end : coverage

   function operation_t get_op();
      bit [2:0] op_choice;
      op_choice = $random;
      case (op_choice)
        3'b000 : return no_op;
        3'b001 : return add_op;
        3'b010 : return sub_op;
        3'b011 : return and_op;
        3'b100 : return xor_op;
        3'b101 : return mul_op;
        3'b110 : return rst_op;
        3'b111 : return rst_op;
      endcase // case (op_choice)
   endfunction : get_op

   function byte get_data();
      // This is how the random gets biased
      bit [1:0] zero_ones;
      zero_ones = $random;
      if (zero_ones == 2'b00)
        return 8'h00;
      else if (zero_ones == 2'b11)
        return 8'hFF;
      else
        return $random;
   endfunction : get_data

   // Sent by the tester, compared by the scoreboard: the final block below
   // demands that the two match.
   int enviadas, chequeadas;

   // Scoreboard loop: checks that everything came out right
   always @(posedge done) begin : scoreboard
      shortint predicted_result;
      bit      predicted_ovf;
      #1;
      case (op_set)
        add_op: predicted_result = A + B;
        sub_op: predicted_result = A - B;
        and_op: predicted_result = A & B;
        xor_op: predicted_result = A ^ B;
        mul_op: predicted_result = A * B;
      endcase // case (op_set)

      // ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
      // output, neither the addition nor the multiplication can overflow.
      predicted_ovf = (op_set == sub_op) && (A < B);

      if ((op_set != no_op) && (op_set != rst_op)) begin
        chequeadas++;
        if (predicted_result != result || predicted_ovf != ovf)
          $error ("FAILED: A: %0h  B: %0h  op: %s result: %0h ovf: %0b",
                  A, B, op_set.name(), result, ovf);
      end
   end : scoreboard

   // Randomize the stimulus
   // get_op and get_data are Constrained Random Data
   // The point of CRD is to build values that are random but legal
   initial begin : tester
      reset_n = 1'b0;
      @(negedge clk);
      @(negedge clk);
      reset_n = 1'b1;
      start = 1'b0;
      repeat (1000) begin
         @(negedge clk);
         op_set = get_op();
         A = get_data();
         B = get_data();
         start = 1'b1;
         case (op_set) // handle the start signal
           no_op: begin
              @(posedge clk);
              start = 1'b0;
           end
           rst_op: begin
              reset_n = 1'b0;
              start = 1'b0;
              @(negedge clk);
              reset_n = 1'b1;
           end
           default: begin
              // Wait for an EDGE with done up, not for done itself: on the
              // one-cycle ops done is a level still up from the previous op,
              // and a plain wait(done) drops this one without a sound.
              enviadas++;
              do @(negedge clk); while (done == 0);
              start = 1'b0;
           end
         endcase // case (op_set)
      end
      $finish;
   end : tester

   // The testbench checks itself: every operation sent has to come back
   // through the scoreboard. A tester that drops operations -- while the
   // coverage counts them anyway -- passes green and has measured nothing.
   final begin
      if (enviadas != chequeadas)
        $error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
      else
        $display ("%0d operations sent, %0d checked", enviadas, chequeadas);
   end
endmodule : top

There is a relative worth naming: illegal_bins. ignore_bins takes the bin out of the count; illegal_bins takes it out of the count and also reports an error if it ever happens. It is for the cases the spec forbids — a reserved opcode, an illegal handshake. The rule of thumb: if it cannot be covered, ignore_bins; if it must not happen, illegal_bins; if it simply has not happened yet, leave it and write the test.

see this slide in the deck ›

Functional coverage

cross: the combinations, and how not to drown

op_00_FF : cross a_leg, b_leg, all_ops;    // 3 x 3 x 5 = 45 bins, almost all meaningless

The arithmetic in the comment is what has to be done out loud, because the number is scary and it has to be: three bins of A by three of B by five operations is 45, and out of those 45 the plan asked for eleven. The other 34 are not wrong — they are surplus, and a surplus bin is as expensive as a missing one: it pins your coverage below 100 % and nobody knows whether it is a real hole. binsof(x) intersect {v} is worth reading in plain words before writing it, because the syntax does not help: "the bins of x that contain v". Then they combine with && and || like any condition. The scale mistake to anticipate: on a real DUT a cross of three coverpoints with automatic bins is thousands of buckets, and a regression never fills them. When somebody says "the coverage is not going up", half the time the problem is an unfiltered cross, not a missing test. And the number to keep at hand for the "Reading the number" slide: the 45 are the LRM count, one bin per enum member. Verilator measures 54, because it hands out the automatic bins by the base type (bit [2:0], 8 values minus the 2 of ignore_bins, that is 3 x 3 x 6). The 9 extra are the 3'b110 crossed, and they are exactly the 9 that stay at zero.

see this slide in the deck ›

Functional coverage

The VTALU cross

code/u2/convencional/vtalu_tb.sv · #cross-mul-bins
         bins mul_00 = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins mul_FF = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_max = binsof (all_ops) intersect {mul_op} &&
                        (binsof (a_leg.ones) && binsof (b_leg.ones));

         ignore_bins others_only =
                         binsof(a_leg.others) && binsof(b_leg.others);
el archivo entero, 256 líneas
// Typical TB:
// ----------
//  * Functional Coverage
//  * Self Checking
//  * Constrained Random Stimulus

// Why is this bad?
//  * No modularity (everything in a single file)
//  * Almost nothing here can be reused
module top;

   typedef enum bit[2:0] {no_op  = 3'b000,
                          add_op = 3'b001,
                          sub_op = 3'b010,
                          and_op = 3'b011,
                          xor_op = 3'b100,
                          mul_op = 3'b101,
                          rst_op = 3'b111} operation_t;
   byte         unsigned        A;
   byte         unsigned        B;
   bit          clk;
   bit          reset_n;
   wire [2:0]   op;
   bit          start;
   wire         done;
   wire [15:0]  result;
   wire         ovf;
   operation_t  op_set;

   assign op = op_set;

   vtalu DUT (.A, .B, .clk, .op, .reset_n, .start, .done, .ovf, .result);

`ifdef VLT_TRACE
   // Waves for GTKWave:  VLT_TRACE=1 bash run.sh  &&  gtkwave vtalu.vcd
   // Behind an `ifdef because $dumpvars does not compile without --trace, and
   // the define comes from the same VLT_TRACE that adds the flag (common.sh).
   initial begin
      $dumpfile("vtalu.vcd");
      $dumpvars;
   end
`endif

   //op_cov: make sure every operation, and every interaction between them, is covered
   covergroup op_cov;
      coverpoint op_set {
         bins single_cycle[] = {[add_op : xor_op], rst_op,no_op};
         bins multi_cycle = {mul_op};

         // => are transitional coverage. From => To
         // Transition bins. Verilator 5.052 does NOT implement them yet: any form
         // with "=>" or "[* n]" aborts with an Internal Error. Minimal repro in
         // code/verilator/repro-cg-transition.sv. They stay because they are part
         // of the topic; when Verilator supports them, drop the ifndef, nothing else.
`ifndef VERILATOR
         bins opn_rst[] = ([add_op:mul_op] => rst_op);
         bins rst_opn[] = (rst_op => [add_op:mul_op]);

         bins sngl_mul[] = ([add_op:xor_op],no_op => mul_op);
         bins mul_sngl[] = (mul_op => [add_op:xor_op], no_op);

         bins twoops[] = ([add_op:mul_op] [* 2]);
         bins manymult = (mul_op [* 3:5]);
`endif
      }
   endgroup

   covergroup zeros_or_ones_on_ops;
      // Check the corners where the inputs are all 0s or all 1s
      all_ops : coverpoint op_set {
         ignore_bins null_ops = {rst_op, no_op};}

      a_leg: coverpoint A {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }

      b_leg: coverpoint B {
         bins zeros = {8'h00};
         bins others= {[8'h01:8'hFE]};
         bins ones  = {8'hFF};
      }
      // VTALU rev2: the borrow of the subtraction, the row of the plan that the
      // scoreboard can only close by looking at TWO outputs. Without this bin, a
      // subtraction that never went negative looks exactly like one that did.
      borrow: coverpoint ((op_set == sub_op) && (A < B)) {
         bins hubo_borrow = {1};
         ignore_bins resto = {0};
      }

      op_00_FF:  cross a_leg, b_leg, all_ops {
         bins add_00 = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins add_FF = binsof (all_ops) intersect {add_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins sub_00 = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins sub_FF = binsof (all_ops) intersect {sub_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins and_00 = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins and_FF = binsof (all_ops) intersect {and_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins xor_00 = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins xor_FF = binsof (all_ops) intersect {xor_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_00 = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.zeros) || binsof (b_leg.zeros));

         bins mul_FF = binsof (all_ops) intersect {mul_op} &&
                       (binsof (a_leg.ones) || binsof (b_leg.ones));

         bins mul_max = binsof (all_ops) intersect {mul_op} &&
                        (binsof (a_leg.ones) && binsof (b_leg.ones));

         ignore_bins others_only =
                         binsof(a_leg.others) && binsof(b_leg.others);
      }
   endgroup

   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end

   op_cov oc;
   zeros_or_ones_on_ops c_00_FF;

   initial begin : coverage
      oc = new();
      c_00_FF = new();

      forever begin @(posedge clk);
         oc.sample();
         c_00_FF.sample();
      end
   end : coverage

   function operation_t get_op();
      bit [2:0] op_choice;
      op_choice = $random;
      case (op_choice)
        3'b000 : return no_op;
        3'b001 : return add_op;
        3'b010 : return sub_op;
        3'b011 : return and_op;
        3'b100 : return xor_op;
        3'b101 : return mul_op;
        3'b110 : return rst_op;
        3'b111 : return rst_op;
      endcase // case (op_choice)
   endfunction : get_op

   function byte get_data();
      // This is how the random gets biased
      bit [1:0] zero_ones;
      zero_ones = $random;
      if (zero_ones == 2'b00)
        return 8'h00;
      else if (zero_ones == 2'b11)
        return 8'hFF;
      else
        return $random;
   endfunction : get_data

   // Sent by the tester, compared by the scoreboard: the final block below
   // demands that the two match.
   int enviadas, chequeadas;

   // Scoreboard loop: checks that everything came out right
   always @(posedge done) begin : scoreboard
      shortint predicted_result;
      bit      predicted_ovf;
      #1;
      case (op_set)
        add_op: predicted_result = A + B;
        sub_op: predicted_result = A - B;
        and_op: predicted_result = A & B;
        xor_op: predicted_result = A ^ B;
        mul_op: predicted_result = A * B;
      endcase // case (op_set)

      // ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
      // output, neither the addition nor the multiplication can overflow.
      predicted_ovf = (op_set == sub_op) && (A < B);

      if ((op_set != no_op) && (op_set != rst_op)) begin
        chequeadas++;
        if (predicted_result != result || predicted_ovf != ovf)
          $error ("FAILED: A: %0h  B: %0h  op: %s result: %0h ovf: %0b",
                  A, B, op_set.name(), result, ovf);
      end
   end : scoreboard

   // Randomize the stimulus
   // get_op and get_data are Constrained Random Data
   // The point of CRD is to build values that are random but legal
   initial begin : tester
      reset_n = 1'b0;
      @(negedge clk);
      @(negedge clk);
      reset_n = 1'b1;
      start = 1'b0;
      repeat (1000) begin
         @(negedge clk);
         op_set = get_op();
         A = get_data();
         B = get_data();
         start = 1'b1;
         case (op_set) // handle the start signal
           no_op: begin
              @(posedge clk);
              start = 1'b0;
           end
           rst_op: begin
              reset_n = 1'b0;
              start = 1'b0;
              @(negedge clk);
              reset_n = 1'b1;
           end
           default: begin
              // Wait for an EDGE with done up, not for done itself: on the
              // one-cycle ops done is a level still up from the previous op,
              // and a plain wait(done) drops this one without a sound.
              enviadas++;
              do @(negedge clk); while (done == 0);
              start = 1'b0;
           end
         endcase // case (op_set)
      end
      $finish;
   end : tester

   // The testbench checks itself: every operation sent has to come back
   // through the scoreboard. A tester that drops operations -- while the
   // coverage counts them anyway -- passes green and has measured nothing.
   final begin
      if (enviadas != chequeadas)
        $error ("sent %0d operations, the scoreboard checked %0d", enviadas, chequeadas);
      else
        $display ("%0d operations sent, %0d checked", enviadas, chequeadas);
   end
endmodule : top

Worth reading mul_00 and mul_max in plain words, one after the other, because the difference between || and && is the one that gets copied wrong, and here both bins belong to the same operation: "a multiplication where A or B is 0x00" against "a multiplication with A and B at 0xFF". The first is two cases, the second is a single one — the maximum product, the top corner of the input space. And here it is worth killing the misunderstanding that comes on its own, because it is expensive: FF × FF does not overflow anything. It gives FE01, which fits exactly in the 16 bits of result —8 bits by 8 never go past 16— and that is why the DUT forces ovf to 0 in every multiplication: assign ovf = es_mult ? 1'b0 : ovf_1c;. The bin is called mul_max and not mul_ovf for precisely that reason: the case is worth something for being the maximum of the input space, not for overflowing. The only operation that overflows is the subtraction, and only when A < B. The honesty of this slide is part of the course, so it is worth saying and not rushing past: the 86.8 % the example reports is not the number Questa would give. Verilator ignores the filter and measures the whole cross, and on top it adds a value the enum does not have: where the LRM counts 45, Verilator counts 54 (the 9 extra are the 3'b110 crossed, and they are the 9 that stay at zero). That is why the percentage comes out lower, and for a reason that is not the testbench's. It is written, dated and with a version in docs/en/verilator.md. The useful question for the classroom: does that invalidate the exercise? No. What gets learned —writing the cross, filtering it, and knowing which bin belongs to which row of the plan— is identical. The only thing that cannot be done with this tool is signing off on coverage closure, and that is not the goal of the course.

see this slide in the deck ›

Functional coverage

Transition bins: when the order matters

bins opn_rst[] = ([add_op:mul_op] => rst_op);   // A and then B
bins sngl_mul[] = ([add_op:xor_op], no_op => mul_op);
bins twoops[]  = ([add_op:mul_op] [* 2]);       // twice in a row
bins manymult  = (mul_op [* 3:5]);              // between 3 and 5 times in a row

It is the only place in the course where the free tool falls short, and it is worth saying head on instead of hiding it: the concept belongs to the language, not to the simulator, and on Questa or VCS these bins run. There are two more cousins worth naming: [-> n] (goto, n times not necessarily in a row) and [= n] (non-consecutive). They are in coverage.svh of the analysis ports.

see this slide in the deck ›

Functional coverage

The knobs: when a bin counts as covered

coverpoint op_set {
   option.at_least = 10;                  // a bin only counts at 10 hits
   bins single[] = {[add_op:xor_op]};
}
coverpoint A { option.auto_bin_max = 8; } // 8 automatic buckets, not 64

This is the slide that puts an asterisk on every percentage of the unit, and that is why it only arrives now: at_least = 1 means the tool tells you covered with a single hit. For a bin that represents a value of an enum that is perfect —either it happened or it did not—. For the bin that represents the maximum product of the multiplier, one hit is an anecdote, not a verification. The field rule, and it is worth giving because the question comes on its own: at_least high on the bins that represent a rare case, default on the ones that represent a value. Raising it for the whole covergroup is not rigour, it is a regression that never closes and a number nobody looks at. auto_bin_max is the flip side of the bracket trap, the one on the bins slide. There the problem was a surplus bin; here it is the other way round: if the coverpoint is an int and you write it no bins, the tool invents 64 ranges that do not correspond to any row of the plan. The number that comes out is real and means nothing. And the usual honesty: the four knobs are the LRM's, not Verilator's. What this tool does with each one is measured, dated and with a repro in code/verilator/repro-cg-options.sv. The at_least one is the worst of the four because it is silent: in the covergroup you write it, it compiles, and it does nothing.

see this slide in the deck ›

Functional coverage

Reading the number

$ make u2/convencional
Coverage Summary:
  covergroup : 86.8% (66/76)

This is where it is worth running it live and opening the coverage.dat: the student has to see that the ten zeros have the same name, and that the name is not in the operation_t. The lesson is to read the report and not the number: the number says 86.8 % and the report says "nothing is missing, the tool invented a bucket". On a commercial tool the automatic bin of an enum is one per member (IEEE 1800, 19.5) and this gives 100 % from the start; it is noted in docs/en/verilator.md. And the punchline: the exercise of the day uses precisely 3'b110 for the shift, so the new operation fills the phantom bucket and the coverage goes from 86.8 % to 100 % — 77 out of 77, with the shift's own bin added. The one who raises it is the same one who wrote the test, and the checker counts bins and not the percentage, because we have just seen the percentage cannot read.

see this slide in the deck ›

Functional coverage

The verification plan, whole and in one table

# Feature Scenario Stimulus Check Measure
1 ALU the six operations random scoreboard coverpoint op_set
2 ALU operands at 00 and at FF dist biased to the edges scoreboard cross op_00_FF
3 mult maximum product: FF × FF directed case scoreboard, 16 bits bin mul_max
4 reset operate after a reset rst_op interleaved scoreboard bin rst_op => op
5 mult a mult after a single-cycle one random scoreboard transition bin
6 ALU the same operation twice in a row random scoreboard bin [* 2]
7 sub subtract too much: A < B, and ovf goes up random scoreboard, two outputs bin hubo_borrow
8 sub A == B: the result is 0 and ovf does not go up random scoreboard, two outputs bin sub_00/sub_FF
9 ovf ovf does not go up for any other operation random assertion c_ovf
10 protocol the operands are not touched with start up random assertion cover property
11 protocol done arrives, and before 5 cycles random assertion cover property
12 protocol no_op is the only one that does not answer random assertion cover property

This is the slide that answers "and what is this for?" for the rest of day 1, and it is worth saying head on why it only arrives now: the three right-hand columns are stimulus, self-checking and coverage — the three parts the previous unit showed loose. The plan is the table that makes them one thing. The row that gets the most discussion is the maximum-product one, and that is good: FF × FF is the case a short test does not fill, and that is why its stimulus column says directed case: the plan is decided before knowing how long the random will run. There you see that the plan does not only measure, it also decides which test has to be written. It is the d5c exercise, the last one of the day 5 afternoon, with this same row. The three protocol rows are worth naming and moving on: they are the half the course does not touch until day 7, and they serve to plant the idea that a scoreboard does not check everything. A serious verification plan has both columns. And what the student takes home: docs/en/verification-plan.md is the same table as an artifact, with an empty template. The day 7 capstone is handed in with the plan filled — which is exactly what gets handed in on a project.

see this slide in the deck ›

Functional coverage

Summary of the unit

Closing of the most misunderstood concept of the course. The sentence to take home: coverage does not tell you the design is right, it tells you that you tested it. A DUT that is missing a whole feature gives 100 % of code coverage and 0 % of the functional coverage nobody wrote. And the operational mistake that repeats the most, to leave it burned in: the missing sample(). It compiles, it runs, and the report reads zero.

see this slide in the deck ›

Interfaces and BFM

First: the signals stop being loose

code/u2/interfaces-bfm/vtalu_bfm.sv · #signals-and-clock
interface vtalu_bfm;
   // brings in operation_t. e.g. (no_op  = 3'b000)
   import vtalu_pkg::*;

   byte unsigned        A;
   byte unsigned        B;
   bit                  clk;
   bit                  reset_n;
   wire          [ 2:0] op;
   bit                  start;
   wire                 done;
   wire          [15:0] result;
   wire                 ovf;
   operation_t          op_set;

   assign op = op_set;

   // The BFM is a first step towards modularity
   // It takes care of driving every low-level signal
   // and keeps the protocol in one single place

   // DUT clock
   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end
el archivo entero, 71 líneas
interface vtalu_bfm;
   // brings in operation_t. e.g. (no_op  = 3'b000)
   import vtalu_pkg::*;

   byte unsigned        A;
   byte unsigned        B;
   bit                  clk;
   bit                  reset_n;
   wire          [ 2:0] op;
   bit                  start;
   wire                 done;
   wire          [15:0] result;
   wire                 ovf;
   operation_t          op_set;

   assign op = op_set;

   // The BFM is a first step towards modularity
   // It takes care of driving every low-level signal
   // and keeps the protocol in one single place

   // DUT clock
   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end

   // DUT reset
   task reset_alu();
      reset_n = 1'b0;
      @(negedge clk);
      @(negedge clk);
      reset_n = 1'b1;
      start = 1'b0;
   endtask : reset_alu

   // The protocol handling can now be changed in one single place.
   // Fix it here once, and the fix propagates through the rest of the code
   task send_op(input byte iA, input byte iB, input operation_t iop,
                output shortint alu_result);

      op_set = iop;

      if (iop == rst_op) begin
         @(posedge clk);
         reset_n = 1'b0;
         start = 1'b0;
         @(posedge clk);
         #1;
         reset_n = 1'b1;
      end else begin
         @(negedge clk);
         A = iA;
         B = iB;
         start = 1'b1;
         if (iop == no_op) begin
            @(posedge clk);
            #1;
            start = 1'b0;
         end else begin
            do @(negedge clk); while (done == 0);
            start = 1'b0;
         end
      end

   endtask : send_op

endinterface : vtalu_bfm

First step towards UVM and it does not have a single line of UVM. The immediate gain is a maintenance one and it is worth measuring: with loose signals, splitting the testbench into modules forces every one of them through every port list, and a new signal gets wired in all of them. Here it gets declared in the interface, and the only port that changes is the DUT's. One detail of types that costs you: op is a wire [2:0] and op_set is the operation_t. The assign op = op_set is the bridge between the testbench enum and the DUT wires. It is the only place in the course where the seam between the two worlds is visible. And the modport, which does not show up in the rest of the course and does in anybody else's RTL: it is a view of the interface with the directions declared —modport dut (input A, input B, output result)—, and the module asks for it in its port list (vtalu_bfm.dut bus). It is good for two things, and both are worth it: it lets the compiler stop the DUT if it writes a signal that was an input as far as it is concerned, and it lets whoever reads the interface know who drives what without opening the RTL. It is not here because the BFM belongs to the testbench and drives everything; at work the interface almost always comes with two or three modports and what you need to know is which one to ask for.

see this slide in the deck ›

Interfaces and BFM

Then: the protocol hides inside a task

code/u2/interfaces-bfm/vtalu_bfm.sv · #send_op
   // The protocol handling can now be changed in one single place.
   // Fix it here once, and the fix propagates through the rest of the code
   task send_op(input byte iA, input byte iB, input operation_t iop,
                output shortint alu_result);

      op_set = iop;

      if (iop == rst_op) begin
         @(posedge clk);
         reset_n = 1'b0;
         start = 1'b0;
         @(posedge clk);
         #1;
         reset_n = 1'b1;
      end else begin
         @(negedge clk);
         A = iA;
         B = iB;
         start = 1'b1;
         if (iop == no_op) begin
            @(posedge clk);
            #1;
            start = 1'b0;
         end else begin
            do @(negedge clk); while (done == 0);
            start = 1'b0;
         end
      end

   endtask : send_op
el archivo entero, 71 líneas
interface vtalu_bfm;
   // brings in operation_t. e.g. (no_op  = 3'b000)
   import vtalu_pkg::*;

   byte unsigned        A;
   byte unsigned        B;
   bit                  clk;
   bit                  reset_n;
   wire          [ 2:0] op;
   bit                  start;
   wire                 done;
   wire          [15:0] result;
   wire                 ovf;
   operation_t          op_set;

   assign op = op_set;

   // The BFM is a first step towards modularity
   // It takes care of driving every low-level signal
   // and keeps the protocol in one single place

   // DUT clock
   initial begin
      clk = 0;
      forever begin
         #10;
         clk = ~clk;
      end
   end

   // DUT reset
   task reset_alu();
      reset_n = 1'b0;
      @(negedge clk);
      @(negedge clk);
      reset_n = 1'b1;
      start = 1'b0;
   endtask : reset_alu

   // The protocol handling can now be changed in one single place.
   // Fix it here once, and the fix propagates through the rest of the code
   task send_op(input byte iA, input byte iB, input operation_t iop,
                output shortint alu_result);

      op_set = iop;

      if (iop == rst_op) begin
         @(posedge clk);
         reset_n = 1'b0;
         start = 1'b0;
         @(posedge clk);
         #1;
         reset_n = 1'b1;
      end else begin
         @(negedge clk);
         A = iA;
         B = iB;
         start = 1'b1;
         if (iop == no_op) begin
            @(posedge clk);
            #1;
            start = 1'b0;
         end else begin
            do @(negedge clk); while (done == 0);
            start = 1'b0;
         end
      end

   endtask : send_op

endinterface : vtalu_bfm

The sentence that orders the section: the protocol gets written once. If tomorrow the DUT asks for two cycles of start, this task gets touched and nothing else. The #1 after the @(posedge clk) is the same precaution as the scoreboard of the conventional testbench: moving the signals one delta after the edge, not on the edge. And the hook for day 4: the driver of the put and get ports is exactly this task, but inside a class and fed by a port. The operation —hiding the wire— is the same; what changes is where the data comes from.

see this slide in the deck ›

Interfaces and BFM

The top, with the BFM in the middle

code/u2/interfaces-bfm/top.sv
module top;
   vtalu_bfm bfm ();
   // Tester: generates the stimulus
   tester tester_i (bfm);
   // Coverage: provides functional coverage
   coverage coverage_i (bfm);
   // ScoreBoard: checks the results
   scoreboard scoreboard_i (bfm);

   vtalu DUT (
       .A(bfm.A),
       .B(bfm.B),
       .op(bfm.op),
       .clk(bfm.clk),
       .reset_n(bfm.reset_n),
       .start(bfm.start),
       .done(bfm.done),
       .ovf(bfm.ovf),
       .result(bfm.result)
   );
endmodule : top

It is worth reading this top.sv next to the previous section's, because the diff is the whole section: where there was a list of logic declared in the top and wired by hand to the DUT, now there is one line that instantiates the BFM and one that plugs it in. What has to be pointed out, because it is what repeats for the rest of the course: the DUT connects to bfm.A, bfm.B, bfm.op… The interface is not an abstract object, it is the same old wires grouped under a name. Nothing became slower or stranger — it became harder to connect wrong. And the detail that pays for itself the day the DUT grows: adding a signal to the protocol means touching the interface, and everyone who uses it sees it appear. In the previous version you had to remember to wire it in every instance.

see this slide in the deck ›

Interfaces and BFM

Scoreboard: the same check, now talking to the BFM

code/u2/interfaces-bfm/scoreboard.sv
module scoreboard (
    vtalu_bfm bfm
);
   import vtalu_pkg::*;

   always @(posedge bfm.done) begin
      shortint predicted_result;
      bit      predicted_ovf;
      #1;
      case (bfm.op_set)
         add_op: predicted_result = bfm.A + bfm.B;
         sub_op: predicted_result = bfm.A - bfm.B;
         and_op: predicted_result = bfm.A & bfm.B;
         xor_op: predicted_result = bfm.A ^ bfm.B;
         mul_op: predicted_result = bfm.A * bfm.B;
      endcase  // case (op_set)

      // ovf belongs to sub and to nobody else: with 8-bit inputs and a 16-bit
      // output, neither the addition nor the multiplication can overflow.
      predicted_ovf = (bfm.op_set == sub_op) && (bfm.A < bfm.B);

      if ((bfm.op_set != no_op) && (bfm.op_set != rst_op))
         if (predicted_result != bfm.result || predicted_ovf != bfm.ovf)
            $error(
                "FAILED: A: %0h  B: %0h  op: %s result: %0h ovf: %0b",
                bfm.A,
                bfm.B,
                bfm.op_set.name(),
                bfm.result,
                bfm.ovf
            );

   end
endmodule : scoreboard

What has to be pointed out is what did not change. The verification logic is identical; what got modularized is the wiring. That is the whole section. The bfm. in front of every signal is the first time the student sees an interface hierarchy, and it is worth writing it on the board: top.bfm.A. In the tests that same reference is going to arrive through the config_db into a class, and there the prefix will be a handle instead of an instance name. The debt that stays open: the scoreboard still watches the signals directly. Only in the analysis ports is it going to receive transactions from a monitor and stop knowing that a clk exists.

see this slide in the deck ›

Interfaces and BFM

Tester: it asks for operations, it does not move wires

code/u2/interfaces-bfm/tester.sv · #stimulus-loop
   initial begin
      byte unsigned iA;
      byte unsigned iB;
      operation_t   op_set;
      shortint      result;

      bfm.reset_alu();
      repeat (1000) begin : random_loop
         op_set = get_op();
         iA = get_data();
         iB = get_data();
         bfm.send_op(iA, iB, op_set, result);
      end : random_loop
      $finish;
   end  // initial begin
el archivo entero, 46 líneas
// The Tester module is simpler now, because it no
// longer has to deal with low-level signals
module tester (
    vtalu_bfm bfm
);
   import vtalu_pkg::*;

   function operation_t get_op();
      bit [2:0] op_choice;
      op_choice = $random;
      case (op_choice)
         3'b000: return no_op;
         3'b001: return add_op;
         3'b010: return sub_op;
         3'b011: return and_op;
         3'b100: return xor_op;
         3'b101: return mul_op;
         3'b110: return rst_op;
         3'b111: return rst_op;
      endcase  // case (op_choice)
   endfunction : get_op

   function byte get_data();
      bit [1:0] zero_ones;
      zero_ones = $random;
      if (zero_ones == 2'b00) return 8'h00;
      else if (zero_ones == 2'b11) return 8'hFF;
      else return $random;
   endfunction : get_data

   initial begin
      byte unsigned iA;
      byte unsigned iB;
      operation_t   op_set;
      shortint      result;

      bfm.reset_alu();
      repeat (1000) begin : random_loop
         op_set = get_op();
         iA = get_data();
         iB = get_data();
         bfm.send_op(iA, iB, op_set, result);
      end : random_loop
      $finish;
   end  // initial begin
endmodule : tester

The comparison with the conventional testbench is the slide: over there the tester raised start, waited for done and lowered start, mixed in with the stimulus generation. Here it only generates stimulus. Worth counting how many places of the conventional testbench knew about the protocol —the tester and the scoreboard— and how many move start now: one, the send_op. The scoreboard still waits for done on its own; that debt gets paid by the monitor of the analysis ports. And the hook for day 4: in the put and get ports this same tester is going to be a class with a put_port, and the BFM is going to end up on the other side of a driver. The operation is the same one we did here —hiding the wire— repeated one level up.

see this slide in the deck ›

Interfaces and BFM

The protocol rule, written in the simulator's language

// IMMEDIATE: a statement. It checks the present, where the thread goes by.
assert (op_set != no_op || done == 0)
   else $error("no_op should not raise done");

// CONCURRENT: a declaration with a clock. It plugs itself in once and
// checks EVERY edge of the simulation, on its own, forever.
a_operandos_estables :
assert property (@(negedge clk) start |=> $stable(A) && $stable(B));

This is a hook, not the section: the syntax must not be explained here, and least of all the |=> or the edge. The only thing that has to stick is that the protocol rule can be written in a form the simulator understands. Worth pointing out the asymmetry that opens up: the BFM complies with the protocol, and that is enough while all the stimulus goes through the BFM. As soon as a legacy module, a VIP from another team or a driver written in a hurry shows up, complying stops being the same as checking. If somebody asks why one says negedge and the other does not, the short answer is "because the BFM writes on the negedge, and that is only fully visible in the assertions". That is the question we want to plant.

see this slide in the deck ›

Interfaces and BFM

Where each thing gets checked

The sentence worth leaving on the board for the whole course: the scoreboard checks what the DUT computes; the assertions, how it is talked to. What is left of day 1 and the six days that follow are the first half; the second arrives at the end, and it arrives on this same example. There is no need to promise more than that. The assertions come back with the property of this slide, write it for real, and show why the edge chosen is not a detail.

see this slide in the deck ›

Interfaces and BFM

The hole the BFM leaves: the timing

code/u2/clocking/sin_clocking.sv · #three-samples
      // 1. Sample AT the edge. The DUT always_ff updates d_out with a
      //    nonblocking assignment, which lands after this initial already ran:
      //    you read the OLD value.
      @(posedge clk);
      en_el_flanco = d_out;

      // 2. The same edge, one delta later. The nonblocking assignment landed
      //    already and you read the NEW value. That "#1" is visible nowhere and
      //    changes the result.
      @(posedge clk);
      #1;
      un_delta_despues = d_out;

      // 3. The opposite edge: the trick the whole course BFM uses. It works,
      //    but it asks whoever reads it to know why.
      @(negedge clk);
      en_el_flanco_opuesto = d_out;
el archivo entero, 50 líneas
// Sampling by hand: without a clocking block you have to pick the edge AND the
// delta, in every task, over and over. This top runs the three variants people
// write in their first week: one reads the OLD value, the other two the NEW.
//
//   bash run.sh
module dut_reg (input bit clk, input byte unsigned d_in, output byte unsigned d_out);
   // The DUT counts: on every edge, the output becomes the input + 1.
   always_ff @(posedge clk) d_out <= d_in + 8'd1;
endmodule

module top_sin;
   bit           clk = 0;
   byte unsigned d_in, d_out;

   always #5 clk = ~clk;
   dut_reg u_dut (.*);

   byte unsigned en_el_flanco, un_delta_despues, en_el_flanco_opuesto;

   initial begin
      d_in = 8'd10;

      // 1. Sample AT the edge. The DUT always_ff updates d_out with a
      //    nonblocking assignment, which lands after this initial already ran:
      //    you read the OLD value.
      @(posedge clk);
      en_el_flanco = d_out;

      // 2. The same edge, one delta later. The nonblocking assignment landed
      //    already and you read the NEW value. That "#1" is visible nowhere and
      //    changes the result.
      @(posedge clk);
      #1;
      un_delta_despues = d_out;

      // 3. The opposite edge: the trick the whole course BFM uses. It works,
      //    but it asks whoever reads it to know why.
      @(negedge clk);
      en_el_flanco_opuesto = d_out;

      $display("d_in = %0d, and the DUT computes d_in + 1 = %0d", 8'd10, 8'd11);
      $display("  sampled AT the posedge        : %0d", en_el_flanco);
      $display("  sampled one delta later       : %0d", un_delta_despues);
      $display("  sampled at the negedge        : %0d", en_el_flanco_opuesto);
      $display("");
      $display("Three lines that look the same and are not. And that decision");
      $display("is repeated in every task of the interface.");
      $finish;
   end
endmodule : top_sin

Run this live, it takes two seconds: make u2/clocking. The three lines it prints are the whole slide. The first one is the surprising one and it is worth explaining slowly: sampling on the posedge gives the old value, because the DUT's always_ff updates with a nonblocking assignment and that one lands after this initial already ran. It is not a Verilator bug and not an oddity: it is the LRM region order, and it is the same on Questa. The second one is the worst of the three, and it is the one to mark as a silent trap: it works, and it works by accident. The #1 does not say which problem it solves, it is not documented, and the day somebody deletes it because "it did nothing" the testbench keeps compiling and starts lying. It is the same #1 as the scoreboard of the conventional testbench, and the difference is a single one: there the slide says which race it avoids. An explained #1 is a patch that knows it is a patch; a loose #1 is the trap. And the third is ours. Worth being honest with the group: the course drives on negedge and samples on posedge because it is the trick you can understand without having seen clocking blocks, not because it is what gets written on a project.

see this slide in the deck ›

Interfaces and BFM

Why the edge alone is not enough: the regions

The regions of an edge: Preponed, Active and NBA, and the three samples that give different values

This figure is the one to leave on screen while reading the three samples of the previous slide. Without it the explanation is "the old value", which sounds like a quirk of the simulator; with it, it is a consequence of the order, and the order is in the LRM. The scheduler has more regions than the four in the drawing —docs/clocking-blocks.md (in Spanish) lists them all— but these are enough to explain the three cases, and adding the others changes no answer. The question worth asking the group before showing the answer: if the DUT writes in NBA and your initial wakes up in Active, which one runs first? That is where it shows on its own why sampling on the edge gives the value from before. And the hook with day 7: the amber bar on the left is the region where SVA samples. It is the same figure that explains why an assertion does not check what happened but what it saw, and why the @() you give it changes the result.

see this slide in the deck ›

Interfaces and BFM

The clocking block: the edge and the delta, declared once

code/u2/clocking/con_clocking.sv · #reg_bfm
interface reg_bfm (input bit clk);
   byte unsigned d_in, d_out;

   // The only two timing lines in the testbench:
   //   input  #1step  samples the STABLE value right before the edge, which is
   //                  what the hardware sees. Never what the nonblocking
   //                  assignment just wrote on that same edge.
   //   output #0      drives ON the edge, in the nonblocking region, so the DUT
   //                  does not see it until the next edge.
   clocking cb @(posedge clk);
      default input #1step output #0;
      output d_in;
      input  d_out;
   endclocking

   modport tb (clocking cb);
endinterface : reg_bfm
el archivo entero, 73 líneas
// The same DUT, with a clocking block: the edge and the delta are declared ONCE,
// inside the interface, and no task ever picks them again.
//
//   bash run.sh
module dut_reg (input bit clk, input byte unsigned d_in, output byte unsigned d_out);
   always_ff @(posedge clk) d_out <= d_in + 8'd1;
endmodule

interface reg_bfm (input bit clk);
   byte unsigned d_in, d_out;

   // The only two timing lines in the testbench:
   //   input  #1step  samples the STABLE value right before the edge, which is
   //                  what the hardware sees. Never what the nonblocking
   //                  assignment just wrote on that same edge.
   //   output #0      drives ON the edge, in the nonblocking region, so the DUT
   //                  does not see it until the next edge.
   clocking cb @(posedge clk);
      default input #1step output #0;
      output d_in;
      input  d_out;
   endclocking

   modport tb (clocking cb);
endinterface : reg_bfm

module top_con;
   bit clk = 0;
   always #5 clk = ~clk;

   reg_bfm bfm (clk);
   dut_reg u_dut (.clk(clk), .d_in(bfm.d_in), .d_out(bfm.d_out));

   byte unsigned primera, segunda, tercera;

   initial begin
      // Driving is "<=" against the clocking block. There is no edge to pick.
      bfm.cb.d_in <= 8'd10;

      // @(bfm.cb) is "wait for this clocking block's edge". There are THREE, and
      // the reason is the whole lesson:
      //   1st edge  the output #0 only now puts d_in at 10, in the nonblocking
      //             region, so the DUT has not seen it yet.
      //   2nd edge  now the DUT takes d_in=10 and computes d_out <= 11.
      //   3rd edge  the input #1step samples the STABLE value before this edge,
      //             which is already 11.
      // A testbench without a clocking block hides this behind a #1 and works by
      // accident. Here the arithmetic is visible and does not depend on the tool.
      @(bfm.cb);
      @(bfm.cb);
      @(bfm.cb);

      // And sampling is reading cb.d_out. It is read three times, at three
      // different points of the same cycle, and gives the same value all three.
      primera = bfm.cb.d_out;
      #2;
      segunda = bfm.cb.d_out;
      #3;
      tercera = bfm.cb.d_out;

      $display("d_in = %0d, and the DUT computes d_in + 1 = %0d", 8'd10, 8'd11);
      $display("  cb.d_out, right after the edge  : %0d", primera);
      $display("  cb.d_out, two units later       : %0d", segunda);
      $display("  cb.d_out, five later            : %0d", tercera);
      $display("");
      $display("The same number all three times, and not a single #1 in the testbench.");
      $display("The edge and the delta were declared once, in the clocking block.");

      if (primera != 8'd11 || segunda != 8'd11 || tercera != 8'd11)
        $fatal(1, "the clocking block should give 11 all three times");
      $finish;
   end
endmodule : top_con

This is the slide you have to be able to repeat in an interview, because clocking block is one of the first things people get asked and the good answer is not "it synchronizes": it is it declares the sampling and driving instant once, in the interface, so that no task picks it again. The detail that confuses and is worth previewing: in the example you need three @(cb) to see the 11, and not two. One for the output #0 to put the data, another for the DUT to take it, and the third because the input #1step samples what is stable before the edge. The arithmetic is in plain sight in the code and does not depend on the simulator — which is exactly the difference with the #1. If somebody asks why the course does not use it from day 1: because to understand which problem it solves you have to have had the problem. Only now, with the BFM written and the three lines running, does the question make sense. Verilator supports it since 5.x and the example ends in $fatal if the sampling stops giving 11: it is the safety net that warns if a release changes the semantics.

see this slide in the deck ›

Interfaces and BFM

Are they mandatory? No — and it is worth knowing why

The question The answer
Can a correct TB be written without clocking blocks? Yes. With NBA in the driver and scheduler discipline, there is no race
What should a team use? Use them on synchronous interfaces — it is what the lowRISC/OpenTitan style guide says

This slide is the one that keeps the course from lying by simplifying, and it is worth spelling out: "always use clocking blocks" is a style rule, not a theorem. Whoever repeats it as if it were the latter is in for an uncomfortable argument with the first senior they run into. Dave Rich's sentence that orders the topic —and it is worth quoting verbatim— is from 2014: "If you are already familiar with Verilog testbenches, then you probably don't need them." He held it in 2022 and in 2024. It does not say they are wrong: it says they are not a necessary condition. And on the other side is lowRISC, which in OpenTitan's DV Coding Style Guide makes them mandatory for synchronous signals. They do not contradict each other: Dave answers "can I?" and lowRISC answers "what do we all do here?". It is exactly the difference between well-used malloc/free and a style guide that mandates smart pointers — an expert being able to manage memory by hand does not make RAII useless. Where they really move the needle, and this is what to retain: reusable agents and VIP —whoever writes them is far from whoever uses them—, gate-level with SDF, and protocols where the spec defines setup/hold. There they stop being style. The long material, with the sources, is in docs/clocking-blocks.md (in Spanish).

see this slide in the deck ›

Interfaces and BFM

The price: two names for the same wire

code/u2/clocking/mezcla.sv · #two-reads
      // The two reads, at the SAME instant, on the SAME wire.
      por_cb = bfm.cb.d_out;
      crudo  = bfm.d_out;

      $display("at the same instant, on the same wire:");
      $display("  bfm.cb.d_out (what the clocking block sampled) : %0d", por_cb);
      $display("  bfm.d_out    (the live wire)                   : %0d", crudo);
      $display("");
      $display("They differ by one cycle, and no warning says so. A monitor that");
      $display("reads through the clocking block and a scoreboard that reads the");
      $display("raw wire do not fail every time: they fail when the data changes.");
el archivo entero, 53 líneas
// The trap that comes with the clocking block: mixing the two domains.
//
// Once a signal belongs to a clocking block, there are TWO names for the same
// wire -- bfm.d_out and bfm.cb.d_out -- and they are NOT the same value. One is
// the live wire; the other is what the clocking block sampled #1step before the
// edge. On a signal that changes every cycle they differ by exactly one cycle.
//
// Nothing warns about this: it compiles, it runs, and a scoreboard that reads
// one of them while the monitor reads the other fails intermittently.
//
//   bash run.sh
module contador (input bit clk, output byte unsigned d_out);
   // Counts on every edge, so the live wire and the sampled one never agree.
   always_ff @(posedge clk) d_out <= d_out + 8'd1;
endmodule

interface cnt_bfm (input bit clk);
   byte unsigned d_out;
   clocking cb @(posedge clk);
      default input #1step;
      input d_out;
   endclocking
endinterface : cnt_bfm

module top_mezcla;
   bit clk = 0;
   always #5 clk = ~clk;

   cnt_bfm bfm (clk);
   contador u_cnt (.clk(clk), .d_out(bfm.d_out));

   byte unsigned por_cb, crudo;

   initial begin
      repeat (4) @(bfm.cb);

      // The two reads, at the SAME instant, on the SAME wire.
      por_cb = bfm.cb.d_out;
      crudo  = bfm.d_out;

      $display("at the same instant, on the same wire:");
      $display("  bfm.cb.d_out (what the clocking block sampled) : %0d", por_cb);
      $display("  bfm.d_out    (the live wire)                   : %0d", crudo);
      $display("");
      $display("They differ by one cycle, and no warning says so. A monitor that");
      $display("reads through the clocking block and a scoreboard that reads the");
      $display("raw wire do not fail every time: they fail when the data changes.");

      if (por_cb == crudo)
        $fatal(1, "they were expected to differ: the example stopped showing the trap");
      $finish;
   end
endmodule : top_mezcla

This is the flip side of the previous slide and the reason the course does not use them in its BFM: used badly they are worse than not using them, because the failure mode is intermittent and the waveform looks fine. The real mistake, the one that shows up in the forums, has two shapes and it is worth naming both. One is mixing the event: @(posedge vif.clk) and then reading vif.cb.foo. The other is mixing the access: the monitor reads vif.cb.data and the scoreboard —or an assertion, or a debug $display— reads vif.data. Both end the same way: two temporal views of the same wire, and a mismatch that shows up once every twenty runs. Worth running the example and leaving the number in plain sight: 3 and 4. Nobody argues with a 3 and a 4 on the same wire. And the moral that hooks into the silent traps appendix, where this one is written down: in verification the expensive error is not the one that breaks, it is the one that lies. A half-done clocking block lies.

see this slide in the deck ›

Interfaces and BFM

Summary of the unit

This is the most important section of day 1 and the one that looks least like it. Everything that comes after —classes, factory, agents, sequences— rearranges pieces above this line. The line itself never gets touched again. The short way to say it: from here on the testbench stops talking in wires and starts talking in operations. That is the first step towards UVM, and it is already taken, without having written a single class. If the group comes from RTL, the analogy closes on its own: the BFM is to the testbench what a device driver is to the operating system.

see this slide in the deck ›

Review · Day 1

1 of 7 · The question of the day

A thousand random operations, the scoreboard reported no error and the log ends in PASS. What are you missing to say the DUT is verified?

Watch it fail — the PASS of a scoreboard that never saw an error says nothing: it may not have compared. VTALU_BUG=1 flips one bit of the result and the testbench has to fail; make mutante demands it for the three testbenches of days 1 and 2.

see this slide in the deck ›

Review · Day 1

2 of 7 · The ALU spec

While the VTALU is running an operation, what do start and the operands have to do?

Stable until done — it is the DUT protocol, and it is exactly the reason the BFM exists: to wrap that rule in a single place so that no test forgets it. The first distractor describes a real protocol —pulse start, latched operands— that this DUT does not have.

see this slide in the deck ›

Review · Day 1

3 of 7 · Functional coverage

The test runs 1000 random operations and the code coverage reads 100 %. What does that tell you about the verification?

Very little — code coverage measures the DUT; functional coverage measures the spec. A feature the designer never wrote gives 100 % of lines and 0 % of what matters, and the report is not going to tell you.

see this slide in the deck ›

Review · Day 1

4 of 7 · covergroup

You declare a covergroup, you new() it, you run a thousand operations and the report reads 0 %. What is the first thing to look at?

The sample() — a covergroup does not sample itself: somebody has to call it, on the edge or when a transaction arrives. Without that call the code compiles, runs, and the report reads 0 without a single warning.

see this slide in the deck ›

Review · Day 1

5 of 7 · Interfaces and BFM

What does the testbench gain when the protocol moves into a BFM?

It stops talking in signals — the BFM translates one operation into a handshake of signals. The tester never touches a wire again; the scoreboard and the coverage keep reading them, through the interface, until a monitor sends them transactions. It is the first step towards UVM.

see this slide in the deck ›

Review · Day 1

6 of 7 · The verification plan

In the verification plan, what goes in the Measure column?

Which bin gets filled — the five columns are Feature, Scenario, Stimulus, Check and Measure, and this is the one that costs the most: it forces you to decide beforehand what is going to be counted. If it is left empty, nobody is going to find out that the scenario never happened — a case the random never touched and that has no bin is indistinguishable from one that happened a thousand times.

see this slide in the deck ›

Review · Day 1

7 of 7 · clocking block

Are clocking blocks needed to write a UVM testbench without races?

They are not needed, and they are worth using anyway — they belong to SystemVerilog, not to UVM, and no uvm_driver samples for you. What avoids the race is understanding the scheduler: a driver that drives with <= against a DUT that registers with <= is already deterministic. The clocking block does not replace that understanding, it wraps it — and that is where it pays off: reusable agents, VIP, gate-level and protocols with setup/hold in the spec.

see this slide in the deck ›

Exercise · Day 1 · 1 of 2

A new operation, end to end

cd code/ejercicios/d1 && bash run.sh

Half an hour, and it is worth leaving them alone: the statement and the README are enough. The guaranteed stumble is the width: the hardware shifts by B[2:0], not by the whole B, so with B = 8'h20 it shifts zero places and not thirty-two. The scoreboard model has to do the same or the TB they wrote themselves is going to accuse them. It is warned about in the README, and it still happens. The third step is the most skipped one and the only one that fails silently: bins single_cycle[] covers the range [add_op : xor_op], which reaches up to 3'b100. shr_op falls outside every bin, the simulation passes green and — here is the worst part — the report still says 100 %: 76 out of 76. The missing bin does not lower the percentage because it never made it into the denominator. There is the lesson that holds for the rest of the course: an opcode nobody measures is an opcode nobody verified, and the percentage is precisely the metric that cannot see it. With the solution it is 77 out of 77, starting from 86.8 % (66 out of 76). The ten that were missing were a single one: the bucket Verilator invents for 3'b110, the value the enum did not have — and the shift is precisely 3'b110, so it fills it along the way. Worth showing both numbers and not only the percentage: that is why the run.sh demands, on top of the 100 %, that single_cycle has one more bin: a half-done job reaches the 100 % too.

see this slide in the deck ›

Exercise · Day 1 · 2 of 2

The waves: when the log is not enough

cd code/ejercicios/d1b && bash run.sh

Half an hour, and it is the exercise that justifies the debug appendix: there it says that the waves are the tool 47 % of the work gets done with, and up to here the course had not made them use one even once. The bug is a protocol one and it is the same one that comes back in the capstone: send_op counts edges instead of waiting for the handshake. For single-cycle operations it makes no difference; the multiplication takes four, so send_op returns too early, the next stimulus overwrites A and B, and when done finally goes up the scoreboard compares the new operands against the old result. That is why the two times stage 1 asks for are not bureaucracy: the second is the done of the first multiplication, and whoever looks for it in the viewer sees that A and B have already changed. That is the answer to the why, and there is no way to read it in the log. The question to throw at the group when they finish: if the multiplier went from four edges to five, which of the two versions of send_op finds out? Neither counts right; only the one that waits for done keeps working. It is literally the clue of the APB wait state of day 7. And this is the 47 % from the trends slide, seen up close: the log says failed and for the why you have to go somewhere else. Today that somewhere else is the waves; on day 3, in reporting, it is the log that learns to say it.

see this slide in the deck ›

Six out of seven, again

Closing the day

Thirty seconds, and they are the ones that close the arc of the day: the first slide asked a question and this one answers it with what the student wrote, not with theory. If the group is tired, put the 14 % slide back up for a second before this one: the question was "how do you know you verified?", and the short answer is "I have the plan, I watched it fail and I know which bin is missing". What not to over-promise: this does not prevent the respin. What it prevents is sending to the fab a PASS nobody knew the meaning of. The last bullet is the one day 2 opens with "Yesterday we left…": a tester that only multiplies is copying the file and deleting five lines. That is the problem classes solve, and it is worth leaving as a problem and not as a promise.

see this slide in the deck ›