The Test Bench

The focus should be on system-level testing rather than component-level verification. The components themselves have already been verified by their respective owners.

Ideally, the test bench should allow for the following:

• Execute system-level test cases in a reasonable time frame. With system-level test cases, I mean test cases where the DUT is the SoC.
• A short lather-rinse-repeat cycle of making code changes and testing them on a system-level DUT.
• Full signal visibility into the build, to aid test case development as well as debugging.
• Reasonably easy automated testing. With the caveat that automated testing is never truly easy.

Verilator

Boxlambda uses Verilator to create test benches. Verilator is a compiler. It compiles, or rather verilates, an HDL design into a C++ model. It then picks up any user-provided C++ testbench/wrapper code and compiles the whole thing into an executable, optionally with the ability to generate traces.

C++ is not an ideal language for test case development, but it'll get the job done, and it's a compiled language, so it's fast.

A simple Test Bench

The proof-of-concept test bench for BoxLambda is based on the example code included in the Verilator distribution:

https://github.com/verilator/verilator/blob/master/examples/make_tracing_c/sim_main.cpp

I included UARTSIM, the UART co-simulation class that ZipCPU provides along with the UART Verilog implementation in the wbuart32 repository:

https://github.com/epsilon537/wbuart32/tree/master/bench/cpp

The test bench does the following:

1. Instantiate the verilated Hello World model and the UARTSIM co-simulation object.
2. Optionally, controlled by a command-line option, enable tracing.
3. Optionally, controlled by a command-line option, wait for an OpenOCD connection.
4. Run the model for a fixed number of clock cycles.

5. While running the model:

   1. Feed characters into UARTSIM's transmit path, i.e. towards the model.
   2. Feed the model's UART output to UARTSIM and UARTSIMs output to the model's UART input.
   3. Capture and display the decoded UARTSIM output and the GPIO outputs.

6. Pass/Fail criterium: After running the model for the set number of clock cycles, match the captured UART and GPIO outputs against the expected results.

As suggested by ZipCPU in his Verilog tutorial, I use nCurses for positional printing inside the terminal windows. This way, I can easily build a display that refreshes, rather than scrolls, whenever the model produces new UART or GPIO data to display.

Current test bench source code examples:

• projects/hello_world/sim/sim_main.cpp.
• projects/picolibc_test/sim/sim_main.cpp.
• projects/ddr_test/sim/sim_main.cpp

Are we running in a Simulation?

Software running on Ibex needs to know whether it's running in a simulation or on FPGA, so it can adjust timings such as the LED blink period. I'm using GPIO1 bits 3:0 for this purpose. In a simulation, I set these bits to 4'bf. On FPGA I set them to something else. The hello.c test program includes the following check:

  //GPIO1 bits3:0 = 0xf indicate we're running inside a simulator.
  if ((gpio_get_input(&gpio1) & 0xf) == GPIO1_SIM_INDICATOR)
    uart_printf(&uart0, "This is a simulation.\n");    
  else
    uart_printf(&uart0, "This is not a simulation.\n");

Files and command line options

All files created by Verilator go in the <project_dir>/generated/ subdirectory. The name of the generated executable is Vmodel. As you can see in the sim_main.cpp source code above, Vmodel accepts a few command line options:

• Vmodel -t: Execute with waveform tracing enabled. The program generates a .fst trace file in the current directory. .fst files can be viewed with gtkwave.

Gtkwave View of Waveform Trace Generated by Hello World Verilator Test Bench

• Vmodel -i: Run in interactive mode, vs. the default batch mode. In interactive mode, the program may wait for keypresses. Batch mode is used for non-interactive automated testing.