Bug reproduction and root cause
We investigate a bug report, work to reproduce the failure, and check a proposed fix against the same requirement.
How the check works
The same check follows the bug through the fix.
Reproduce the failure
Find an input sequence that exposes the reported behaviour.
Explain the cause
Trace the failure back to the responsible logic.
Check the fix
Apply the same check to the revised design.
If the failure does not reproduce, the result records the checks, assumptions, and remaining unknowns.
What you receive
- A reproducing test or counterexample trace when the failure can be reproduced.
- A cause analysis with supporting evidence and remaining unknowns.
- The result of checking the same requirement on the fixed revision, when available.
- If the failure does not reproduce, a record of the checks, assumptions, and remaining unknowns.
When this is useful
- A failure appears in a long regression or on an FPGA, but not in a short test.
- A fix went in and nobody can show the original failure is gone.
- Two teams disagree about whether a report is a real design bug.
Problem, approach, and required inputs
The problem
You have a failure report but cannot make the failure happen reliably. Without a reliable replay, nobody can confirm the cause or show that a fix really works.
What we do
We express the expected behaviour as a test or property and investigate the reported revision. If a violating sequence is found, we use it to isolate the cause and check the same requirement on the revised design. Tool limits or incomplete assumptions may leave the result unresolved; the report records those limits.
What we will need from you
- The failure report and available logs.
- Access to the RTL revision it was seen on, under NDA.
- The proposed fixed revision, if available.
Example from real work
Real issue on an open source RISC-V core
Showing a debug halt let a register write slip through
A closed bug report on a RISC-V core: a control register write went through while debug halted the core. We reproduced it and proved the maintainers' fix holds.
More real issues
- Open source RISC-V core, bug report 196 · Bug reproduced, fix verifiedCatching an interrupt that broke debug single stepping
- Open source RISC-V core, bug report 323 · Bug reproduced, fix verifiedReproducing a decoder bug that rejected zext.h as illegal
- Open source RISC-V core, bug report 333 · Bug reproduced, fix verifiedShowing debug registers could be accessed outside debug mode
- Open source RISC-V core, bug report 410 · Bug reproduced, fix verifiedShowing a RISC-V core ran before halting for debug at reset
- Open source RISC-V core, bug report 558 · Bug reproduced, fix verifiedProving the fix for an instruction count bug in debug mode
Technical details
How the result is checked
The trace can be inspected in a compatible waveform viewer. The property and the run scripts are included, so your team can replay the failure and repeat the check on the fix.
Tools used for this kind of work
- SymbiYosys
- Yosys
- Verilator
- SystemVerilog Assertions
Scripts are handed over as agreed in the scope. Properties are standard SystemVerilog Assertions and can be adapted to a commercial tool. If we have not tested your simulator or flow, the scope says so.
Discuss this problem.
Tell us what you are trying to verify and what is blocking you. An engineer replies, and the scope comes back in writing after a call.
Discuss a verification problem