Debugger

commit: 7609f4075a9f5cd7a83c65efb35ddc270a03f425

In this commit, the debugger is based on RISC-V External Debug Support Version 0.13.2. The most of the function is implemented. However, the riscv-test/debug has not been passed yet. The remained issues will be fixed in the future.

The debug solution is based on the model in GDB -> OpenOCD -> FT2232 -> Debugger -> Core

Core

DMode

Independent of MMode, SMode, UMode in Privilege Level, DMode is an inner user invisible register which flags whether the core is in Debug Mode.

DCSR

DCSR is a register for debuging, whoes fields afface the behavior of debug

ebreakm，ebreaks，ebreaku respectively affect the behavior of ebreak instruction in three privilege levels: 0：Cause BreakPiont Exception 1：Enter Debug Mode
causerecords the reason why Debug Mode is entered. There are 5 reason

The softwave breakpoint causes by ebreak
The hardwave breakpoint causes by trigger
The unmaskable interrupt cause by haltreq
Re-enter debug mode by stepping
he unmaskable interrupt cause by resethaltreq（can merge in 3）

step The control field of stepping. The debug mode will be re-entered once the instruction is committed, if this field is set and the core is not in debug mode.
prv Record the privilege level while entering the debug mode. The privilege level will be updated according to this field when return from debug mode.

DPC

Record the pc while entering the debug mode and re-direct pc when return from debug mode. It's very simular to mepc.

Be aware, the softwave breakpoint causes by ebreak is implemented by modifying the instruction to ebreak at the porper address by OpenOCD. The instruction will be modified back after the breakpoint is reached. As a result, the instruction is not actually executed when the breakpoint is triggered. And the dpc should be updated to the address of this instruction.

DSCRATCH 0/1/2

Dscratch works as a small stack, which save and resume the value in Regfile temply while executing the Abstract Command

The number of dscratch is implementation dependent with Abstract Command.

DRET

Simalar to MRET, the core will return from debug mode, update the pc with dpc, update the Privilege Level with prv, after the DRET is executed.

eBreak

Depending on the configuration of dcsr and Privilege Level, either a softwave exception or sofewave breakpoint will be caused by ebreak.

Debug Interrupt

An unmaskable interrupt input should be provided by the core. Once the unmaskable interrupt comes, the pc will be re-direct to a pre-set address of Debug ROM, and fetch instruction from that peripheral. The Debug Interrupt will keep asserting until the debugger comfirms that the core is halted.

Debug Transport Module (DTM)

As a bridge between hardwave and debug logic, DTM can access registers of DM with the register access from hardwave interface. The DTM may work in an independent clock domain, for the frequence is different from the core. The connection between DTM and DM may be a self-defined bus.

JTAG

Base on IEEE Std 1149.1-2013, JTAG works as finite-state-machine. The next state is depend on the TMS, while the TCK driving the clock. With the driven by TCK, one bit is shifted-out to TDO or one bit is shifted-in from TDI. A serial mode it is.

JTAG TAP

Debug Module Interface (DMI)

DMI works as a CrossBar between DTM and a numbers of DM. Also the timing path may cross the clock domain at DMI.

Debug Module (DM)

The cores enter the debug mode with the lead of DM, and make a specific behavior to inquire or modify the current state of the SoC. The number of cores, that DM can debug with is 1~2^20.

Function Unit

A fully implemented DM has 3 sets of Bus Interface:

Self-define DMI Bus Port as slave interface
A Bus Port as slave interface for instruction fetching and data accessing from cores
A System Bus Port as master interface to access the whole memeory region of the SoC

Private Memory Region of DMI

A number of registers in DM can be accessed by DTM. By writing these registers and reading back, the DTM can instrcut DM to operate and get result back.

Public Memory Region of SoC

Once entering the debug mode, the cores make a specific behavior by fetching instructions from these region and exchange information with DM by reading or writing the specific region.

DEBUG_ROM_HALTED A write-only region for cores. The halted cores will read out its hart id and write it into this region to info DM that it's halted.
DEBUG_ROM_GOING A write-only region for cores. The halted cores will read out its hart id and write it into this region to info DM that it has received the next instruction and is moving to GOING state.
DEBUG_ROM_RESUMING A write-only region for cores. The halted cores will read out its hart id and write it into this region to info DM that it's leaving the debug mode.
DEBUG_ROM_EXCEPTION A write-only region for cores. The halted cores will read out its hart id and write it into this region to info DM that an exception has been caused during debuging.
WhereTo

WhereTo is an address-variable jal instruction. When Abstract Command should be executed, the implementation modifies the target address and the instruction flow will be led to:

Abstract Command ROM
Program Buff

Abstract Command ROM

Abstract Command ROM is a small instruction region that can vary limited by DM for function of Abstract Command. It works as followed:

Reading/Writing the Register-Files: The values will be exchange with DATA by load/store instructions
Reading/Writing the CSR Registers: The values in Integer Register will be protected in dscratch temporarily. Then the Integer Register and load/store instructions will be used to exchange the values between the CSR and DATA
Post-Execution: Abstract Command ROM may lead the instruction flow to Program Buff if the feature of Post-Execution is required
Access Memory: The values in Integer Register will be protected in dscratch temporarily. Then the Integer Register and load/store instructions will be used to exchange the values between the specific address and DATA

Program Buff

Program Buff is a small RAM that can be both accessed by Private Memory Region of DMI and Public Memory Region of SoC. The DM can modify the instructions inside the Program Buff, then the cores will make more complex debugging behavior by fetching instruction from Program Buff.

The last instruction in Program Buff must be ebreak. The impebreak feild in dmstatus infos OpenOCD that whether the ebreak should be written by OpenOCD.

The avivable length of Program Buff should be determined by the implementation. The projbufsize feild in abstractcs infos OpenOCD that how much space can be used during operating the Quick Access. The Quick Access may be splited or failed if the Program Buff is too small to used.

DATA The DATA can be both accessed by Private Memory Region of DMI and Public Memory Region of SoC. The infomation exchanges between DM and the Cores through DATA.
DEBUG_ROM_FLAGS

A read-only field for the cores. Each hart has one-Byte here, and they will poll this region. The DM will modify this field if the hart is needed to operate(GOING, RESUMING).

Debug ROM Debug ROM is a implementation-dependent pre-compile code region. The softwave code can be refered at riscv-isa-sim. The linker file must be be in line with the implementation.

Once the hart entering the debug mode, the entry will be arrived firstly. At the infinite loop of halt, the hart will read out its HartID and write it into to the write-only register DEBUG_ROM_HALTED, which infos DM that the hart is halted. The field belongs to hart in DEBUG_ROM_FLAGS will be read out then, the next command from DM will be gotten.

going The HartID will be written into the write-only register DEBUG_ROM_GOING, which infos DM that the hart is entering the GOING state, and the hart will go to whereto finally, at where it will be led to the other debug region.
resume The HartID will be written into the write-only register DEBUG_ROM_RESUMING，which infos DM that the hart is leaving the debug mode，and the hart will go to DRET finally and resume from debug mode.

similarly, if an exception is caused during debuging, the HartID will be written into the write-only register DEBUG_ROM_EXCEPTION, which infos DM that the exception is caused. Then the ebreak will be used to re-direct instruction flow to infinite loop of halt, and waiting for the following command from DM.

// See LICENSE.SiFive for license details.

#include "riscv/encoding.h"
#include "riscv/debug_rom_defines.h"

        .option norvc
        .global entry
        .global exception

        // Entry location on ebreak, Halt, or Breakpoint
        // It is the same for all harts. They branch when 
        // their GO or RESUME bit is set.

entry:
       jal zero, _entry
resume:
       jal zero, _resume
exception:
       jal zero, _exception

_entry:
        // This fence is required because the execution may have written something
        // into the Abstract Data or Program Buffer registers.
        fence
        csrw CSR_DSCRATCH, s0  // Save s0 to allow signaling MHARTID

        // We continue to let the hart know that we are halted in order that
        // a DM which was reset is still made aware that a hart is halted.
        // We keep checking both whether there is something the debugger wants
        // us to do, or whether we should resume.
entry_loop:
        csrr s0, CSR_MHARTID
        sw   s0, DEBUG_ROM_HALTED(zero)
        lbu  s0, DEBUG_ROM_FLAGS(s0) // 1 byte flag per hart. Only one hart advances here.
        andi s0, s0, (1 << DEBUG_ROM_FLAG_GO)
        bnez s0, going
        csrr s0, CSR_MHARTID
        lbu  s0, DEBUG_ROM_FLAGS(s0) // multiple harts can resume  here
        andi s0, s0, (1 << DEBUG_ROM_FLAG_RESUME)
        bnez s0, resume
        jal  zero, entry_loop

_exception:
        sw      zero, DEBUG_ROM_EXCEPTION(zero) // Let debug module know you got an exception.
        ebreak

going:
        csrr s0, CSR_DSCRATCH            // Restore s0 here
        sw zero, DEBUG_ROM_GOING(zero)   // When debug module sees this write, the GO flag is reset.
        fence
        fence.i
        jalr zero, zero, %lo(whereto)    // Debug module will put different instructions and data in the RAM, 
                                         // so we use fence and fence.i for safety. (rocket-chip doesn't have this
                                         // because jalr is special there)

_resume:
        csrr s0, CSR_MHARTID
        sw   s0, DEBUG_ROM_RESUMING(zero) // When Debug Module sees this write, the RESUME flag is reset.
        csrr s0, CSR_DSCRATCH   // Restore s0
        dret

        // END OF ACTUAL "ROM" CONTENTS. BELOW IS JUST FOR LINKER SCRIPT.

.section .whereto
whereto:
        nop
        // Variable "ROM" This is : jal x0 abstract, jal x0 program_buffer,
        //                or jal x0 resume, as desired.
        //                Debug Module state machine tracks what is 'desired'.
        //                We don't need/want to use jalr here because all of the
        //                Variable ROM contents are set by
        //                Debug Module before setting the OK_GO byte.

State Control

The basic function of debugger is forcing the specific harts to enter the debug mode, and resume from the debug mode.

One hart can be selected in the hartsel feilds in dmcontrol, and additional more harts can ben seleted with hawindowsel and hawindow. For the address space may be comsumed a lot if 2^20 Boolean registers were maped, only a small 32-bits space is selected by hawindowsel, and their state are exposed by hawindow for user accessing.
DM can force the seleted harts entering in or resuming from halt by dmcontrol, observe their state with dmstatus
DW can drive a global reset with ndmreset feild of dmcontrol, which can be used for reseting the other module on the SoC.

Abstract Command

Access Register

Basically, one debugger must be able to read the Regiter Files when the hart halted. Furthermore, writing registers, accessing other registers, accessing without halting the harts are the enhancement feature.

Access Register is accomplished by transforming the instructions in the Abstract Command ROM, which can exchange values between hart registers and the DATA register. Additionally, the Access Register command optionally supports complex functions such as (1) register auto-increment and (2) execution of code in Program Buff before completion.

Quick Access

This feature is optional and is used to assist debuggers in performing complex and highly flexible debugging functions. OpenOCD first writes instructions to Program Buff, then directs the instruction flow to Program Buff.

Access Memory

This feature is optional and is used to access memory from the hart’s perspective. Access Register is accomplished by transforming the instructions in the Abstract Command ROM, which can exchange values between memory and the DATA register.

Since this feature essentially uses Load/Store instructions to access memory, with bus requests entering through the LSU in the hart, it produces results that are not completely identical to those produced by requesting through System Bus Access from System Bus entry:

Access to hart space
Cache consistency
Bus access permissions

Additionally, the Access Memory optionally supports complex functions such as (1) optional physical/virtual address access and (2) address auto-increment.

System Bus Access

As an additional optional feature, the DM can act as a Master to access the System Bus under the control of another set of registers which can be accessed by DMI. Compared to accessing the bus through the hart, this feature does not require halting the hart. So it has higher efficiency than accessing the bus through LSU.

The debugger (OpenOCD) accesses the bus via DM in 3 ways:

It should be noted that currently (June 2022), OpenOCD’s functional code is not complete relative to specification 0.13.2, and designs that fully comply with specifications cannot be guaranteed to be supported by OpenOCD. The source code of OpenOCD needs to be analyzed and compared.

Trigger

The feature of hardware breakpoint has not been implemented yet.

FT2232

An adapter is required to connect from USB to JTAG. Compared to commercial adapter, using FT2232 to homebrew a USB-TO-JTAG circuit is an open-source and cheap solution.

The solution can be refer from an very old project

Configuration of Chip

For a raw FT2232, FT Prog, which can be download from FTDI, is required to flash the EEPROM to store the configuration of FT2232

For the FT2232D, only channel-0 can be configed to JTAG mode:

config function of port-A to 234FIFO
config drivcer of port-A to D2XX

Driver Configuration of Windows

zadig is used to modify the driver of FT2232. Replace the driver of interface-0 to libusbK （ or WinUSB, it depends on version )

Driver Configuration of Linux refer from 《RISCV-V架构于嵌入式开发快速入门》 ISBN 978-7-115-49413-9

Comfirming the device 0403:6010 Future Techonlogy Devices International is existed.

lsusb

adding the device into the group plugdev

sudo echo -e "SUBSYSTEM=="usb", ATTR{idVendor}=="0403", ATTR{idProduct}=="6010", MODE="664", GROUP="plugdev"\nSUBSYSTEM=="tty", ATTRS{idVendor}=="0403", ATTRS{idProduct}=="6010", MODE="664", GROUP="plugdev"" >> /etc/udev/rules.d/99-openocd.rules

adding the user into the plugdev

sudo usermod -a -G plugdev WHO-AM-I

BitBang

To simulate in softwave environment, such as verilator and vcs, BitBang can be used.

connecting host of JTAG (SimJTAG.v) in to the environment
adding the driver:SimJTAG.cc, remote_bitbang.h, remote_bitbang.cc

This will open a socket after the softwave simulation starting . OpenOCD can connect to the debug environment through a specified port and operate the corresponding signal on the JTAG port. Due to speed mismatch, there will be a problem of slow simulation speed. During simulation:

Use as simple configuration as possible, reduce the capacity of the cache and reduce the number of the channels in each pipeline.
Reduce the IO, turn off unnecessary waveform output and log printing
Compilation optimization
Enable multi-threaded simulation

OpenOCD

The OpenOCD access the inner register in DTM directly through JTAG interface. Using -d if logging is required.

Also the debug interface and configuration of harts should be declared in configuration

Bitbang

To use BitBang Mode, the configuration of --enable-remote-bitbang should be enabled during the compilation

The demo run-time configuration is as followed:

adapter speed     10

interface remote_bitbang
remote_bitbang_host localhost
remote_bitbang_port 16666

set _CHIPNAME riscv
jtag newtap $_CHIPNAME cpu -irlen 5 -expected-id 0x00001

set _TARGETNAME $_CHIPNAME.cpu

target create $_TARGETNAME riscv -chain-position $_TARGETNAME



riscv set_command_timeout_sec 20

init
halt

FT2232

To use FT2232, the configuration of --enable-ftdi should be enabled during the compilation

The demo run-time configuration is as followed:

adapter_khz     1000

interface ftdi
ftdi_device_desc "Dual RS232-HS"
ftdi_vid_pid 0x0403 0x6010

ftdi_layout_init 0x0008 0x001b
#ftdi_layout_signal nSRST -oe 0x0020 -data 0x0020

set _CHIPNAME riscv
jtag newtap $_CHIPNAME cpu -irlen 5 -expected-id 0x00001

set _TARGETNAME $_CHIPNAME.cpu

set _TARGETNAME_0 $_CHIPNAME.cpu0
set _TARGETNAME_1 $_CHIPNAME.cpu1
set _TARGETNAME_2 $_CHIPNAME.cpu2
set _TARGETNAME_3 $_CHIPNAME.cpu3

target create $_TARGETNAME_0 riscv -chain-position $_TARGETNAME -coreid 0 -rtos hwthread
target create $_TARGETNAME_1 riscv -chain-position $_TARGETNAME -coreid 1 -rtos hwthread
target create $_TARGETNAME_2 riscv -chain-position $_TARGETNAME -coreid 2 -rtos hwthread
target create $_TARGETNAME_3 riscv -chain-position $_TARGETNAME -coreid 3 -rtos hwthread

target smp $_TARGETNAME_0 $_TARGETNAME_1 $_TARGETNAME_2 $_TARGETNAME_3

init
halt

The Port 3333 will be listened for GDB connection, and the Port 4444 will be listened for telnet connection.

GDB

As an open-source debugger, GDB can be easily used with following instruction to control OpenOCD:

target remote localhost::3333 connect OpenOCD
set remotetimeout 2000 set the timeout limited to 20 seconds for the bitbang mode that runs slowly
file YOUR-FILE-NAME load the debug file into GDB
load download the debug file into the memory of the SoC
compare-sections compare whether the memory completely identical to the file
b main insert a breakpiont at main
next step
info regs print the state of registers
c continue
p $pc，p/x $pc print the value of pc (in hex)
set $pc=0x80000000 set the value of pc

telnet

The command user send to GDB is abstract, the specific operation is determined by the OpenOCD. The lower level interface is provided by OpenOCD for detail debugging. To manually access the inner registers in the DM, an user-defined behavior can be operated.

The telnet instruction is as followed:

riscv dmi_read 0x00 raed DM at 0x00
riscv dmi_write 0x00 0xAA write 0xAA into DM at 0x00

riscv-test/debug

For the completeness functionnal test of the debugger , the rscripts under riscv-test/debug can be used

According to the memory map, define the linker file xxx.lds at target/RISC-V/
According to the feature of the harts, define the hart configuration file xxx_test.py at target/RISC-V/
Run the test script python ./gdbserver.py targets/RISC-V/rift_test.py in riscv-test/debug, and test the test cases in the testlib.py file one by one
Check the logs under the log folder

Uh oh!

Debugger

Debugger

Core

DMode

DCSR

DPC

DSCRATCH 0/1/2

DRET

eBreak

Debug Interrupt

Debug Transport Module (DTM)

JTAG

Debug Module Interface (DMI)

Debug Module (DM)

Function Unit

Private Memory Region of DMI

Public Memory Region of SoC

State Control

Abstract Command

Access Register

Quick Access

Access Memory

System Bus Access

Trigger

FT2232

Configuration of Chip

Driver Configuration of Windows

Driver Configuration of Linux refer from 《RISCV-V架构于嵌入式开发快速入门》 ISBN 978-7-115-49413-9

BitBang

OpenOCD

Bitbang

FT2232

GDB

telnet

riscv-test/debug

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally