[Bug]: Debugger crashes at breakpoint

[steradiant]

Is there an existing issue for this?

• I have searched the existing issues

Unusable feature

The entire debugger since it crashes at a breakpoint.

Current Behavior

I'm using ARM GNU GDB version 14.3 (also tested with 13.2.rel1). When I hit a breakpoint in a specific file, the debugger crashes. The debug console shows the following output:

/data/jenkins/workspace/GNU-toolchain/arm-14-5/src/binutils-gdb--gdb/gdb/frame.c:659: internal-error: get_frame_id: Assertion `stashed' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) [answered Y; input not from terminal]

This is a bug, please report it.  For instructions, see:
<https://bugs.linaro.org/>.

/data/jenkins/workspace/GNU-toolchain/arm-14-5/src/binutils-gdb--gdb/gdb/frame.c:659: internal-error: get_frame_id: Assertion `stashed' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Create a core file of GDB? (y or n) [answered Y; input not from terminal]
0010026 W Client 1: Packet IO connection unexpectedly closed during receive: ([WinError 10054] Eine vorhandene Verbindung wurde vom Remotehost geschlossen) [packet_io]
gdb exited with code 3
gdbserver killed by signal SIGTERM
gdbserver stopped

This makes the debugger unusable. The file where this occurs is compiled with -O3. However, this should not normally be an issue, as debugging works with other interfaces.

Furthernore I get the warnings/errors in the debug console:

Failed to read peripheral GPIOA: Error: readMemory failed @ 0x48000000 for 24 bytes: CodeExpectedError: Unable to read memory., session=a94c29ac-0a40-4e7a-9ded-79c337f435b2
Failed to read peripheral GPIOA: Error: readMemory failed @ 0x4800001c for 12 bytes: CodeExpectedError: Unable to read memory., session=a94c29ac-0a40-4e7a-9ded-79c337f435b2
Internal error: Failed to update peripheral GPIOA after memory reads: Attempt to access memory outside buffer bounds

where I don't know why and if they have something to do with my issue.

Expected Behavior

Debugger just stops at breakpoint and continues after a click on continue or when stepping further.

Steps To Reproduce

No response

Affected operating systems

• all
• Windows amd64
• Linux amd64
• Linux arm64
• MacOS arm64

Environment

No response

[jreineckearm]

Thanks for raising this, @steradiant .

This needs further investigation but could indeed be related to compiling with -O3 and GDB getting lost due to unexpected debug info as a result of the higher optimization level.

Furthernore I get the warnings/errors in the debug console:

I don't think they are related to the GDB crash, but aren't nice, either. Such errors come from the Peripheral Inspector and can among others show if a peripheral isn't clocked/powered but expanded for display (though #670 suggests you are seeing these in a different scenario). The error messages need improvement, they look a little too scary in their current form.

Can you please help us with some more information about your use case? Is it the same as in #670 ?
Also, have you tried to compile with -O0 or -O1? Appreciate that this may not be a solution for you. But it may help to narrow things down while we try to set up a reproducer.

[steradiant]

Hello @jreineckearm,
Thanks for taking care of this issue. The debugger crashes happened when the debugger hit a breakpoint while using an optimization level higher than -O0 (e.g., also with -O1). And yes, it’s the same setup as in #670.

Over the past few days, I’ve done some more investigations. I probably didn’t emphasize in the original report that I’m running the debugger standalone and not with the Arm Keil Studio pack. My build environment uses Makefiles and the Arm Toolchain for Embedded.

I noticed that I wasn’t using a *.cbuild-run.yml file, which I also didn’t need for other debuggers (like cortex-debug), since the necessary info should be available from sources like the target configuration, startup file, etc. Are there templates for these .yml files, or recommended ways to generate one properly? I tried creating a suitable *.cbuild-run.yml, and for now, this issue (and #670) seems to have disappeared.

However, the following error persists:
Internal error: Failed to update peripheral GPIOC after memory reads: Attempt to access memory outside buffer bounds
This happens for peripherals expanded in the Peripherals view. I’ve seen it for GPIOA and GPIOC, both active on the bus and working fine in my program.

There are also some error messages from VS Code (not in the debugger console) that appear but don’t seem to affect functionality for now.

During debugger startup:

• Unable to read memory
• gdb is not running
• Cannot read properties of undefined (reading 'sendCommand')

During debugging:

• Variable object not found
• Selected thread is running
• Cannot handle variable frame requests while target is running

These don’t occur consistently, and I haven’t found a clear pattern yet.

I’ll continue investigating until I can confidently say the bug is fully resolved.

[jreineckearm]

Thanks for sharing more information, @steradiant . And for also confirming that the problems go away when implementing a *.cbuild-run.yml file. I was suspecting something like debug sequences missing that configure the STM32G474 DBGMCU_CR when I saw your comment about not using the CMSIS Solution extension on #670 . Depending on the running application, this could lead to unexpected failing memory/core register accesses.

Are there templates for these .yml files, or recommended ways to generate one properly?

The recommended way is to let the CMSIS-Toolbox collect all the required information from relevant CMSIS-Packs and a CMSIS solution, and to write the *.cbuild-run.yml file that is consumed by pyOCD.

However, the following error persists:
Internal error: Failed to update peripheral GPIOC after memory reads: Attempt to access memory outside buffer bounds

We need to follow up on this error, this clearly is a bug in the Peripheral Inspector extension.

There are also some error messages from VS Code (not in the debugger console) that appear but don’t seem to affect functionality for now.

I haven't seen the three errors during startup yet. But at least two of the three during debug are known and captured in #566 . They can happen when you step/run the CPU while a VS Code GUI update is in progress. We are currently looking into ways to mitigate them.

[steradiant]

The recommended way is to let the CMSIS-Toolbox collect all the required information from relevant CMSIS-Packs and a CMSIS solution, and to write the *.cbuild-run.yml file that is consumed by pyOCD.

I understand that, but using CMSIS-Toolbox adds quite an overhead for my projects. On top of that, I haven’t found a practical way to get it working with my setup using the Arm Toolchain for Embedded. The description also mentions that the tool can be used in standalone mode. For example, Keil doesn’t require such a configuration file because the necessary information is available elsewhere (likely in the DFP pack). I’m not entirely sure where Keil retrieves this data, but it works without additional configuration.

It would be helpful if the debugger could be used fully standalone because it has many great features! If the missing *.cbuild-run.yml file is indeed the main reason for these issues, it might be worth mentioning this in the documentation. A minimal example with the required fields to get started, along with a link to more detailed guidance for advanced configurations, could make things easier for users.

[jreineckearm]

Thanks for the feedback on the docs. We will have a look if we can make things clearer around the *.cbuild-run.yml usage.

A workaround to make the debugger also work for your use case could be to separately set up one of the example or blank projects with the CMSIS Solution extension, and to extract the generated *.cbuild-run.yml, *.dbgconf, launch.json, and tasks.json files as a starting point. Then you hopefully only need to make minor adjustments to point to your build output. We should already have a variety of examples for your board that you could use:

[jreineckearm]

Lowering priority given usage of *.cbuild-run.yml relaxes impact. Some details in here still need to be addressed though but probably after next release.