Re: How to get a stack trace in RTEMS

Stanislav Pankevich <[email protected]> Thu, 27 Jun 2024 10:27:03 +0200
Newsgroups gmane.os.rtems.user
Message-ID <CAFXpGYYopek1XSzUMpvrBYGeZJC_cxFJvuoH6CuWHJdg6B3DMg@mail.gmail.com>
--===============3971870233889480829==
Content-Type: multipart/alternative; boundary="000000000000f7e8ad061bdae5f3"

--000000000000f7e8ad061bdae5f3
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

A small further update here to support the discussion further. It turns out
that there is an undocumented function that, in contrast to
_Unwind_Backtrace, allows passing a given stack:

*    In my practice standard _Unwind_Backtrace failed to switch to
pre-signal stack. I've managed to get some before-signal stacks by calling
internal libgcc __gnu_Unwind_Backtrace - it has an extra argument being
"current registry values" - so it operates on given stack, not on current
stack.*
https://stackoverflow.com/a/30515756/598057

The challenge with this is how to access the crashed stack frame, i.e., the
right context has to be passed to __gnu_Unwind_Backtrace.

    _Unwind_Reason_Code
    __gnu_Unwind_Backtrace(_Unwind_Trace_Fn trace, void *
trace_argument, phase2_vrs * entry_vrs);
    ...
    pre_signal_state.core =3D *reinterpret_cast<const
core_regs*>(&(signal_context->uc_mcontext.arm_r0));
    __gnu_Unwind_Backtrace(tracer, &state, &pre_signal_state);


In this SO example, they are using the signal_context while what we have in
the RTEMS, for example in bsp_fatal_extension() is:

    const rtems_exception_frame *exception_frame =3D (const
rtems_exception_frame *) code;

Could anyone advise on how we access the original crashed stack trace from
the code variable? We confirm that _Unwind_Backtrace can work on both Zynq
UltraScale+ and TMS570. It would be great to get it or
__gnu_Unwind_Backtrace to work with the real user frames.

Thanks,
Stanislav Pankevich

On Wed, Jun 26, 2024 at 4:14=E2=80=AFPM Stanislav Pankevich <s.pankevich@gm=
ail.com>
wrote:

> Hi Sebastian,
>
> We had to add -funwind-tables to have this trace() called, and we can now
> see the function names as follows. The challenge is that the functions of
> the exception handler are printed, not the code that led to the exception=
.
> Now we need to figure out how to switch this code to print our function
> names instead.
>
> FUNC NAME: bsp_fatal_extension
> FUNC NAME: _User_extensions_Iterate
> FUNC NAME: _Terminate
> FUNC NAME: _ARM_Exception_default
>
> static _Unwind_Reason_Code trace(_Unwind_Context *ctx, void *arg)
> {
>     (void)arg;
>
>     const uint32_t func_entry  =3D _Unwind_GetRegionStart(ctx);
>
>     char          *func_name =3D "??";
>     uint8_t       *canary    =3D (uint8_t *)(func_entry - 0x4);
>     const uint32_t len       =3D (*((uint32_t *)(func_entry - 0x4))) &
> 0x00FFFFFF;
>     if (*canary =3D=3D 0xFF)
>     {
>         func_name =3D (char *)((uint32_t)canary - len);
>         *canary   =3D '\0';
>     }
>     printk("FUNC NAME: %s\n", func_name);
>
>     return _URC_NO_REASON;
> }
>
> Thanks,
> Stanislav Pankevich
>
> On Wed, Jun 26, 2024 at 11:25=E2=80=AFAM Sebastian Huber <
> sebastian.huber-L1vi/[email protected]> wrote:
>
>> Hello Stanislav,
>>
>> On 26.06.24 11:15, Stanislav Pankevich wrote:
>> > Dear RTEMS community,
>> >
>> > My colleague George cannot register to post to this forum, getting 403
>> > when subscribing at https://lists.rtems.org/mailman/listinfo/users
>> > <https://lists.rtems.org/mailman/listinfo/users>. I am posting this
>> > question on his behalf.
>> >
>> > ---
>> >
>> > We are currently seeing an RTEMS_FATAL_SOURCE_EXCEPTION right at the
>> > start of execution and can debug the program once the exception handle=
r
>> > has been reached. From here we can inspect the call stack however we
>> > only see the trace that led to the final function call (void
>> > bsp_reset(void) while loop) that handles the exception, not the trace
>> > that caused the exception. We would like to implement an exception
>> > handler that prints the trace that caused the exception to be
>> triggered.
>> > Is this something that is already implemented in RTEMS? We have
>> > developed an RTEMS port to the Zynq UltraScale+ MPSoC Cortex R5 RPU
>> > running in lockstep. Code execution is from the shared DDR4 memory wit=
h
>> > a 256MB region allocated.
>>
>> I have some improvements for this BSP in my patch queue. It adds support
>> for the split mode for example.
>>
>> >
>> > What we want is to have the actual stack trace to be printed to the
>> console.
>> >
>> > Here is an example of what we have now in GDB:
>> >
>> > bsp_reset@0x0017b062
>> > (.../rtems/bsps/arm/xilinx-zynqmp-rpu/start/bspreset.c:40)
>> > _User_extensions_Iterate@0x00186854
>> > (.../rtems/cpukit/score/src/userextiterate.c:194)
>> > _User_extensions_Fatal@0x00183108
>> > (.../rtems/cpukit/include/rtems/score/userextimpl.h:467)
>> > _Terminate@0x00183108 (.../rtems/cpukit/score/src/interr.c:55)
>> > rtems_fatal@0x0018adc2 (.../rtems/cpukit/include/rtems/fatal.h:160)
>> > _ARM_Exception_default@0x0018adc2
>> > (.../rtems/cpukit/score/cpu/arm/arm-exception-default.c:37)
>> > save_more_context@0x00186cfc
>> > (.../rtems/cpukit/score/cpu/arm/armv4-exception-default.S:176)
>> >
>> > This is currently what is printed on the serial port:
>> >
>> > *** FATAL ***
>> > fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)
>> >
>> > R0   =3D 0x00000056 R8  =3D 0x00ac8aec
>> > R1   =3D 0xf000e3a0 R9  =3D 0x00ac8a5c
>> > R2   =3D 0x00000020 R10 =3D 0x00000000
>> > R3   =3D 0x00000020 R11 =3D 0x00000000
>> > R4   =3D 0x00000056 R12 =3D 0x010e6630
>> > R5   =3D 0x010e67c0 SP  =3D 0x010e65d4
>> > R6   =3D 0x010e6698 LR  =3D 0x001854ae
>> > R7   =3D 0x001867f5 PC  =3D 0x001854ae
>> > CPSR =3D 0x200f01f7 VEC =3D 0x00000004
>> > RTEMS version: 6.0.0.62f15c07482dd953663143554e78b4bf287ccb83
>> > RTEMS tools: 12.2.1 20230224 (RTEMS 6, RSB
>> > 4c73a76c802588d3864c64ee4dd48a84b953201a, Newlib 17ac400)
>>
>> Getting stack traces on arm is not that easy since you need a
>> description of the stack frames. You can try to use the unwinder from
>> libgcc:
>>
>> static _Unwind_Reason_Code trace(_Unwind_Context *ctx, void *arg)
>> {
>>    (void)arg;
>>    printk("%08" PRIuPTR "\n", (uintptr_t)_Unwind_GetIP(ctx));
>>    return _URC_NO_REASON;
>> }
>>
>> static void f(int i);
>>
>> static void h(int i)
>> {
>>    f(i + 1);
>> }
>>
>> static void f(int i)
>> {
>>    if (i > 10) {
>>      _Unwind_Backtrace(trace, NULL);
>>    } else {
>>       h(i + 1);
>>    }
>> }
>>
>> --
>> embedded brains GmbH & Co. KG
>> Herr Sebastian HUBER
>> Dornierstr. 4
>> 82178 Puchheim
>> Germany
>> email: sebastian.huber-L1vi/[email protected]
>> phone: +49-89-18 94 741 - 16
>> fax:   +49-89-18 94 741 - 08
>>
>> Registergericht: Amtsgericht M=C3=BCnchen
>> Registernummer: HRB 157899
>> Vertretungsberechtigte Gesch=C3=A4ftsf=C3=BChrer: Peter Rasmussen, Thoma=
s D=C3=B6rfler
>> Unsere Datenschutzerkl=C3=A4rung finden Sie hier:
>> https://embedded-brains.de/datenschutzerklaerung/
>>
>

--000000000000f7e8ad061bdae5f3
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div>A small further update here to suppo=
rt the discussion further. It turns out that there is an undocumented funct=
ion that, in contrast to _Unwind_Backtrace, allows passing a given stack:</=
div><div></div><div><p><i>=C2=A0=C2=A0=C2=A0 In my practice standard _Unwin=
d_Backtrace failed to switch to pre-signal stack. I&#39;ve managed to get s=
ome before-signal stacks by calling internal=20
libgcc __gnu_Unwind_Backtrace - it has an extra argument being &quot;curren=
t=20
registry values&quot; - so it operates on given stack, not on current stack=
.</i></p></div><div></div><div><a href=3D"https://stackoverflow.com/a/30515=
756/598057">https://stackoverflow.com/a/30515756/598057</a></div><div><br><=
/div><div>The challenge with this is how to access the crashed stack frame,=
 i.e., the right context has to be passed to __gnu_Unwind_Backtrace.</div><=
div><pre class=3D"gmail-default gmail-s-code-block"><code class=3D"gmail-hl=
js gmail-language-cpp">    _Unwind_Reason_Code
    __gnu_Unwind_Backtrace(_Unwind_Trace_Fn trace, <span class=3D"gmail-hlj=
s-type">void</span> * trace_argument, phase2_vrs * entry_vrs);</code><code =
class=3D"gmail-hljs gmail-language-cpp">      <br>    ...<br>    pre_signal=
_state.core =3D *<span class=3D"gmail-hljs-built_in">reinterpret_cast</span=
>&lt;<span class=3D"gmail-hljs-type">const</span> core_regs*&gt;(&amp;(sign=
al_context-&gt;uc_mcontext.arm_r0));
    __gnu_Unwind_Backtrace(tracer, &amp;state, &amp;pre_signal_state);</cod=
e></pre></div><div><br></div><div>In this SO example, they are using the si=
gnal_context while what we have in the RTEMS, for example in bsp_fatal_exte=
nsion() is:</div><div><br></div><div>=C2=A0=C2=A0=C2=A0 <span style=3D"font=
-family:monospace">const rtems_exception_frame *exception_frame =3D (const =
rtems_exception_frame *) code;</span></div><div><br></div></div><div>Could =
anyone advise on how we access the original crashed stack trace from the <f=
ont size=3D"2"><span style=3D"font-family:monospace">code</span></font> var=
iable? We confirm that _Unwind_Backtrace can work on both Zynq UltraScale+ =
and TMS570. It would be great to get it or=C2=A0<span style=3D"font-family:=
monospace">__gnu_Unwind_Backtrace</span> to work with the real user frames.=
<br></div><div><br></div><div>Thanks,</div><div>Stanislav Pankevich<br></di=
v><div><br></div><div><div><div><div class=3D"gmail_quote"><div dir=3D"ltr"=
 class=3D"gmail_attr">On Wed, Jun 26, 2024 at 4:14=E2=80=AFPM Stanislav Pan=
kevich &lt;<a href=3D"mailto:[email protected]">[email protected]</=
a>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0p=
x 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><d=
iv dir=3D"ltr"><div>Hi Sebastian,</div><div><br></div><div>We had to add -f=
unwind-tables to have this trace() called, and we can now see the function =
names as follows. The challenge is that the functions of the exception hand=
ler are printed, not the code that led to the exception. Now we need to fig=
ure out how to switch this code to print our function names instead.<br></d=
iv><div><br></div><div>FUNC NAME: bsp_fatal_extension<br>FUNC NAME: _User_e=
xtensions_Iterate<br>FUNC NAME: _Terminate<br>FUNC NAME: _ARM_Exception_def=
ault<br><br></div><div>static _Unwind_Reason_Code trace(_Unwind_Context *ct=
x, void *arg)<br>{<br>=C2=A0 =C2=A0 (void)arg;<br><br>=C2=A0 =C2=A0 const u=
int32_t func_entry =C2=A0=3D _Unwind_GetRegionStart(ctx);<br><br>=C2=A0 =C2=
=A0 char =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0*func_name =3D &quot;??&quot;;<b=
r>=C2=A0 =C2=A0 uint8_t =C2=A0 =C2=A0 =C2=A0 *canary =C2=A0 =C2=A0=3D (uint=
8_t *)(func_entry - 0x4);<br>=C2=A0 =C2=A0 const uint32_t len =C2=A0 =C2=A0=
 =C2=A0 =3D (*((uint32_t *)(func_entry - 0x4))) &amp; 0x00FFFFFF;<br>=C2=A0=
 =C2=A0 if (*canary =3D=3D 0xFF)<br>=C2=A0 =C2=A0 {<br>=C2=A0 =C2=A0 =C2=A0=
 =C2=A0 func_name =3D (char *)((uint32_t)canary - len);<br>=C2=A0 =C2=A0 =
=C2=A0 =C2=A0 *canary =C2=A0 =3D &#39;\0&#39;;<br>=C2=A0 =C2=A0 }<br>=C2=A0=
 =C2=A0 printk(&quot;FUNC NAME: %s\n&quot;, func_name);<br><br>=C2=A0 =C2=
=A0 return _URC_NO_REASON;<br>}</div><div><br></div><div>Thanks,</div><div>=
Stanislav Pankevich<br></div></div><br><div class=3D"gmail_quote"><div dir=
=3D"ltr" class=3D"gmail_attr">On Wed, Jun 26, 2024 at 11:25=E2=80=AFAM Seba=
stian Huber &lt;<a href=3D"mailto:sebastian.huber-L1vi/[email protected]" targe=
t=3D"_blank">sebastian.huber-L1vi/[email protected]</a>&gt; wrote:<br></div><bl=
ockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-lef=
t:1px solid rgb(204,204,204);padding-left:1ex">Hello Stanislav,<br>
<br>
On 26.06.24 11:15, Stanislav Pankevich wrote:<br>
&gt; Dear RTEMS community,<br>
&gt; <br>
&gt; My colleague George cannot register to post to this forum, getting 403=
 <br>
&gt; when subscribing at <a href=3D"https://lists.rtems.org/mailman/listinf=
o/users" rel=3D"noreferrer" target=3D"_blank">https://lists.rtems.org/mailm=
an/listinfo/users</a> <br>
&gt; &lt;<a href=3D"https://lists.rtems.org/mailman/listinfo/users" rel=3D"=
noreferrer" target=3D"_blank">https://lists.rtems.org/mailman/listinfo/user=
s</a>&gt;. I am posting this <br>
&gt; question on his behalf.<br>
&gt; <br>
&gt; ---<br>
&gt; <br>
&gt; We are currently seeing an RTEMS_FATAL_SOURCE_EXCEPTION right at the <=
br>
&gt; start of execution and can debug the program once the exception handle=
r <br>
&gt; has been reached. From here we can inspect the call stack however we <=
br>
&gt; only see the trace that led to the final function call (void <br>
&gt; bsp_reset(void) while loop) that handles the exception, not the trace =
<br>
&gt; that caused the exception. We would like to implement an exception <br=
>
&gt; handler that prints the trace that caused the exception to be triggere=
d. <br>
&gt; Is this something that is already implemented in RTEMS? We have <br>
&gt; developed an RTEMS port to the Zynq UltraScale+ MPSoC Cortex R5 RPU <b=
r>
&gt; running in lockstep. Code execution is from the shared DDR4 memory wit=
h <br>
&gt; a 256MB region allocated.<br>
<br>
I have some improvements for this BSP in my patch queue. It adds support <b=
r>
for the split mode for example.<br>
<br>
&gt; <br>
&gt; What we want is to have the actual stack trace to be printed to the co=
nsole.<br>
&gt; <br>
&gt; Here is an example of what we have now in GDB:<br>
&gt; <br>
&gt; bsp_reset@0x0017b062 <br>
&gt; (.../rtems/bsps/arm/xilinx-zynqmp-rpu/start/bspreset.c:40)<br>
&gt; _User_extensions_Iterate@0x00186854 <br>
&gt; (.../rtems/cpukit/score/src/userextiterate.c:194)<br>
&gt; _User_extensions_Fatal@0x00183108 <br>
&gt; (.../rtems/cpukit/include/rtems/score/userextimpl.h:467)<br>
&gt; _Terminate@0x00183108 (.../rtems/cpukit/score/src/interr.c:55)<br>
&gt; rtems_fatal@0x0018adc2 (.../rtems/cpukit/include/rtems/fatal.h:160)<br=
>
&gt; _ARM_Exception_default@0x0018adc2 <br>
&gt; (.../rtems/cpukit/score/cpu/arm/arm-exception-default.c:37)<br>
&gt; save_more_context@0x00186cfc <br>
&gt; (.../rtems/cpukit/score/cpu/arm/armv4-exception-default.S:176)<br>
&gt; <br>
&gt; This is currently what is printed on the serial port:<br>
&gt; <br>
&gt; *** FATAL ***<br>
&gt; fatal source: 9 (RTEMS_FATAL_SOURCE_EXCEPTION)<br>
&gt; <br>
&gt; R0 =C2=A0 =3D 0x00000056 R8 =C2=A0=3D 0x00ac8aec<br>
&gt; R1 =C2=A0 =3D 0xf000e3a0 R9 =C2=A0=3D 0x00ac8a5c<br>
&gt; R2 =C2=A0 =3D 0x00000020 R10 =3D 0x00000000<br>
&gt; R3 =C2=A0 =3D 0x00000020 R11 =3D 0x00000000<br>
&gt; R4 =C2=A0 =3D 0x00000056 R12 =3D 0x010e6630<br>
&gt; R5 =C2=A0 =3D 0x010e67c0 SP =C2=A0=3D 0x010e65d4<br>
&gt; R6 =C2=A0 =3D 0x010e6698 LR =C2=A0=3D 0x001854ae<br>
&gt; R7 =C2=A0 =3D 0x001867f5 PC =C2=A0=3D 0x001854ae<br>
&gt; CPSR =3D 0x200f01f7 VEC =3D 0x00000004<br>
&gt; RTEMS version: 6.0.0.62f15c07482dd953663143554e78b4bf287ccb83<br>
&gt; RTEMS tools: 12.2.1 20230224 (RTEMS 6, RSB <br>
&gt; 4c73a76c802588d3864c64ee4dd48a84b953201a, Newlib 17ac400)<br>
<br>
Getting stack traces on arm is not that easy since you need a <br>
description of the stack frames. You can try to use the unwinder from <br>
libgcc:<br>
<br>
static _Unwind_Reason_Code trace(_Unwind_Context *ctx, void *arg)<br>
{<br>
=C2=A0 =C2=A0(void)arg;<br>
=C2=A0 =C2=A0printk(&quot;%08&quot; PRIuPTR &quot;\n&quot;, (uintptr_t)_Unw=
ind_GetIP(ctx));<br>
=C2=A0 =C2=A0return _URC_NO_REASON;<br>
}<br>
<br>
static void f(int i);<br>
<br>
static void h(int i)<br>
{<br>
=C2=A0 =C2=A0f(i + 1);<br>
}<br>
<br>
static void f(int i)<br>
{<br>
=C2=A0 =C2=A0if (i &gt; 10) {<br>
=C2=A0 =C2=A0 =C2=A0_Unwind_Backtrace(trace, NULL);<br>
=C2=A0 =C2=A0} else {<br>
=C2=A0 =C2=A0 =C2=A0 h(i + 1);<br>
=C2=A0 =C2=A0}<br>
}<br>
<br>
-- <br>
embedded brains GmbH &amp; Co. KG<br>
Herr Sebastian HUBER<br>
Dornierstr. 4<br>
82178 Puchheim<br>
Germany<br>
email: <a href=3D"mailto:sebastian.huber-L1vi/[email protected]" target=3D"_bla=
nk">sebastian.huber-L1vi/[email protected]</a><br>
phone: +49-89-18 94 741 - 16<br>
fax:=C2=A0 =C2=A0+49-89-18 94 741 - 08<br>
<br>
Registergericht: Amtsgericht M=C3=BCnchen<br>
Registernummer: HRB 157899<br>
Vertretungsberechtigte Gesch=C3=A4ftsf=C3=BChrer: Peter Rasmussen, Thomas D=
=C3=B6rfler<br>
Unsere Datenschutzerkl=C3=A4rung finden Sie hier:<br>
<a href=3D"https://embedded-brains.de/datenschutzerklaerung/" rel=3D"norefe=
rrer" target=3D"_blank">https://embedded-brains.de/datenschutzerklaerung/</=
a><br>
</blockquote></div>
</blockquote></div></div></div></div></div>

--000000000000f7e8ad061bdae5f3--

--===============3971870233889480829==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
users mailing list
[email protected]
http://lists.rtems.org/mailman/listinfo/users
--===============3971870233889480829==--