Re: evl: ptrace continue without hardware singlestepping support

Tobias Schaffner <[email protected]> writes:

> Hi Philippe,
>
> the RISC-V port has an open issue regarding OOB synchronization of
> threads in the EVL ptrace implementation, which leads to a deadlock
> when running the ptrace-sync test.
>
> RISC-V currently does not support hardware-based single stepping, and
> GDB’s behavior on `continue` with multiple threads is different.
>
> If hardware single stepping is available, GDB single-steps the hitter
> with PTRACE_SINGLESTEP and continues the other threads with
> PTRACE_CONT. All threads are continued and can pass the
> evl_wait_event() barrier in handle_ptstop_event(). The hitter breaks
> again and is finally continued.
>
> In contrast, if hardware single stepping is not available, GDB adds a
> breakpoint to the next instruction, wakes only the hitter with
> PTRACE_CONT, and waits for it to make its step before waking its
> siblings. However, the hitter is not able to make its step because it
> waits in the kernel for its siblings in evl_wait_event(), which are
> not going to be woken by GDB until the hitter makes its step.
>
> The attached strace files show the behavior of GDB on both
> architectures when continuing the breaking worker thread in
> `ptrace.c`.
>
> Jan already pointed out that there may be a general problem with
> expecting GDB to always continue all threads, and I think he is
> right. Changing the scheduler-locking mode in All-Stop Mode [1], as
> well as switching to Non-Stop Mode [2], should lead to similar issues.
>

Stop mode would be an issue if the synchronization fully happened on
PTCONT, but the evl core requires threads to wait on the sync barrier
only if they received PTSTOP in the first place. Did you observe any
actual issue with enabling non-stop mode by testing on your end? I just
tested with ptrace-sync and observed the right behavior. This does not
preclude the issue observed with sw breakpoints though, but this would
be a different problem.

root@corei7-64-evl:~# __EVL_DEBUGGEE__=1 gdb /usr/libexec/evl/tests/ptrace-sync 
GNU gdb (GDB) 16.3
Copyright (C) 2024 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-blade-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/libexec/evl/tests/ptrace-sync...
(gdb) b bp
warning: could not convert 'bp' from the host encoding (ANSI_X3.4-1968) to UTF-32.
This normally should not happen, please file a bug report.
Breakpoint 1 at 0x26db: bp. (3 locations)
(gdb) set non-stop on
(gdb) r
Starting program: /usr/libexec/evl/tests/ptrace-sync 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/libthread_db.so.1".
[New Thread 0x7ffff7d166c0 (LWP 595)]
[New Thread 0x7ffff7d056c0 (LWP 596)]

Thread 1 "gdb-main:592" hit Breakpoint 1.1, bp () at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
warning: 59	../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c: No such file or directory
(gdb) info thr
  Id   Target Id                                        Frame 
* 1    Thread 0x7ffff7d17b40 (LWP 592) "gdb-main:592"   bp ()
    at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
  2    Thread 0x7ffff7d166c0 (LWP 595) "gdb-test:592.1" (running)
  3    Thread 0x7ffff7d056c0 (LWP 596) "gdb-test:592.2" (running)
(gdb) c
Continuing.

Thread 3 "gdb-test:592.2" hit Breakpoint 1.3, bp ()
    at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
59	in ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c
(gdb) info thr
  Id   Target Id                                        Frame 
* 1    Thread 0x7ffff7d17b40 (LWP 592) "gdb-main:592"   (running)
  2    Thread 0x7ffff7d166c0 (LWP 595) "gdb-test:592.1" (running)
  3    Thread 0x7ffff7d056c0 (LWP 596) "gdb-test:592.2" bp ()
    at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
(gdb) thr 3
[Switching to thread 3 (Thread 0x7ffff7d056c0 (LWP 596))]
#0  bp () at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
59	in ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c
(gdb) c
Continuing.

Thread 2 "gdb-test:592.1" hit Breakpoint 1.3, bp ()
    at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
59	in ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c
(gdb) info thr
  Id   Target Id                                        Frame 
  1    Thread 0x7ffff7d17b40 (LWP 592) "gdb-main:592"   (running)
  2    Thread 0x7ffff7d166c0 (LWP 595) "gdb-test:592.1" bp ()
    at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
* 3    Thread 0x7ffff7d056c0 (LWP 596) "gdb-test:592.2" (running)
(gdb) thr 2
[Switching to thread 2 (Thread 0x7ffff7d166c0 (LWP 595))]
#0  bp () at ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c:59
59	in ../../../../git/xenomai/v4/libevl/tests/ptrace-sync.c
(gdb) c
Continuing.
[Thread 0x7ffff7d056c0 (LWP 596) exited]
[Thread 0x7ffff7d166c0 (LWP 595) exited]
[Inferior 1 (process 592) exited normally]

> Do you have a suggestion how to solve this?
>

Regarding sw breakpoints, there may be two options based on your
findings:

- defer the PTCONT notification to the real-time core
  (inband_ptcont_notify) until after the ptracer notices the wake up of
  the debuggee, allowing it to wake up the siblings in turn. Currently,
  PTSTOP/PTCONT are sent by the debuggee around its stop point.

- check whether we could monitor the PTSTEP event which is sent by the
  ptracer when it resumes a debuggee.

I'll have a look at this.

-- 
Philippe.