Re: Debugging issue with gdbserver and a daemon on the target

Laszlo Papp <[email protected]>
Newsgroups gmane.comp.gdb.devel
Message-ID <CAOMwXhOQkPH-6By+C0zcu3RAQMhWypC18ZsFrLK+ncdAcq3b0Q@mail.gmail.com>
On Tue, Aug 19, 2014 at 5:12 PM, Pedro Alves <[email protected]> wrote:
> On 08/19/2014 05:02 PM, Laszlo Papp wrote:
>> On Tue, Aug 19, 2014 at 4:58 PM, Pedro Alves <[email protected]> wrote:
>>> On 08/19/2014 04:44 PM, Laszlo Papp wrote:
>>>
>>>>
>>>> gdbserver --attach 192.168.0.32:2345 pid-of-my-daemon
>>>>
>>>
>>>> (gdb) bt
>>>> #0  0x44ad26ec in select () at ../sysdeps/unix/syscall-template.S:81
>>>> #1  0x0002ac08 in bar (timeout=10, name=0x42f30 <yy_ec+56> "foo") at
>>>> src/socket.c:906
>>>> #2  0x0003284c in main (argc=0, argv=0x0) at src/bar.c:679
>>>> (gdb)
>>>>
>>>>
>>>> ... And then I do some communication with the daemon where the foo
>>>> function is executed based on the logs, but the breakpoint is not hit.
>>>> I wished to try hardware breakpoints, but they are not presented on my
>>>> hardware.
>>>>
>>>> Furthermore, if I use the same workflow on a binary that is
>>>> "one-shot", i.e. not running continuously as a daemon, the debugging
>>>> workflow for stopping at main works with exactly the aforementioned
>>>> software breakpoint issue.
>>>>
>>>> I am completely clueless at this point. Do you know how I can debug a
>>>> daemon with gdbserver?
>>>
>>> "daemon" and "select" makes me think "fork".  If the daemon is handling
>>> requests by forking a child, and then it's the child that calls 'foo',
>>> then this is expected, as GDBserver doesn't know how to follow forks
>>> currently.  It's WIP, patches have been posted.  Meanwhile, the usual
>>> thing to do it to attach to the child process the daemon spawns instead
>>> of the main daemon pid.  You'll usually do that by adding a busyloop in
>>> the child somewhere, like:
>>>
>>> volatile int gdb_here;
>>>
>>> while (!gdb_here)
>>>   sleep (1);
>>>
>>> and after attaching to the child, do "print gdb_here = 1; continue".
>>>
>>> Thanks,
>>> Pedro Alves
>>
>> Thanks Pedro for the prompt reply. Unfortunately, I am already
>> attaching to the child right after the fork. I wonder if this can
>> happen if some source file was missing?
>
> It shouldn't.  Source files are only used for display.  Where to
> place a breakpoint is derived from the debug info in the binary.
>
> I'd suggest just trying to step through the code instead of
> putting a break at "foo", and see if that much works.  On an
> arm system, stepping is actually implemented with magic
> breakpoints behind the scenes.
>
>> Btw:
>>
>> gdbserver --version
>> GNU gdbserver (GDB) 7.5.1
>>
>> arm-polatis-linux-gnueabi-gdb --version
>> GNU gdb (GDB) 7.5.1
>>
>
> Knee-jerk reaction is to suggest a more recent GDB/GDBserver.  Note
> building these isn't very hard.  There aren't that many
> dependencies.
>
> Thanks,
> Pedro Alves

Hmm, it seems that the stripped binary on the target and the one on
the host were out-of-sync. This is really strange since I have not
changed the source code. Seems different compilations still can get
out-of-sync for the same code so that when I rebuild the same source
code, I always need to update the binary on the target, too?

Anyway, now I only have problems with finding the sources file to view
them in cgdb. I do not know why it is wrong, but it seems to be. As
you can see the paths are set up for dwarf correctly:

readelf --debug-dump
./tmp/work/foo-foo-linux-gnueabi/foo-core-image-dbg/1.0-r0/rootfs/usr/bin/.debug/foo
| grep DW_AT_comp_dir
    <35>   DW_AT_comp_dir    :
/usr/src/debug///////////////////////////////////////////////////////////////////////////////////eglibc/2.17-r3/eglibc-2.17/libc/csu
    <df>   DW_AT_comp_dir    : (indirect string, offset: 0x31):
/usr/src/debug/eglibc/2.17-r3/eglibc-2.17/libc/csu
    <183>   DW_AT_comp_dir    :
/usr/src/debug///////////////////////////////////////////////////////////////////////////////////eglibc/2.17-r3/eglibc-2.17/libc/csu
    <22d>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <102f>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <2e98>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <3e34>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <471a>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <4c5f>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <55b9>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <84b8>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <94d1>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <a1e9>   DW_AT_comp_dir    : (indirect string, offset: 0x749):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/bar
    <af53>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <c382>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <c952>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <d281>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <dcc0>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <e093>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <e953>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <efd6>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <fb07>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <10cd9>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <127e6>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <132ab>   DW_AT_comp_dir    : (indirect string, offset: 0x3874):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/baz
    <13bf3>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <1556d>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <162d4>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <17481>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <17900>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <18481>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <1903f>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <19432>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <19524>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <19fe1>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <1d2b4>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <1dc34>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <1f2c1>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <1fa4f>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <1fe13>   DW_AT_comp_dir    : (indirect string, offset: 0x585a):
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
    <203e4>   DW_AT_comp_dir    : (indirect string, offset: 0x7cfa):
/usr/src/debug/flex/2.5.35-r3/flex-2.5.35
    <2042a>   DW_AT_comp_dir    : (indirect string, offset: 0x31):
/usr/src/debug/eglibc/2.17-r3/eglibc-2.17/libc/csu
    <2056c>   DW_AT_comp_dir    : (indirect string, offset: 0x7de0):
/usr/src/debug/eglibc/2.17-r3/eglibc-2.17/libc/io
    <20833>   DW_AT_comp_dir    :
/usr/src/debug///////////////////////////////////////////////////////////////////////////////////eglibc/2.17-r3/eglibc-2.17/libc/csu
    DW_AT_comp_dir     DW_FORM_string
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_string
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_strp
    DW_AT_comp_dir     DW_FORM_string

... yet, gdb says src/bar.c cannot be found even though it should be
in the aforementioned
/usr/src/debug/foo-git/AUTOINC+0c2cbe33e653afa335ca25156d293552001228fa-r0/git/meh
path, provided my sysroot setting is good above, but if that was not
good, it would not load the binaries anyway, right? So, if I set that
path one line above with the "-d" option to gdb, then the source file
can be viewed. What may be going on here?

Thanks in advance. I am so desperately lost. :(
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.