Re: [PATCH 7/8] Windows gdb: Implement XState (Intel AVX) support

Hannes Domani <[email protected]>
Newsgroups gmane.comp.gdb.patches
Message-ID <[email protected]>
 Am Freitag, 17. Juli 2026 um 21:06:16 MESZ hat Schimpe, Christina <[email protected]> Folgendes geschrieben:

> > -----Original Message-----
> > From: Hannes Domani <[email protected]>
> > Sent: Freitag, 17. Juli 2026 18:27
> > To: [email protected]; Schimpe, Christina
> > <[email protected]>
> > Subject: Re: [PATCH 7/8] Windows gdb: Implement XState (Intel AVX) support
> >
> >  Am Freitag, 17. Juli 2026 um 17:26:14 MESZ hat Schimpe, Christina
> > <[email protected]> Folgendes geschrieben:
> >
> > > > -----Original Message-----
> > > > From: Hannes Domani <[email protected]>
> > > > Sent: Freitag, 17. Juli 2026 16:50
> > > > To: [email protected]; Schimpe, Christina
> > > > <[email protected]>
> > > > Subject: Re: [PATCH 7/8] Windows gdb: Implement XState (Intel AVX)
> > > > support
> > > >
> > > >  Am Freitag, 17. Juli 2026 um 16:16:31 MESZ hat Schimpe, Christina
> > > ><[email protected]> Folgendes geschrieben:
> > > >
> > > > > > -----Original Message-----
> > > > > > From: Hannes Domani <[email protected]>
> > > > > > Sent: Freitag, 17. Juli 2026 15:47
> > > > > > To: [email protected]; Schimpe, Christina
> > > > > > <[email protected]>
> > > > > > Subject: Re: [PATCH 7/8] Windows gdb: Implement XState (Intel
> > > > > > AVX) support
> > > > > >
> > > > > >  Am Freitag, 17. Juli 2026 um 14:54:59 MESZ hat Schimpe,
> > > > > >Christina <[email protected]> Folgendes geschrieben:
> > > > > >
> > > > > > > Hi Hannes,
> > > > > > >
> > > > > > > Thank you for working on this.
> > > > > > > It appears that this patch uses the same commit message header
> > > > > > > as patch #8. Is it intentional that these are separate commits?
> > > > > >
> > > > > > This one is the gdb part, and #8 is the gdbserver part, that is
> > > > > > stated in the title.
> > > > >
> > > > > Oups, I wonder how I came to that conclusion.  Sorry for that.
> > > > >
> > > > > >
> > > > > > > In any case, IMO, commit messages should not have identical headers.
> > > > > > > Also, this patch seems big enough for a commit message which
> > > > > > > is not header only. :)
> > > > >
> > > > > This part still stands for the commit message. Since it's not only
> > > > > avx registers, I believe it makes sense to share more details here.
> > > >
> > > > Yes, I will do that.
> > > >
> > > >
> > > > > > > Do any AVX-* specific tests pass on Windows now? If so, it
> > > > > > > would be helpful to mention that in the commit message as well.
> > > > > >
> > > > > > I actually planned to add that info, but forgot.
> > > > > >
> > > > > > These tests then pass on Windows:
> > > > > > gdb.arch/i386-avx.exp
> > > > > > gdb.arch/i386-avx512.exp
> > > > >
> > > > > What's the state for SSE  (gdb.arch/i386-sse.exp) ?
> > > >
> > > > i386-sse.exp works since patch #1, it was only a compile issue of
> > > > the test itself.
> > > > I will mention this in the patch as well.
> > >
> > > Thanks.
> > >
> > > >
> > > > > I also saw you introduced some code for PKRU and the shadow stack
> > > > pointer.
> > > > > We have GDB tests for those registers, too:
> > > > > - gdb.arch/i386-pkru.exp
> > > > > - gdb.arch/amd64-shadow-stack*.exp
> > > >
> > > > For these I just added the equivalent code as is done on Linux,
> > > > without really knowing what they are for.
> > > >
> > > > i386-pkru.exp tells me:
> > > >
> > > > (gdb) print have_pkru()
> > > > $1 = 0
> > > > (gdb) PASS: gdb.arch/i386-pkru.exp: probe PKRU support
> > > > UNSUPPORTED: gdb.arch/i386-pkru.exp: processor does not support
> > > > protection key feature.
> > > >
> > > > Do only certain CPU's have this register?
> > >
> > > Most recent CPUs should have it. To be sure you can check if the
> > > corresponding bit is configured in xcr0.
> >
> > I thought that's what have_pkru() does, but I might be wrong about that.
> 
> Yes, this should work for windows, too.
> 
> > > I believe for windows this should be the mask returned by
> > get_xstate_features_mask.
> >
> > You probably mean GetEnabledXStateFeatures, but yes, it also tells me my CPU
> > doesn't support this.
> >
> >
> > > If your cpu does have it, I think you must look at the test in more
> > > detail to find out why it's unsupported. It might need some adaptions for
> > windows.
> > Google tells me only not all recent CPUs have this, is mine one of them?:
> > 11th Gen Intel(R) Core(TM) i7-11850H
> 
> I would have expected this but cannot say for sure.  And I don't know the state for
> Memory Protection Keys windows support.

I have tested this now on another PC, one where have_pkru() returns 1, so
this CPU really should have PKRU.

But GetEnabledXStateFeatures still does not return the PKRU bit, and I can
find no equivalent XSTATE_PKRU define in winnt.h, this makes me believe
that windows just doesn't support this feature.

So I'm fine with removing that part.


> > > > And amd64-shadow-stack.exp:
> > > >
> > > > (gdb) print $pl3_ssp
> > > > $1 = (void *) 0x0
> > > > (gdb) FAIL: gdb.arch/amd64-shadow-stack.exp: test shadow stack
> > > > support
> > > >
> > > > No idea if that should work, since Windows is supplying data for the
> > > > $pl3_ssp register.
> > > >
> > > >
> > > > Hannes
> > >
> > > Without knowing any of the details in windows I believe adding full
> > > support for CET shadow stack might need some more changes. At least for
> > linux, this was the case:
> > > https://inbox.sourceware.org/gdb-patches/20250821171029.1555603-1-chri
> > > [email protected]/
> > >
> > > I believe it would be better to handle this in a separate patch-(series).
> > > This also applies for any other register that you add and whose dedicated
> > test does not pass.
> > >
> > > Does that make sense to you?
> >
> > I was wondering if it isn't the compiler or linker on windows (or maybe
> > windows itself?) that doesn't support this -fcf-protection=return stuff.
> 
> I don't know the state for windows but would be curious about it. :-)
> At least for Kernel Mode, I could find some documentation for it.

I did some more tests here as well.

It turns out binutils/gcc really don't support this on windows yet.
To enable this in the executable, at least a special marker bit has to be
set in the Extended DLL Characteristics [1].
With msvc you can do that with the /CETCOMPAT linker argument [2].

So I compiled amd64-shadow-stack.c with msvc and /CETCOMPAT, and tried it
out with my gdb build.
Before I always got $pl3_ssp=0, and with this new exe I always get $pl3_ssp=1.

So I had this wrong, because LocateXStateFeature returns for XSTATE_CET_U
this struct:
    typedef struct _XSAVE_CET_U_FORMAT {
      DWORD64 Ia32CetUMsr;
      DWORD64 Ia32Pl3SspMsr;
    } XSAVE_CET_U_FORMAT, *PXSAVE_CET_U_FORMAT;

So $pl3_ssp showed Ia32CetUMsr, when it should have shown Ia32Pl3SspMsr
instead.

With that fixed, I tried again debugging the /CETCOMPAT executable:

(gdb) r
Starting program: C:\qiewer\git\amd64-shadow-stack.exe

Breakpoint 1, 0x00007ff725812e41 in ?? ()
1: $pl3_ssp = (void *) 0x4feff0
(gdb) x/1i $pc
=> 0x7ff725812e41:      jmp    0x7ff725817680
(gdb) x/4a $pl3_ssp
0x4feff0:       0x7ffd62fee957 <KERNEL32!BaseThreadInitThunk+23>        0x7ffd63647c1c <ntdll!RtlUserThreadStart+44>
0x4ff000:       Cannot access memory at address 0x4ff000
(gdb) si
...
(gdb) si
0x00007ff725817684 in ?? ()
1: $pl3_ssp = (void *) 0x4feff0
(gdb) x/1i $pc
=> 0x7ff725817684:      call   0x7ff725812f1d
(gdb) si
0x00007ff725812f1d in ?? ()
1: $pl3_ssp = (void *) 0x4fefe8
(gdb) x/4a $pl3_ssp
0x4fefe8:       0x7ff725817689  0x7ffd62fee957 <KERNEL32!BaseThreadInitThunk+23>
0x4feff8:       0x7ffd63647c1c <ntdll!RtlUserThreadStart+44>    Cannot access memory at address 0x4ff000
(gdb) x/1i $pc
=> 0x7ff725812f1d:      jmp    0x7ff725817c30
(gdb) si
...
(gdb) si
0x00007ff725817cde in ?? ()
1: $pl3_ssp = (void *) 0x4fefe8
(gdb) x/1i $pc
=> 0x7ff725817cde:      ret
(gdb) si
0x00007ff725817689 in ?? ()
1: $pl3_ssp = (void *) 0x4feff0
(gdb) x/1i $pc
=> 0x7ff725817689:      add    $0x28,%rsp
(gdb) x/4a $pl3_ssp
0x4feff0:       0x7ffd62fee957 <KERNEL32!BaseThreadInitThunk+23>        0x7ffd63647c1c <ntdll!RtlUserThreadStart+44>
0x4ff000:       Cannot access memory at address 0x4ff000
(gdb)

call and ret modify the shadow stack which is pointed to by $pl3_ssp, is that
how it's supposed to work?
I'm unsure, because amd64-shadow-stack-cmds.exp makes it look like $pl3_ssp
should change when moving up or down the frames.

Is $pl3_ssp maybe supposed to provide Ia32Pl3SspMsr[$frame] instead?
How does that work on linux?

In any case, I tried modifying $pl3_ssp like it is done in
amd64-shadow-stack.exp.

(gdb) p $pl3_ssp
$10 = (void *) 0x4fefe8
(gdb) p $pl3_ssp=0x12345678
$11 = (void *) 0x12345678
(gdb) ni
error return C:/gdb/src/gdb.git/gdb/x86-windows-nat.c:242 was 1660: The thread context could not be updated because this has been restricted for the process.
0x00007ffd63720574 in ntdll!ZwMapViewOfSection () from C:\WINDOWS\SYSTEM32\ntdll.dll

So windows doesn't allow modifying the shadow stack pointer.

(gdb) p $pl3_ssp
$14 = (void *) 0x4fefe8
(gdb) p ((void**)$pl3_ssp)[0]
$15 = (void *) 0x7ff725817689
(gdb) p ((void**)$pl3_ssp)[0]=0x12345678
Cannot access memory at address 0x4fefe8

And looks like modifying the shadow stack entries is also prohibited.


[1] https://learn.microsoft.com/en-us/windows/win32/debug/pe-format#extended-dll-characteristics
[2] https://learn.microsoft.com/en-us/cpp/build/reference/cetcompat


Hannes
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.