Re: Mesa 3D driver for custom GPU hardware?
Dylan Barrie <[email protected]> Fri, 29 May 2026 10:13:39 -0700
| Newsgroups | gmane.comp.video.mesa3d.devel |
|---|---|
| Message-ID | <CAHXBcjS9R6dQpHiQgb=Kq7coRqX=ko2+Uw4RRg45+A=jWKKPzg@mail.gmail.com> |
--000000000000f9d80d0652f7f799 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Apologies for the half-sent reply, Gmail has gotten super fun with all of its... changes. Thanks for the info, Erik! In general, most Mesa drivers works in a similar way; they communicate > with a kernel mode driver that programs the actual hardware. So you'd > still need one of those. > On Windows, my understanding is that kernel module drivers would > generally expose a WDDM2 driver, and we'd communicate with that, > somehow. > However, we don't have any GPU drivers that communicate with a kernel > mode driver on Windows upstream in Mesa yet. This is all still work-in- > progress > Faith Ekstrand had a presentation about this on XDC 2024, which you can > see here: > https://indico.freedesktop.org/event/6/contributions/296/ > Here's an MR relating to that topic: > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29945 > So yeah, I think choosing to do this on Windows kinda enables "Hard > Mode" for your driver, but it's not impossible to do. I'm already doing very much that - I've got a WDDM2 driver (that just so happens to be the "display-only" portion, as far as the OS is concerned) and I utilize the DxgkddiEscape <https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/d3dkmddi/nc= -d3dkmddi-dxgkddi_escape> function to send data back-and-forth between the user and kernel mode drivers. So far, has worked great! As Faith points out in her presentation slides, having a full D3D12 stack in the kernel driver is generally a requirement for all of the windowing system interop stuff - having it use the GPU to composite the display, allowing D3D stuff to live in a window and not forced to be full-screen, etc. At the moment, I'm okay with bypassing all of that and pretending that full screen is the only thing it can do. I'm mostly interested in running games, and games typically take up the entire screen anyway! I would usually say "look at the most modern driver in Mesa", which > probably would be NVK at this point, and copy that as a starting-point. > But you probably want to look at other actively developed drivers as > well, such as RADV, Turnip and PanVK. It's usually better to look at > what multiple drivers are doing, and try to understand why their > approaches sometimes differ. > Other than that, we have a lot of documentation at docs.mesa3d.org, > including for the Vulkan Runtime, NIR, how to submit patches etc. Awesome, I'll start there! Communication with the window system happens using shared memory. This > would typically be something like DMAbufs on Linux, and AFAIK WDDM > shared memory handles on Windows. > Doing it this way lets the operating system do things like multi-GPU, > where it can render on one GPU and display on another. There's a lot of > little details to this, but the gist is that we need some shared memory > objects, fences to control the timing of the access to the memory > objects and some memory layout negotiation to all ends agree on these. > I only know the Linux side here well, and in that case these pieces > would be dma_buf, dma_fence and DRM modifiers. I've got a very "simple" mechanism for fencing ops at the moment, where I basically just wait for a device address to be set to a given value and stall/delay the host until that value is DMA'd back. I've been considering adding a better mechanism (interrupts, something) and some more hardware to make it a bit cleaner in that regard, but yeah I'll take a look at how it works for Linux in Mesa to see how I might want to adapt my own setup. Just a heads up: making a GPU driver is a huge task. Doing it alone > makes it even harder. So consider teaming up with someone, and make > sure it's as easy for others as possible to help out! > Also, one of the biggest job of a GPU driver is making the shader > compiler. There's a lot of videos of XDC and FOSDEM presentation about > the shader compilers in Mesa, those would probably worth a look. Don't I know it! I've been doing rendering work for games for my entire career (and earlier, honestly!), and part of the reason I started this project was to dig a bit further into the lower-level details of the hardware and driving it. Doing it alone is half the fun. ;) I took a few shortcuts, but the shader compiler is already done on my side - I'm cheating a bit by utilizing the very fast A53 cores in the Zynq UltraScale+ as shader execution units, utilizing their NEON instructions to go wide. I have a custom SPIR-V-to-NEON-intrinsics translator, so as long as whatever high-level language can be compiled down to SPIR-V, I can translate it to what my hardware needs. I use Clang compiled as a static library to actually compile the translated code down to an executable, and the driver manages copying it to the device and setting it up for execution as needed during command-list generation. It's certainly not perfect, and it's only really been tested on the few small shaders I've written or adapted for Quake and Doom 3, but it does work! Thanks again! - Dylan On Fri, May 29, 2026 at 1:57=E2=80=AFAM Erik Faye-Lund <erik.faye-lund@coll= abora.com> wrote: > On Thu, 2026-05-28 at 23:33 -0700, Dylan Barrie wrote: > > Hi all! > > I've spent the last couple years developing a fully-custom, FPGA- > > based GPU (www.furygpu.com) that I've recently gotten to the point > > where it should be mostly compatible with the requirements of Vulkan > > (shaders, for one!), to the degree that running simple programs would > > likely be possible. I've been driving the hardware with a custom API > > that is basically an API-copy of Vulkan, but given that the Mesa 3D > > project has already done a lot of the heavy lifting in terms of all > > of the infrastructure, I'm thinking it would probably be worthwhile > > to make use of it. It gets a bit tiring having to re-write every > > program I want to run! > > That's some impressive progress there for sure! Cool project :) > > > Before I get started on this, I had a few questions that I hope those > > here can answer: > > 1. I'm currently using a kernel-mode driver (on Windows) to > > communicate with the actual hardware. Is there existing > > infrastructure to deal with that kind of separation in Mesa 3D > > already? I've made efforts to slim down the user-mode <-> kernel-mode > > interface as much as possible, and it basically comes down to just > > setting up DMA ops and telling the GPU to start executing from a > > given address. > > In general, most Mesa drivers works in a similar way; they communicate > with a kernel mode driver that programs the actual hardware. So you'd > still need one of those. > > On Windows, my understanding is that kernel module drivers would > generally expose a WDDM2 driver, and we'd communicate with that, > somehow. > > However, we don't have any GPU drivers that communicate with a kernel > mode driver on Windows upstream in Mesa yet. This is all still work-in- > progress > > Faith Ekstrand had a presentation about this on XDC 2024, which you can > see here: > https://indico.freedesktop.org/event/6/contributions/296/ > > Here's an MR relating to that topic: > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29945 > > So yeah, I think choosing to do this on Windows kinda enables "Hard > Mode" for your driver, but it's not impossible to do. > > > 2. Is there an existing driver that would be a good starting point > > in terms of best practices for working within the Mesa 3D ecosystem? > > I would usually say "look at the most modern driver in Mesa", which > probably would be NVK at this point, and copy that as a starting-point. > But you probably want to look at other actively developed drivers as > well, such as RADV, Turnip and PanVK. It's usually better to look at > what multiple drivers are doing, and try to understand why their > approaches sometimes differ. > > Other than that, we have a lot of documentation at docs.mesa3d.org, > including for the Vulkan Runtime, NIR, how to submit patches etc. > > > 3. How does Mesa 3D interact with the OS's windowing system? My > > current setup is a kernel-mode display-only driver (like would be > > used for a USB->HDMI converter or something), which allows Windows to > > treat my GPU as a display output. When it comes time for an > > application to actually use the GPU hardware, the driver switches to > > outputting the rendered image to the display and the OS continues > > believing it's driving the display, none the wiser. > > Communication with the window system happens using shared memory. This > would typically be something like DMAbufs on Linux, and AFAIK WDDM > shared memory handles on Windows. > > Doing it this way lets the operating system do things like multi-GPU, > where it can render on one GPU and display on another. There's a lot of > little details to this, but the gist is that we need some shared memory > objects, fences to control the timing of the access to the memory > objects and some memory layout negotiation to all ends agree on these. > > I only know the Linux side here well, and in that case these pieces > would be dma_buf, dma_fence and DRM modifiers. > > > 4. Is there anything else I should be aware of or understand > > before starting? > > Just a heads up: making a GPU driver is a huge task. Doing it alone > makes it even harder. So consider teaming up with someone, and make > sure it's as easy for others as possible to help out! > > Also, one of the biggest job of a GPU driver is making the shader > compiler. There's a lot of videos of XDC and FOSDEM presentation about > the shader compilers in Mesa, those would probably worth a look. > > Oh, and also, if you're serious about this, consider attending XDC! > https://xdc2026.x.org/ > > > Thanks for any insight you all may have for me! > > > > - Dylan > > Thanks for sharing your cool project, and good luck! > > - Erik > --000000000000f9d80d0652f7f799 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Apologies for the half-sent reply, Gmail has gotten super = fun with all of its... changes.<br><br>Thanks for the info, Erik!<div><br><= /div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo= rder-left:1px solid rgb(204,204,204);padding-left:1ex">In general, most Mes= a drivers works in a similar way; they communicate<br>with a kernel mode dr= iver that programs the actual hardware. So you'd<br>still need one of t= hose.</blockquote><div><blockquote class=3D"gmail_quote" style=3D"margin:0p= x 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><b= r></blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Windows= , my understanding is that kernel module drivers would<br>generally expose = a WDDM2 driver, and we'd communicate with that,<br>somehow.</blockquote= ><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border= -left:1px solid rgb(204,204,204);padding-left:1ex"><br>However, we don'= t have any GPU drivers that communicate with a kernel<br>mode driver on Win= dows upstream in Mesa yet. This is all still work-in-<br>progress</blockquo= te><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bord= er-left:1px solid rgb(204,204,204);padding-left:1ex"><br>Faith Ekstrand had= a presentation about this on XDC 2024, which you can<br>see here:<br><a hr= ef=3D"https://indico.freedesktop.org/event/6/contributions/296/" rel=3D"nor= eferrer" target=3D"_blank">https://indico.freedesktop.org/event/6/contribut= ions/296/</a></blockquote><blockquote class=3D"gmail_quote" style=3D"margin= :0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"= ><br>Here's an MR relating to that topic:<br><a href=3D"https://gitlab.= freedesktop.org/mesa/mesa/-/merge_requests/29945" rel=3D"noreferrer" target= =3D"_blank">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29945= </a></blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px = 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>So y= eah, I think choosing to do this on Windows kinda enables "Hard<br>Mod= e" for your driver, but it's not impossible to do.</blockquote><di= v><br></div><div>I'm already doing very much that - I've got a WDDM= 2 driver (that just so happens to be the "display-only" portion, = as far as the OS is concerned) and I utilize the=C2=A0 <a href=3D"https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/d= 3dkmddi/nc-d3dkmddi-dxgkddi_escape">DxgkddiEscape </a> function to send dat= a back-and-forth between the user and kernel mode drivers. So far, has work= ed great!</div><div><br></div><div>As Faith points out in her presentation = slides, having a full D3D12 stack in the kernel driver is generally a requi= rement for all of the windowing system interop stuff - having it use the GP= U to composite the display, allowing D3D stuff to live in a window and not = forced to be full-screen, etc. At the moment, I'm okay with bypassing a= ll of that and pretending that full screen is the only thing it can do. I&#= 39;m mostly interested in running games, and games typically take up the en= tire screen anyway!=C2=A0</div></div><div><br></div><div><blockquote class= =3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg= b(204,204,204);padding-left:1ex">I would usually say "look at the most= modern driver in Mesa", which<br>probably would be NVK at this point,= and copy that as a starting-point.<br>But you probably want to look at oth= er actively developed drivers as<br>well, such as RADV, Turnip and PanVK. I= t's usually better to look at<br>what multiple drivers are doing, and t= ry to understand why their<br>approaches sometimes differ.</blockquote><blo= ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left= :1px solid rgb(204,204,204);padding-left:1ex"><br>Other than that, we have = a lot of documentation at=C2=A0<a href=3D"http://docs.mesa3d.org/" rel=3D"n= oreferrer" target=3D"_blank">docs.mesa3d.org</a>,<br>including for the Vulk= an Runtime, NIR, how to submit patches etc.</blockquote><div><br></div><div= >Awesome, I'll start there!=C2=A0</div></div><div><br></div><blockquote= class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px so= lid rgb(204,204,204);padding-left:1ex">Communication with the window system= happens using shared memory. This<br>would typically be something like DMA= bufs on Linux, and AFAIK WDDM<br>shared memory handles on Windows.</blockqu= ote><div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e= x;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br></blockquote= ><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border= -left:1px solid rgb(204,204,204);padding-left:1ex">Doing it this way lets t= he operating system do things like multi-GPU,<br>where it can render on one= GPU and display on another. There's a lot of<br>little details to this= , but the gist is that we need some shared memory<br>objects, fences to con= trol the timing of the access to the memory<br>objects and some memory layo= ut negotiation to all ends agree on these.</blockquote><blockquote class=3D= "gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(2= 04,204,204);padding-left:1ex"><br>I only know the Linux side here well, and= in that case these pieces<br>would be dma_buf, dma_fence and DRM modifiers= .</blockquote><div><br></div><div>I've got a very "simple" me= chanism for fencing ops at the moment, where I basically just wait for a de= vice address to be set to a given value and stall/delay the host until that= value is DMA'd back. I've been considering adding a better mechani= sm (interrupts, something) and some more hardware to make it a bit cleaner = in that regard, but yeah I'll take a look at how it works for Linux in = Mesa to see how I might want to adapt my own setup.=C2=A0</div></div><div><= br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8e= x;border-left:1px solid rgb(204,204,204);padding-left:1ex">Just a heads up:= making a GPU driver is a huge task. Doing it alone<br>makes it even harder= . So consider teaming up with someone, and make<br>sure it's as easy fo= r others as possible to help out!</blockquote><blockquote class=3D"gmail_qu= ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20= 4);padding-left:1ex"><br>Also, one of the biggest job of a GPU driver is ma= king the shader<br>compiler. There's a lot of videos of XDC and FOSDEM = presentation about<br>the shader compilers in Mesa, those would probably wo= rth a look.</blockquote><div><br></div><div>Don't I know it! I've b= een doing rendering work for games for my entire career (and earlier, hones= tly!), and part of the reason I started this project was to dig a bit furth= er into the lower-level details of the hardware and driving it. Doing it al= one is half the fun. ;)</div><div><br></div><div>I took a few shortcuts, bu= t the shader compiler is already done on my side - I'm cheating a bit b= y utilizing the very fast A53 cores in the Zynq UltraScale+ as shader execu= tion units, utilizing their NEON instructions to go wide. I have a custom S= PIR-V-to-NEON-intrinsics translator, so as long as whatever high-level lang= uage can be compiled down to SPIR-V, I can translate it to what my hardware= needs. I use Clang compiled as a static library to actually compile the tr= anslated code down to an executable, and the driver manages copying it to t= he device and setting it up for execution as needed during command-list gen= eration. It's certainly not perfect, and it's only really been test= ed on the few small shaders I've written or adapted for Quake and Doom = 3, but it does work!=C2=A0</div><div><br></div><div>Thanks again!</div><div= ><br></div><div>=C2=A0- Dylan</div></div><br><div class=3D"gmail_quote gmai= l_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, May 29, 20= 26 at 1:57=E2=80=AFAM Erik Faye-Lund <<a href=3D"mailto:erik.faye-lund@c= ollabora.com">[email protected]</a>> wrote:<br></div><blockqu= ote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px= solid rgb(204,204,204);padding-left:1ex">On Thu, 2026-05-28 at 23:33 -0700= , Dylan Barrie wrote:<br> > Hi all!<br> > I've spent the last couple years developing a fully-custom, FPGA-<= br> > based GPU (<a href=3D"http://www.furygpu.com" rel=3D"noreferrer" targe= t=3D"_blank">www.furygpu.com</a>) that I've recently gotten to the poin= t<br> > where it should be mostly compatible with the requirements of Vulkan<b= r> > (shaders, for one!), to the degree that running simple programs would<= br> > likely be possible. I've been driving the hardware with a custom A= PI<br> > that is basically an API-copy of Vulkan, but given that the Mesa 3D<br= > > project has already done a lot of the heavy lifting in terms of all<br= > > of the infrastructure, I'm thinking it would probably be worthwhil= e<br> > to make use of it. It gets a bit tiring having to re-write every<br> > program I want to run!<br> <br> That's some impressive progress there for sure! Cool project :)<br> <br> > Before I get started on this, I had a few questions that I hope those<= br> > here can answer:<br> > =C2=A0=C2=A0=C2=A01. I'm currently using a kernel-mode driver (on = Windows) to<br> > communicate with the actual hardware. Is there existing<br> > infrastructure to deal with that kind of separation in Mesa 3D<br> > already? I've made efforts to slim down the user-mode <-> ke= rnel-mode<br> > interface as much as possible, and it basically comes down to just<br> > setting up DMA ops and telling the GPU to start executing from a<br> > given address.<br> <br> In general, most Mesa drivers works in a similar way; they communicate<br> with a kernel mode driver that programs the actual hardware. So you'd<b= r> still need one of those.<br> <br> On Windows, my understanding is that kernel module drivers would<br> generally expose a WDDM2 driver, and we'd communicate with that,<br> somehow.<br> <br> However, we don't have any GPU drivers that communicate with a kernel<b= r> mode driver on Windows upstream in Mesa yet. This is all still work-in-<br> progress<br> <br> Faith Ekstrand had a presentation about this on XDC 2024, which you can<br> see here:<br> <a href=3D"https://indico.freedesktop.org/event/6/contributions/296/" rel= =3D"noreferrer" target=3D"_blank">https://indico.freedesktop.org/event/6/co= ntributions/296/</a><br> <br> Here's an MR relating to that topic:<br> <a href=3D"https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/29945"= rel=3D"noreferrer" target=3D"_blank">https://gitlab.freedesktop.org/mesa/m= esa/-/merge_requests/29945</a><br> <br> So yeah, I think choosing to do this on Windows kinda enables "Hard<br= > Mode" for your driver, but it's not impossible to do.<br> <br> > =C2=A0=C2=A0=C2=A02. Is there an existing driver that would be a good = starting point<br> > in terms of best practices for working within the Mesa 3D ecosystem?<b= r> <br> I would usually say "look at the most modern driver in Mesa", whi= ch<br> probably would be NVK at this point, and copy that as a starting-point.<br> But you probably want to look at other actively developed drivers as<br> well, such as RADV, Turnip and PanVK. It's usually better to look at<br= > what multiple drivers are doing, and try to understand why their<br> approaches sometimes differ.<br> <br> Other than that, we have a lot of documentation at <a href=3D"http://docs.m= esa3d.org" rel=3D"noreferrer" target=3D"_blank">docs.mesa3d.org</a>,<br> including for the Vulkan Runtime, NIR, how to submit patches etc.<br> <br> > =C2=A0=C2=A0=C2=A03. How does Mesa 3D interact with the OS's windo= wing system? My<br> > current setup is a kernel-mode display-only driver (like would be<br> > used for a USB->HDMI converter or something), which allows Windows = to<br> > treat my GPU as a display output. When it comes time for an<br> > application to actually use the GPU hardware, the driver switches to<b= r> > outputting the rendered image to the display and the OS continues<br> > believing it's driving the display, none the wiser.<br> <br> Communication with the window system happens using shared memory. This<br> would typically be something like DMAbufs on Linux, and AFAIK WDDM<br> shared memory handles on Windows.<br> <br> Doing it this way lets the operating system do things like multi-GPU,<br> where it can render on one GPU and display on another. There's a lot of= <br> little details to this, but the gist is that we need some shared memory<br> objects, fences to control the timing of the access to the memory<br> objects and some memory layout negotiation to all ends agree on these.<br> <br> I only know the Linux side here well, and in that case these pieces<br> would be dma_buf, dma_fence and DRM modifiers.<br> <br> > =C2=A0=C2=A0=C2=A04. Is there anything else I should be aware of or un= derstand<br> > before starting?<br> <br> Just a heads up: making a GPU driver is a huge task. Doing it alone<br> makes it even harder. So consider teaming up with someone, and make<br> sure it's as easy for others as possible to help out!<br> <br> Also, one of the biggest job of a GPU driver is making the shader<br> compiler. There's a lot of videos of XDC and FOSDEM presentation about<= br> the shader compilers in Mesa, those would probably worth a look.<br> <br> Oh, and also, if you're serious about this, consider attending XDC!<br> <a href=3D"https://xdc2026.x.org/" rel=3D"noreferrer" target=3D"_blank">htt= ps://xdc2026.x.org/</a><br> <br> > Thanks for any insight you all may have for me!<br> > <br> > =C2=A0- Dylan<br> <br> Thanks for sharing your cool project, and good luck!<br> <br> - Erik<br> </blockquote></div> --000000000000f9d80d0652f7f799--