Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for passthrough device

On Fri, 2026-08-21 at 07:15 +0000, Duan, Zhenzhong wrote:
> Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe.
> 
> 
> 
> > -----Original Message-----
> > From: Clément MATHIEU--DRIF <[[email protected]](mailto:[email protected])>
> > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for passthrough
> > device
> > 
> > 
> > On Fri, 2026-08-21 at 02:44 +0000, Duan, Zhenzhong wrote:
> > 
> > > Caution: External email. Do not open attachments or click links, unless this email
> > 
> > comes from a known sender and you know the content is safe.
> > 
> > > 
> > > 
> > > Hi Clement,
> > > 
> > > 
> > > 
> > > > -----Original Message-----
> > > > From: Clément MATHIEU--DRIF <[clement.mathieu--
> > > 
> > 
> > [[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))>
> > 
> > > 
> > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for passthrough
> > > > device
> > > > 
> > > > 
> > > > On Wed, 2026-08-12 at 13:26 +0000, Duan, Zhenzhong wrote:
> > > > 
> > > > 
> > > > > Caution: External email. Do not open attachments or click links, unless this
> > > > 
> > > 
> > 
> > email
> > 
> > > 
> > > > 
> > > > comes from a known sender and you know the content is safe.
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > > -----Original Message-----
> > > > > > From: Clément MATHIEU--DRIF <[clement.mathieu--
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > [[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))](mailto:[clement.mathieu--
> > > 
> > 
> > [[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected])))>
> > 
> > > 
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for
> > > > > 
> > > > 
> > > 
> > 
> > passthrough
> > 
> > > 
> > > > 
> > > > > 
> > > > > > device
> > > > > > 
> > > > > > 
> > > > > > On Mon, 2026-08-10 at 10:49 +0000, Duan, Zhenzhong wrote:
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > > Caution: External email. Do not open attachments or click links, unless
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> > this
> > 
> > > 
> > > > 
> > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > email
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > > 
> > > > > > comes from a known sender and you know the content is safe.
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > Hi Clement,
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > > -----Original Message-----
> > > > > > > > From: Clément MATHIEU--DRIF <[clement.mathieu--
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> > [[[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))](mailto:[[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected])
> > om))](mailto:[clement.mathieu--
> > 
> > > 
> > > > 
> > > > > 
> > > > 
> > > > 
> > > > [[[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))](mailto:[clement.mathieu--
> > > 
> > 
> > [[email protected]](mailto:[email protected])](mailto:[[email protected]](mailto:[email protected]))))>
> > 
> > > 
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > > Subject: Re: [PATCH v3 0/6] intel_iommu: Enable PRQ support for
> > > > > > > 
> > > > > > > 
> > > > > > 
> > > > > > 
> > > > > 
> > > > > 
> > > > 
> > > > 
> > > > passthrough
> > > > 
> > > > 
> > > > > 
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > > device
> > > > > > > > 
> > > > > > > > Hi Zhenzhong,
> > > > > > > > 
> > > > > > > > I will read this series early next week.
> > > > > > > > thanks for the respin!
> > > > > > > > 
> > > > > > > > btw, related to this work: have ever seen
> > > > > > > > vfio_device_get_aw_bits returning 47 whereas
> > > > > > > > the iommu supports 48? This leads the following
> > > > > > > > aw test to fail:
> > > > > > > > 
> > > > > > > > ```
> > > > > > > > ret = hiodc->get_cap(hiod, HOST_IOMMU_DEVICE_CAP_AW_BITS,
> > > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > 
> > errp);
> > 
> > > 
> > > > 
> > > > > 
> > > > > > 
> > > > > > > 
> > > > > > > > if (ret < 0) {
> > > > > > > >    return false;
> > > > > > > > }
> > > > > > > > if (s->aw_bits > ret) {
> > > > > > > >    error_setg(errp, "aw-bits %d > host aw-bits %d", s->aw_bits, ret);
> > > > > > > >    return false;
> > > > > > > > }
> > > > > > > > ```
> > > > > > > > 
> > > > > > > > I got this when trying to nest VMs.
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > 
> > > > > > > I am trying to reproduce, what device did you passthrough?
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > I used edu for testing.
> > > > > > An I had aw-bits=48 on both vms.
> > > > > 
> > > > > 
> > > > > 
> > > > > 
> > > > > Reproduced the same with L1(fsts=on) and L2(fsts=off).
> > > > 
> > > > 
> > > > 
> > > > My config has fsts=on on both VMs.
> > > > Which kernel are you using?
> > > > Mine is RHEL10.2 (6.12.0-211.7.3.el10_2.x86_64)
> > > 
> > > 
> > > 
> > > In theory we will not run into that check if fsts=on on both VMs, on my env I see:
> > > 
> > > "qemu-system-x86_64: -device vfio-pci,host=00:04.0,iommufd=iommufd0: vfio
> > 
> > 0000:00:04.0: Failed to allocate hwpt: Operation not supported"
> > 
> > > 
> > > which is expected as we want nesting support in L1 but it only supports fsts.
> > 
> > 
> > Oh yep, I forgot to mention that I'm not attaching iommufd for this test.
> > I agree with what you say when iommufd is present, but without it (weird config,
> > I admit, but interesting for testing), I get:
> > 
> > ```
> > qemu-system-x86_64: -device vfio-pci,host=00:01.0: vfio 0000:00:01.0: Failed to
> > set vIOMMU: aw-bits 48 > host aw-bits 47
> 
> 
> Oh, I see, I reproduced the same.
> 
> 
> > ```
> > 
> > Just wondering why such a 1bit difference.
> 
> 
> host aw-bits 47 is what host kernel supports, not the pure hw limit.  
> For first stage page table, PT_FEAT_SIGN_EXTEND is set,  
> kernel limited max iova ranges to 47bit even though hw supports 48 bit, see below:
> 
> static __always_inline struct pt_range _pt_top_range(struct pt_common *common,  
>                                                      uintptr_t top_of_table)  
> {  
> ...  
>         /*  
>          * The top range will default to the lower region only with sign extend.  
>          */  
>         range.max_vasz_lg2 = max_vasz_lg2;  
>         if (pt_feature(common, PT_FEAT_SIGN_EXTEND))  
>                 max_vasz_lg2--;  
> ...  
> }
> 
> So qemu gets host aw-bits 47. This is strictly following vtd spec:
> 
> Software using first-stage translation structures to translate an IO Virtual Address (IOVA) must use  
> canonical addresses. Additionally, software must limit addresses to less than the minimum of MGAW  
> and the lower canonical address width implied by FSPM (i.e., 47-bit when FSPM is 4-level and 56-bit  
> when FSPM is 5-level).

Oh, I see, good to know!  
That makes sense.  

Thanks!

cmd

> 
> BRs,  
> Zhenzhong