Re: [PATCH v10 7/7] PCI: endpoint: pci-ep-msi: Add embedded doorbell fallback

On Wed, Mar 25, 2026 at 05:56:36PM +0100, Niklas Cassel wrote:
> On Wed, Mar 25, 2026 at 05:43:58PM +0900, Koichiro Den wrote:
> > > > > ...
> > > > > > +	 */
> > > > > > +	if (doorbell->bar == NO_BAR) {
> > > > > > +		phys_base = addr & PAGE_MASK;
> > > > > > +		off = addr - phys_base;
> > > > > > +		map_size = PAGE_ALIGN(off + sizeof(u32));
> > > > > > +
> > > > > > +		iova_base = dma_map_resource(epc->dev.parent, phys_base,
> > > > > > +					     map_size, DMA_FROM_DEVICE, 0);
> > > > > 
> > > > > DB should write. Is it DMA_TO_DEVICE?
> > > > 
> > > > Thanks for reviewing.
> > > > 
> > > > The write is initiated by the peer (RC) and lands in this EP-side doorbell
> > > > target, so from the EP side this is a DMA_FROM_DEVICE mapping.
> > > 
> > > I don't understand.
> > > 
> > > If the RC side does a PCI TLP write, the iATU on the EP side should translate
> > > that to an AXI write, no?
> > 
> > Yes.
> > 
> > > 
> > > If the target address (doorbell register in this case) is only mapped as a
> > > DMA_FROM_DEVICE (read), I would have expected that AXI write to result in an
> > > IOMMU fault.
> > 
> > My understanding is that the inbound MWr becomes a local write on the EP side.
> > Here, the EP controller needs write permission to the doorbell target, so
> > DMA_FROM_DEVICE is intentional.
> > 
> > As noted in the cover letter, IOMMU coverage was tested.
> > 
> > > 
> > > Is the IOMMU really running in strict mode and not in passthrough mode on
> > > your platform?
> > > 
> > > Do you get an IOMMU fault if you change this code to DMA_TO_DEVICE ?
> > 
> > Yes, changing it to DMA_TO_DEVICE triggers an IOMMU fault. I've confirmed this
> > experimentally. This also indicates that it's not passthrough for this mapping
> > on my setup.
> 
> I still think it sounds wrong as per:
> https://docs.kernel.org/core-api/dma-api-howto.html#dma-direction
> 
> The transaction is a write from
> PCIe bus -> PCIe controller iATU -> internal bus -> IOMMU -> PCIe controller
> (the same controller as initiated the transaction).

Yes, I think we're on the same page about this path itself.

On my R-Car S4 setup, changing this to DMA_TO_DEVICE consistently triggers an
IOMMU fault, so at least on this platform the local path used for the doorbell
mapping is IOMMU-visible. That is the case this dma_map_resource() is intended
to cover.

For that path, my understanding is that the doorbell access ends up as a local
write on the EP side, so it needs write permission, hence DMA_FROM_DEVICE.

> Would be interesting why this is not working like normal (when using buffers):
> "For Networking drivers, it’s a rather simple affair.
> For transmit packets, map/unmap them with the DMA_TO_DEVICE direction specifier.
> For receive packets, just the opposite, map/unmap them with the DMA_FROM_DEVICE
> direction specifier."

I think the closer analogy is RX: the data comes from outside, but the device
writes to the target, so it needs write permission.

> 
> Pehaps it is because it is the same device that does a write, that ends up
> writing to it a self? so source an destination is the same?

I'm not sure that "same device" aspect matters here.

(Separately, for the DWC case, I'm also re-checking whether the embedded doorbell
write should ideally stay on the native TRGT0->LBC->CDM path instead, which
would be IOMMU-invisible. I don't want to overstate that point yet, though.)

Best regards,
Koichiro

> 
> 
> Kind regards,
> Niklas