Re: [PATCH v1] ntb_transport: replace wmb() with dma_wmb() in TX paths

Hi DJ ,

Thanks for the review and the explanation.

I understand that dma_wmb() is intended for CPU =E2=86=92 device DMA visibi=
lity, and
since this path writes to MMIO (peer memory window), a full wmb() is requir=
ed
to guarantee that all writes are visible before the notification/doorbell.

I will withdraw this patch and keep the wmb() in place.

Thanks,
Xingbang Liu

Dave Jiang <[email protected]> =E4=BA=8E2026=E5=B9=B44=E6=9C=8824=E6=97=
=A5=E5=91=A8=E4=BA=94 23:32=E5=86=99=E9=81=93=EF=BC=9A
>
>
>
> On 4/24/26 2:23 AM, Xingbang Liu wrote:
> > The TX paths currently use wmb() after memcpy_toio() to ensure that
> > writes to the peer memory window are visible before subsequent
> > notification.
> >
> > The actual ordering requirement is to ensure that data written to the
> > peer memory is observed before the associated notification or status
> > update. This matches the semantics of dma_wmb(), which orders memory
> > writes with respect to external observers.
> >
> > The peer memory window is mapped with ioremap_wc(), allowing
> > write-combining. On weakly ordered architectures such as arm64,
> > dma_wmb() ensures that prior writes are committed from write-combining
> > buffers and become visible to the peer before subsequent operations.
> >
> > This aligns with common patterns in PCIe-based drivers, where it is
> > used to order descriptor or payload writes before notification, while
> > avoiding the stronger ordering semantics of wmb().
> >
> > The ordering with respect to peer observation is completed by the
> > subsequent notification mechanism.
> >
> > No functional change is intended.
> >
> > Signed-off-by: Xingbang Liu <[email protected]>
>
> NAK. dma_wmb() is used for shared memory between CPU and a device. For ex=
ample:
>
> memcpy(dma_buffer, data, size);
> dma_wmb(); /* Ensure CPU writes visible to device DMA */
> writel(DOORBELL, device->reg + DB_OFFSET); /* trigger device DMA from dma=
_buffer */
>
> In this case we are copying data to MMIO and therefore wmb() is needed to=
 ensure all MMIO writes have happened before we proceed.
>
> DJ
>
> > ---
> >  drivers/ntb/ntb_transport.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/ntb/ntb_transport.c b/drivers/ntb/ntb_transport.c
> > index 7cabc82305d6..c8ef46ddcc57 100644
> > --- a/drivers/ntb/ntb_transport.c
> > +++ b/drivers/ntb/ntb_transport.c
> > @@ -1791,7 +1791,7 @@ static void ntb_memcpy_tx_on_stack(struct ntb_que=
ue_entry *entry, void __iomem *
> >  #endif
> >
> >       /* Ensure that the data is fully copied out before setting the fl=
ags */
> > -     wmb();
> > +     dma_wmb();
> >
> >       ntb_tx_copy_callback(entry, NULL);
> >  }
>