[PATCH 15/15] Documentation: PCI: endpoint: Add vNTB DMA export HOWTO
Koichiro Den <[email protected]> Fri, 13 Mar 2026 01:50:05 +0900
| Newsgroups | dev.linux.lists.ntb,org.kernel.vger.dmaengine,org.kernel.vger.linux-doc,org.kernel.vger.linux-kernel,org.kernel.vger.linux-pci |
|---|---|
| Message-ID | <[email protected]> |
Document the new pci-epf-vntb DMA export model. Describe the configfs placement knobs for the DMA slice, the versioned control layout, the two-stage BAR programming model used for shared BARs, and the host-side discovery path through ntb_hw_epf and dw-edma-aux. Also show how to exercise the feature with the ntb_ep_dma test client. Signed-off-by: Koichiro Den <[email protected]> --- Documentation/PCI/endpoint/index.rst | 1 + .../PCI/endpoint/pci-vntb-dma-howto.rst | 83 +++++++++++++++++++ 2 files changed, 84 insertions(+) create mode 100644 Documentation/PCI/endpoint/pci-vntb-dma-howto.rst diff --git a/Documentation/PCI/endpoint/index.rst b/Documentation/PCI/endpoint/index.rst index dd1f62e731c9..c3f93f1da34b 100644 --- a/Documentation/PCI/endpoint/index.rst +++ b/Documentation/PCI/endpoint/index.rst @@ -15,6 +15,7 @@ PCI Endpoint Framework pci-ntb-howto pci-vntb-function pci-vntb-howto + pci-vntb-dma-howto pci-nvme-function function/binding/pci-test diff --git a/Documentation/PCI/endpoint/pci-vntb-dma-howto.rst b/Documentation/PCI/endpoint/pci-vntb-dma-howto.rst new file mode 100644 index 000000000000..c3acc4e6ab37 --- /dev/null +++ b/Documentation/PCI/endpoint/pci-vntb-dma-howto.rst @@ -0,0 +1,83 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============================= +pci-epf-vntb DMA export HOWTO +============================= + +``pci-epf-vntb`` can expose a virtual NTB device to the host while also +exporting an endpoint-integrated DMA instance. The logical DMA export slice may +occupy its own BAR or share a physical BAR with one or more memory windows. + +Configuration knobs +=================== + +The endpoint function instance exposes the following placement controls via +configfs: + +- ``mwN_bar`` +- ``mwN_offset`` +- ``dma_bar`` +- ``dma_offset`` +- ``dma_num_chans`` + +The offsets are BAR-relative. Regions packed into the same physical BAR must be +non-overlapping and must cover the complete programmed BAR aperture. + +``dma_num_chans`` controls how many DMA read channels are delegated to the +peer. If fewer channels are available at runtime, the endpoint falls back to +the maximum available number and reports the effective delegation count in the +kernel log. + +DMA ABI v1 is currently defined for READ channels only. The exported channels +must form a dense prefix of the endpoint READ-channel space, and the +``chans[]`` entries in the ABI header are ordered by remote hardware READ +channel index starting at 0. ``dw-edma-aux`` relies on that ordering when it +reconstructs the remote engine on the host side. + +Control-layout compatibility +============================ + +Offset ``0x0c`` in the control block is used as a control-layout version. The +legacy layout is version 0. Version 1 extends the block with per-MW +``offset/size`` tuples and the DMA locator. + +``pci-epf-vntb`` emits the legacy layout whenever the current configuration can +still be represented by it. This allows a new endpoint kernel to interoperate +with an older ``ntb_hw_epf`` host as long as the deployment uses the historical +one-MW-per-BAR layout and does not enable DMA export. + +Deferred BAR activation +======================= + +BAR subrange mapping cannot be programmed at bind time because the host has not +assigned BAR addresses yet. Shared ``MW/DMA`` BARs therefore use a two-stage +programming model: + +1. bind time: configure the BAR aperture with a temporary plain mapping +2. first host command (``CONFIGURE_MW`` or ``LINK_UP``): re-issue + ``pci_epc_set_bar()`` with the final subrange layout + +This matches the requirements for EPC subrange mapping feature. + +DMA export and ``ntb_ep_dma`` +============================= + +When DMA export is enabled, ``pci-epf-vntb`` publishes a logical DMA locator in +the control block. ``ntb_hw_epf`` parses that locator during probe, reserves an +IRQ slice for the DMA provider, and registers the auxiliary child only after +``LINK_UP`` succeeds. As a vendor-specific frontend example, ``dw-edma-aux`` +binds to that child and exposes a DMA engine provider for the remote endpoint +DMA instance. + +The ``ntb_ep_dma`` test client exercises that path. Both peers load the +module. Each side allocates a local test buffer and publishes the DMA address +and size through scratchpads. The responder only waits and verifies, while the +initiator triggers one transfer with:: + + # cat /sys/kernel/debug/ntb_ep_dma/0000:01:00.0/ready + # echo 1 > /sys/kernel/debug/ntb_ep_dma/0000:01:00.0/run + # cat /sys/kernel/debug/ntb_ep_dma/0000:01:00.0/result + +The initiator side uses the remote exported DMA instance to transfer into the +peer-published buffer. The responder verifies the contents locally and reports +PASS/FAIL back through scratchpads and a doorbell. -- 2.51.0