Re: [PATCH v3] Add VIRTIO_F_DMB (Device Memory Buffer)
"Michael S. Tsirkin" <[email protected]>
| Newsgroups | dev.linux.lists.virtio-comment |
|---|---|
| Message-ID | <[email protected]> |
On Sun, Aug 09, 2026 at 10:09:15PM -0400, Demi Marie Obenour wrote: > On 8/9/26 17:48, Michael S. Tsirkin wrote: > > On Tue, Aug 04, 2026 at 04:12:02PM +0000, Alexander Graf wrote: > >> When a transport uses a shared bounce region for virtqueue memory today, > >> that region is global and shared by multiple devices at once. Devices > >> then contend over one small region, so a busy device can starve the > >> others, and one device's emulation cannot be isolated or torn down on its > >> own because its state lives in memory everyone else is using too. > >> > >> VIRTIO_F_DMB gives each device its own shared memory region, the Device > >> Memory Buffer, that holds that device's virtqueues (Descriptor, Driver > >> and Device Areas) and the buffers the descriptors reference. When the > >> feature is negotiated, every address the driver places in a virtqueue is > >> an offset into that region rather than a physical or bus address, so the > >> device only ever touches memory the driver placed there. Because each > >> device owns its region, its emulation can run in a separate host process > >> and be terminated independently, and devices no longer contend over one > >> shared region. > >> > >> The device reports the region's shmid through a transport-specific > >> mechanism (dmb_shm_id in the PCI common configuration structure, DMBSHMId > >> on MMIO), so the shmid does not collide with device-specific or future > >> transport-specific shared memory regions. The driver locates the region > >> through the existing Shared Memory Regions facility. VIRTIO_F_DMB refines > >> VIRTIO_F_ACCESS_PLATFORM and is negotiated only together with it. > >> > >> To provide the confinement guarantee, the device bounds-checks every > >> driver-supplied offset against the region and sets DEVICE_NEEDS_RESET on > >> violation. > >> > >> The general requirement on shared memory regions forbids a device to > >> expose a region used to control its operation or to stream data, and a > >> Device Memory Buffer is used for both, so it is named as that > >> requirement's exception. > >> > >> The Device Memory Buffer is memory the driver shares with the device on > >> platforms that distinguish shared from private mappings, and the device > >> provides coherency without cache maintenance by the driver. A device > >> that requires the platform's barriers offers VIRTIO_F_ORDER_PLATFORM, > >> and a driver accepts VIRTIO_F_DMB only together with it. Offset 0 in > >> the region is reserved: a device that predates the feature reads a > >> Descriptor Area address of 0 as a queue that is not in use, as the > >> legacy QueuePFN register states. > >> > >> Signed-off-by: Alexander Graf <[email protected]> > >> --- > >> > >> Link: https://lore.kernel.org/virtio-comment/[email protected]/ [v1] > >> Link: https://lore.kernel.org/virtio-comment/[email protected]/ [v2] > >> > >> v1 -> v2: > >> - Replace the fixed reserved shmid 2 and its registry with a shmid the > >> device reports per transport, as Stefan Hajnoczi suggested on v1. > >> - Spell out per-transport compatibility for dmb_shm_id and DMBSHMId. > >> - Set FAILED when the driver cannot find the region, rather than refuse > >> to negotiate. > >> - List VIRTIO_F_DMB under CCW's features reserved for future use. > >> > >> v2 -> v3: > >> - Require the region to be memory the driver shares with the device. > >> - Require coherency; couple VIRTIO_F_DMB to VIRTIO_F_ORDER_PLATFORM, > >> replacing v2's sentence on the areas' interaction with it. > >> - Reserve offset 0 in the region. > >> - Except the Device Memory Buffer from the shared-region prohibition. > >> - Allow a device to expose further shared memory regions. > >> - Move bit 44 into the queue and feature negotiation reserved list. > >> - Aim two cross-references at the Reserved Feature Bits chapter. > >> > >> > >> conformance.tex | 2 + > >> content.tex | 28 ++++++- > >> shared-mem.tex | 199 ++++++++++++++++++++++++++++++++++++++++++++- > >> transport-ccw.tex | 1 + > >> transport-mmio.tex | 15 ++++ > >> transport-pci.tex | 20 +++++ > >> 6 files changed, 262 insertions(+), 3 deletions(-) > >> > >> diff --git a/conformance.tex b/conformance.tex > >> index 9af31e2..cf4259d 100644 > >> --- a/conformance.tex > >> +++ b/conformance.tex > >> @@ -104,6 +104,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} > >> \item \ref{drivernormative:General Initialization And Device Operation / Device Initialization} > >> \item \ref{drivernormative:General Initialization And Device Operation / Device Cleanup} > >> \item \ref{drivernormative:Reserved Feature Bits} > >> +\item \ref{drivernormative:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer} > >> \item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities} > >> \item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects} > >> \item \ref{drivernormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device parts} > >> @@ -191,6 +192,7 @@ \section{Conformance Targets}\label{sec:Conformance / Conformance Targets} > >> \item \ref{devicenormative:Basic Facilities of a Virtio Device / Packed Virtqueues / The Virtqueue Descriptor Table} > >> \item \ref{devicenormative:Basic Facilities of a Virtio Device / Packed Virtqueues / Scatter-Gather Support} > >> \item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions} > >> +\item \ref{devicenormative:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer} > >> \item \ref{devicenormative:Reserved Feature Bits} > >> \item \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device and driver capabilities} > >> \item \ref{devicenormative:Basic Facilities of a Virtio Device / Device groups / Group administration commands / Device resource objects} > >> diff --git a/content.tex b/content.tex > >> index 243ce2d..8c5a844 100644 > >> --- a/content.tex > >> +++ b/content.tex > >> @@ -102,10 +102,10 @@ \section{Feature Bits}\label{sec:Basic Facilities of a Virtio Device / Feature B > >> \begin{description} > >> \item[0 to 23, 41, 42 and 50 to 127] Feature bits for the specific device type > >> > >> -\item[24 to 40, and 43] Feature bits reserved for extensions to the queue and > >> +\item[24 to 40, 43 and 44] Feature bits reserved for extensions to the queue and > >> feature negotiation mechanisms, see \ref{sec:Reserved Feature Bits} > >> > >> -\item[44 to 49, and 128 and above] Feature bits reserved for future extensions. > >> +\item[45 to 49, and 128 and above] Feature bits reserved for future extensions. > >> \end{description} > >> > >> \begin{note} > >> @@ -946,6 +946,15 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} > >> suspend the device by set the SUSPEND bit to 1. > >> See \ref{sec:Basic Facilities of a Virtio Device / Device Status Field}. > >> > >> + \item[VIRTIO_F_DMB(44)] This feature indicates that the device offers a > >> + Device Memory Buffer: a shared memory region, owned by the device, > >> + that holds the virtqueues and the buffers they reference. When this feature > >> + is negotiated, every address that the driver places in a virtqueue is an > >> + offset into the Device Memory Buffer region rather than a physical address > >> + or a bus address. The device is thereby confined to memory the driver has > >> + published in that region. > >> + See \ref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}~\nameref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}. > >> + > >> \end{description} > >> > >> \drivernormative{\section}{Reserved Feature Bits}{Reserved Feature Bits} > >> @@ -982,6 +991,13 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} > >> > >> A driver SHOULD accept VIRTIO_F_NOTIF_CONFIG_DATA if it is offered. > >> > >> +A driver MUST NOT accept VIRTIO_F_DMB unless it also accepts > >> +VIRTIO_F_ACCESS_PLATFORM. If VIRTIO_F_ORDER_PLATFORM is offered, a driver > >> +MUST NOT accept VIRTIO_F_DMB unless it also accepts VIRTIO_F_ORDER_PLATFORM. > > > > Why is that? access platform is for DMA from device of which > > there's none here. Both it and order platform are useful to disable > > to speed up software configs. > > I agree: combining VIRTIO_F_DMB and VIRTIO_F_ACCESS_PLATFORM shouldn't > be combined. I'd treat the combination as an error. Nope. Just let the decision to devices. Again, there needs to be a reason for such restrictions. In fact, access platform is mostly "dma is limited by a viommu". Since with dmb does not need a viommu to be secure, it is in fact reasonable to have a device that offers both as ways to be secure, accepts either VIRTIO_F_DMB or VIRTIO_F_ACCESS_PLATFORM but fails if none negotiated. > >> +For the driver requirements that apply once VIRTIO_F_DMB has been > >> +negotiated, see > >> +\ref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}~\nameref{sec:Basic Facilities of a Virtio Device / Shared Memory Regions / Device Memory Buffer}. > >> + > >> \devicenormative{\section}{Reserved Feature Bits}{Reserved Feature Bits} > >> > >> A device MUST offer VIRTIO_F_VERSION_1. A device MAY fail to operate further > >> @@ -1011,6 +1027,14 @@ \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits} > >> and presents a PCI SR-IOV capability structure, otherwise > >> it MUST NOT offer VIRTIO_F_SR_IOV. > >> > >> +A device that offers VIRTIO_F_DMB MUST also offer VIRTIO_F_ACCESS_PLATFORM. > >> +A device that offers VIRTIO_F_DMB MUST also offer VIRTIO_F_ORDER_PLATFORM if > >> +the device requires the driver to use memory barriers suitable for devices > >> +described by the platform when accessing the Device Memory Buffer region. > > > > This is really vague. On most platforms drivers can map > > in any way they want. What is expected here? What barriers to > > use will very much depend on that. > > > > In particular, how do you expect this mapped? If cached what forces > > the data out of the CPU cache? > > My understanding is that VIRTIO_F_DMB is only a good choice for > untrusted devices implemented in software. Both the original use-case > (protected VMs on Android) and my use-case (Spectrum VMs using untrusted > virtio-vhost-user devices) fall into this category. In this case, > the DMB is just ordinary RAM, so VIRTIO_F_ORDER_PLATFORM isn't needed. > > On the other hand, hardware devices shouldn't implement VIRTIO_F_DMB. > This would require the CPU to read data over PCIe, which is slow. The device can be on a quick bus with cache snooping. > Instead, the driver should use a bounce buffer in system RAM, which > provides the same level of security (assuming hardware DMA protection > of some sort). This is none of spec's business. > I'd be fine with making VIRTIO_F_DMB and VIRTIO_F_ORDER_PLATFORM > mutually exclusive. Forbid devices from offering both, and require > drivers to refuse to use a device that does. Reconsider once a > sensible use-case appears. We should not deal with policy and use-cases. Just provide building blocks. > -- > Sincerely, > Demi Marie Obenour (she/her/hers)