Re: [PATCH v9 0/1] virtio-media: Add device specification

Demi Marie Obenour <[email protected]> Wed, 9 Jul 2025 13:26:46 -0400
Newsgroups	dev.linux.lists.virtio-comment
Message-ID	<[email protected]>
On 7/9/25 05:34, Albert Esteve wrote:
> New attempt of including virtio-media
> device specification.
> 
> v8->v9:
> - Rebased to virtio-1.4
> 
> v7->v8:
> - Merged DQBUF and DQEVENT into the same
>   item for unsupported ioctls
> - Rewrote/clarified a couple paragraphs
>   based on previous version reviews.
> 
> Virtio-media came from a discussion on virtio-dev
> mailing list, which lead to Alex Courbot presenting
> virtio-v4l2[1] specification as an alternative to
> virtio-video.
> 
> Later, virtio-v4l2 was renamed to virtio-media[2]
> and published at:
> 
> https://github.com/chromeos/virtio-media
> 
> The repository above includes a virtio-media driver able
> to pass v4l2-compliance when proxying the vivid/vicodec
> virtual devices or an actual UVC camera using the
> V4L2 vhost device (available in the repository).
> It also includes a FFmpeg-based video encoder
> device. Steps to reproduce are also detailed[3].
> 
> Recently, virtio-media has landed in AOSP[4].
> Also the driver patch has been sent to the
> kernel, currently in its v2 [5].
> 
> Furthermore, virtio-media got a proposal to reserve
> device ID 48, which was finally approved for
> inclusion in v1.4.
> 
> There is some overlap with virtio-video in regards
> to which devices it can handle. However, they take
> different approaches, potentially making them
> the preferable choice for different scenarios.
> Furthermore, virtio-media has matured since it
> was first presented, and there are public efforts
> to have upstream driver and devices.
> 
> But mainly, given that virtio-media will be the virtualization
> solution for media devices for ChromeOS, Android, and
> possibly others, I think is important that it gets
> standardized and included in the specification,
> despite the aforementioned overlap.
> 
> Full PDF: https://drive.google.com/file/d/1sDZXWVlvPbHKKguhekCXMzkpNZZ0g3hV/view?usp=sharing
> PDF with the media section only: https://drive.google.com/file/d/1JjOIzecnkFFlpFBRjPQEUyAOD3BDHAdw/view?usp=sharing
> 
> [1] https://mail.google.com/mail/u/0?ui=2&ik=73ebd65ebd&attid=0.1&permmsgid=msg-f:1767388565327924962&th=1887068940754ee2&view=att&disp=inline&realattid=f_libalimc0
> [2] https://www.mail-archive.com/[email protected]/msg12665.html
> [3] https://github.com/chromeos/virtio-media/blob/main/TRY_IT_OUT.md
> [4] https://cs.android.com/android/platform/superproject/main/+/main:external/virtio-media/
> [5] https://lore.kernel.org/lkml/[email protected]/T/
> 
> Albert Esteve (1):
>   virtio-media: Add virtio media device specification
> 
>  conformance.tex                           |   6 +
>  content.tex                               |   1 +
>  device-types/media/description.tex        | 639 ++++++++++++++++++++++
>  device-types/media/device-conformance.tex |  15 +
>  device-types/media/driver-conformance.tex |  11 +
>  5 files changed, 672 insertions(+)
>  create mode 100644 device-types/media/description.tex
>  create mode 100644 device-types/media/device-conformance.tex
>  create mode 100644 device-types/media/driver-conformance.tex

It's probably too late, but I have some serious concerns about this
device.  Specifically:

1. I don't see a reasonable way to support untrusted virtio-media
   devices.  v4l2 has too many ioctls and I can't realistically
   see a way to enforce that the return values from all of them
   are consistent with each other.  It is possible to only allow
   a fixed format (such as uncompressed ARGB) and only allow the
   video source and/or sink to provide resolutions and frame rates,
   but this means losing a lot of performance and abandoning any
   attempt at zero-copy.

   The use-case for untrusted virtio-media devices is not
   confidential computing, but rather disaggregated systems where
   video sources (like webcams) may be attached to untrusted virtual
   machines.

2. v4l2 is a chatty protocol and using it implies a lot of
   guest <=> host round-trips.  This is bad for performance.
   Is this overhead actually significant, and if so, are there
   plans to reduce it?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
OpenPGP_0xB288B55FFF9C22C1.asc (application/pgp-keys, 7 KB)
OpenPGP_signature.asc (application/pgp-signature, 833 B)