Re: Video standards

Pekka Paalanen <[email protected]> Fri, 5 Apr 2024 12:34:19 +0300
Newsgroups	gmane.linux.xdg.devel
Message-ID	<20240405123419.1e9fe580@eldfell>
On Thu, 4 Apr 2024 17:13:40 -0300
salsaman <[email protected]> wrote:

> Hi,
> the problem with the drm.h header is, it is complicated, still needs
> interpretation, and it lacks some commonly used formats, (e.g YUVA4444p)

They accept additions, if the additions serve userspace
interoperability. There is no requirement to actually use the format in
the kernel.

Pixel formats are complicated, yes. There are too many pixel format
enumerations, every one differently defined, sure. I wouldn't add yet
another system of definitions.

> Also it doesn't address the gamma value (linear, sRGB, bt701), or the yuv
> subspace, (eg Y'CbCr vs bt701), the yuv ramge (16 - 240. 16 - 235 = clamped
> / mpeg. 0 - 255 unclamped, full, jpeg range) or uv sampling position, e.g
> center, top_left)

My opinion is that that none of that is relevant to a pixel format.
These are additional information that must be decoupled from the pixel
format to avoid a combinatorial explosion of the format enumeration,
which is already massive even without them. A pixel format only
describes a part of the memory layout: which set of bits forms a raw
channel value of a pixel, and what are the channel names. Giving any
further meaning to those raw values is for other metadata.

What about colorimetry? Primaries and white point, dynamic range, plus
the difference between encoding colorimetry (container color volume)
and the usable/used colorimetry (target color volume, which is present
in e.g. HDR static metadata typical for BT.2100/PQ signals in the form
of the Mastering Display Color Volume).

What about the assumed viewing environment, if we want to go from just
stimulus towards appearance?

> I can see that having some common definitions would be useful for
> exchanging data between applications. Eg  my app gets a frame buffer and
> metadata XDG_VIDEO_PALETTE_RGB24, XDG_VIDEO_GAMMA_LINEAR
> then I know unambiguously that this is planar RGB 8:8:8 (so forget little /
> big endian) and that the values are encoded with linear (not sRGB) gamma.

> If you want to be more specific with palettes, then you could do so, but it
> might require defining metadata structs,

> I'll try to explain the rationale a bit. In the audio world it is quite
> common for apps to send audio from one to another. Generally speaking they
> would send or receive via an audio server, e.g pulseaudio, jack.
> Now imagine the same for video, 

This sounds like Pipewire. One would develop Pipewire API to carry the
necessary metadata. One could choose to follow something massive like
ITU-T H.274, or maybe follow what we are brewing for Wayland.

To my understanding, Pipewire is already becoming very common among
desktop environments for routing audio and video streams between
applications and system components and devices.


Thanks,
pq
signature.asc (application/pgp-signature, 833 B)