Re: [RFC 24/24] WIP: rust/audio: add GStreamer backend

Hi Markus

On Mon, Dec 1, 2025 at 5:13 PM Markus Armbruster <[email protected]> wrote:
>
> [email protected] writes:
>
> > From: Marc-André Lureau <[email protected]>
> >
> > This patch introduce a rust/audio crate that replaces QEMU audio/
> > mixing/resampling code with GStreamer and Rust. It could potentially
> > remove the need for all the system-specific audio API implementation,
> > since GStreamer has audio elements for
> > ALSA/Pipewire/PulseAudio/jack/OSX/WASAPI etc (removing ~10k loc).
> >
> > TODO:
> > - test on various system, with various configuration to see if this
> >   backend can replace the other QEMU audio backends
> > - add a spicesink/spicesrc to handle spice, or rewrite spice to use
> >   the capture approach used by VNC code. Or drop capture support, and
> >   use custom qemusrc/qemusink for both Spice and VNC, lowering the feature
> >   and behaviour disparity.
> > - build-sys: make gstreamer optional
> > - build-sys: loadable module support
> > - investigate dropping get_buffer_size_out()
> > - investigate improving emulated devices to not require regular
> >   timers (appsrc need-data is called once)
> > - add generic audio backend tests
> > - more tests for the mixing/liveadder behaviour (synchronization)
> > - other: replace audio/dbus with a rust implementation (not using gstreamer)
> >
> > Signed-off-by: Marc-André Lureau <[email protected]>
> > ---
> >  qapi/audio.json               |   29 +
> >  audio/audio-driver_template.h |    2 +
> >  rust/audio/wrapper.h          |   27 +
> >  audio/audio.c                 |    5 +
> >  Cargo.lock                    |  572 ++++++++++++++++--
> >  Cargo.toml                    |    6 +
> >  audio/trace-events            |    5 +
> >  rust/audio/Cargo.toml         |   29 +
> >  rust/audio/build.rs           |   49 ++
> >  rust/audio/meson.build        |   75 +++
> >  rust/audio/src/audio.rs       |  516 ++++++++++++++++
> >  rust/audio/src/bindings.rs    |   32 +
> >  rust/audio/src/gstreamer.rs   | 1070 +++++++++++++++++++++++++++++++++
> >  rust/audio/src/lib.rs         |   99 +++
> >  rust/meson.build              |    6 +
> >  15 files changed, 2467 insertions(+), 55 deletions(-)
> >  create mode 100644 rust/audio/wrapper.h
> >  create mode 100644 rust/audio/Cargo.toml
> >  create mode 100644 rust/audio/build.rs
> >  create mode 100644 rust/audio/meson.build
> >  create mode 100644 rust/audio/src/audio.rs
> >  create mode 100644 rust/audio/src/bindings.rs
> >  create mode 100644 rust/audio/src/gstreamer.rs
> >  create mode 100644 rust/audio/src/lib.rs
> >
> > diff --git a/qapi/audio.json b/qapi/audio.json
> > index 2df87b9710..76dc7cbfa6 100644
> > --- a/qapi/audio.json
> > +++ b/qapi/audio.json
> > @@ -128,6 +128,33 @@
> >      '*out':       'AudiodevAlsaPerDirectionOptions',
> >      '*threshold': 'uint32' } }
> >
> > +    ##
> > +    # @AudiodevGStreamerOptions:
> > +    #
> > +    # Options of the GStreamer audio backend.
> > +    #
> > +    # @in: options of the capture stream
> > +    #
> > +    # @out: options of the playback stream
> > +    #
> > +    # @sink: the name of the GStreamer sink element to use
> > +    #        (default 'autoaudiosink')
> > +    #
> > +    # @source: the name of the GStreamer source element to use
> > +    #        (default 'autoaudiosrc')
>
> Are 'autoaudiosink' and 'autoaudiosrc' well-known GStreamer names, or
> arbitrary?

Yes, they are well-known elements from the base plugins.

>
> > +    #
> > +    # Since: 11.0
> > +    ##
> > +    { 'struct': 'AudiodevGStreamerOptions',
> > +      'data': {
> > +        '*in':        'AudiodevPerDirectionOptions',
> > +        '*out':       'AudiodevPerDirectionOptions',
> > +        '*sink':      'str',
> > +        '*source':    'str'
> > +      }
> > +    }
>
> Unindent by four, and format the doc comment like this:
>
> ##
> # @AudiodevGStreamerOptions:
> #
> # Options of the GStreamer audio backend.
> #
> # @in: options of the capture stream
> #
> # @out: options of the playback stream
> #
> # @sink: the name of the GStreamer sink element to use
> #     (default 'autoaudiosink')
> #
> # @source: the name of the GStreamer source element to use
> #     (default 'autoaudiosrc')
> #
> # Since: 11.0
> ##
>
>
> > +
> > +
> >  ##
> >  # @AudiodevSndioOptions:
> >  #
> > @@ -484,6 +511,7 @@
> >              { 'name': 'sdl', 'if': 'CONFIG_AUDIO_SDL' },
> >              { 'name': 'sndio', 'if': 'CONFIG_AUDIO_SNDIO' },
> >              { 'name': 'spice', 'if': 'CONFIG_SPICE' },
> > +            { 'name': 'gstreamer' },
>
> Short form suffices:
>
>                'gstreamer',
>
> >              'wav' ] }
> >
> >  ##
> > @@ -530,6 +558,7 @@
> >                     'if': 'CONFIG_AUDIO_SNDIO' },
> >      'spice':     { 'type': 'AudiodevGenericOptions',
> >                     'if': 'CONFIG_SPICE' },
> > +    'gstreamer': { 'type': 'AudiodevGStreamerOptions' },
>
> Short form suffices:
>
>        'gstreamer': 'AudiodevGStreamerOptions',
>
> >      'wav':       'AudiodevWavOptions' } }
> >
> >  ##
>
> [...]
>
>

Thanks for the early check, I will likely have to update the schema in
future revision, for making GStreamer optional & module to the very
least.

-- 
Marc-André Lureau