Re: Help with vaapisink

On Fri, 09 Jun 2023 at 09:22, Davide Molteni wrote:
> Hi Victor,
> Thanks for the quick reply.
> I've tried to install Weston and the basic pipeline works correctly.
> The issue now is when I actually try to run a test sample of our application
> (basically a grid that displays multiple video sources). When I've run the
> test software without the window manger directly with fbdevsink I was able to
> display a 3x3 view, but with Weston and vaapisink even a 2x2 view is displayed
> with very poor fps.
> Is it due to the compositor component? I saw that in gstva there is an
> additional vacompositor but unfortunately it is available only from gstreamer
> 1.22 (Ubuntu 23.04), our distribution is not yet ready for it.

I would say it's due compositor, since it uses system memory based frames, so VA
surfaces have to be downloaded from GPU memory to system memory and uploaded
again.

Depending on your driver and gstreamer version, but in gstreamer-vaapi there's a
vaapioverlay element, that is the old version of the vacompositor (vaapioverlay
is very ill named).

> 
> Core of the test application hereafter for reference:
> 
> /* Create gstreamer elements */
>   pipeline = gst_pipeline_new ("vaapi-sample");
> 
>   conv_rgb  = gst_element_factory_make ("vaapipostproc",  "RGBAconverter0");
>   sink      = gst_element_factory_make ("vaapisink", "video-output");
> 
>   /* we add a message handler */
>   bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
>   bus_watch_id = gst_bus_add_watch (bus, bus_call, loop);
>   gst_object_unref (bus);
> 
>   GstElement *compositor = gst_element_factory_make("compositor", "compositor");
>   g_object_set(G_OBJECT(compositor), "background", 1, NULL);
> 
>   /* we add all elements into the pipeline */
>   gst_bin_add_many (GST_BIN (pipeline),
>             compositor, conv_rgb, sink, NULL);
>   gst_element_link_many(compositor, conv_rgb, sink, NULL);
>   for (int i=0; i< grid_rows*grid_columns; i++ )
>   {
>         static int curr_row;
>         static int curr_col;
> 
>         if (i==0)
>         {
>                 curr_row = 0;
>                 curr_col = 0;
>         }
>         else
>         {
>                 curr_row=i/grid_rows;
>                 curr_col=i%grid_rows;
>         }
> 
> 
> 
>         std::string ele_name_compositor_padsink = "sink_" + std::to_string(i);
> 
>         GstPadTemplate* compositor_sink_pad_template = gst_element_class_get_pad_template(GST_ELEMENT_GET_CLASS(compositor), "sink_%u");
>         GstPad *sink_pad = gst_element_request_pad(compositor, compositor_sink_pad_template, ele_name_compositor_padsink.c_str(), NULL);
> 
>         g_object_set((sink_pad), "xpos", curr_col*(display_w/grid_columns), NULL);
>         g_object_set((sink_pad), "ypos", curr_row*(display_h/grid_rows), NULL);
>         g_object_set((sink_pad), "width", display_w/grid_columns, NULL);
>         g_object_set((sink_pad), "height", display_h/grid_rows, NULL);
>         gst_pad_set_active(sink_pad, TRUE);
>         if (target_pad == NULL)
>                 target_pad = sink_pad;
> 
>         file_n = i+1;
>         if (file_n >= argc) file_n = 1;
> 
>         //FILE SRC
>         sprintf(Tmp, "filesrc_%d", i);
>         source[i] = gst_element_factory_make("filesrc", Tmp);
>         g_object_set(G_OBJECT(source[i]), "location", argv[file_n], NULL);
> 
>         sprintf(Tmp, "qtdemux_%d", i);
>         demux[i] = gst_element_factory_make("qtdemux", Tmp);
>         sprintf(Tmp, "queue_%d", i);
>         queue[i] = gst_element_factory_make("queue", Tmp);
>         g_object_set (G_OBJECT (queue[i]), "flush-on-eos", 1 , NULL);
> 
>         obj[i].idx = i;
>         obj[i].queue = queue[i];
> 
>         if (demux[i] && queue[i]) {
>                 g_signal_connect(demux[i], "pad-added", G_CALLBACK(pad_added_handler), (gpointer) &obj[i]);
>         }
> 
> 
>         sprintf(Tmp, "parser_%d", i);
>         parser[i] = gst_element_factory_make("h264parse", Tmp);
>         sprintf(Tmp, "dec_%d", i);
>         dec[i] = gst_element_factory_make("vaapih264dec", Tmp); //only h.264
> 
>         gst_bin_add_many (GST_BIN (pipeline),
>             source[i], demux[i], queue[i], parser[i], dec[i], NULL);
> 
>         gst_element_link_many(source[i], demux[i], NULL);
>         gst_element_link_many(queue[i], parser[i], dec[i], NULL);
>         gst_element_link_pads(dec[i], "src", compositor, ele_name_compositor_padsink.c_str());
>   }
> 
>   g_print ("Now playing\n");
>   gst_element_set_state (pipeline, GST_STATE_PLAYING);
> 
>   
> 
> -----Original Message-----
> From: gstreamer-devel <[email protected]> On Behalf Of Victor Jaquez via gstreamer-devel
> Sent: giovedì 8 giugno 2023 11:21
> To: [email protected]
> Cc: Victor Jaquez <[email protected]>
> Subject: Re: Help with vaapisink
> 
> 
> Hi Davide,
> 
> vaapisink doesn't draw anything with its DRM backend, its drawing method is a no-op, while, either X11 or Wayland, they draw.
> 
> vaapisink is thought to be used in X11 or Wayland only. If you are willing to draw directly with DRM you can try kmssink, though zero copy might not work.
> 
> Another comment, for new code gstva, in gst-plugins-bad, is preferred over gstreamer-vaapi (check with gst-inspect-1.0 va)
> 
> vmjl
> 
> On Wed, 07 Jun 2023 at 12:43, Davide Molteni via gstreamer-devel wrote:
> > Hello,
> > I'm trying to test a simple pipeline using VA-API components, I'm running it on an custom system without any window manager (Yocto distribution), the goal is to use DRM, the pipeline is the following:
> >
> > gst-launch-1.0 filesrc location=$1 ! qtdemux ! queue ! h264parse ! 
> > vaapih264dec ! vaapipostproc ! 
> > 'video/x-raw(memory:VASurface),format=BGRx'! vaapisink display=4 
> > display-name=/dev/dri/renderD128
> >
> > The pipeline works correctly and reaches the PLAYING state but I don't see anything on the HDMI monitor, if I use fbdevsink instead I see the decoded video (but obviously it consumes a huge amount of CPU).
> >
> > Hereafter some additional information:
> >
> > root@mn-el00d08919cf41:~# vainfo
> > error: can't connect to X server!
> > libva info: VA-API version 1.14.0
> > libva info: User environment variable requested driver 'i965'
> > libva info: Trying to open /usr/lib/dri/i965_drv_video.so libva info: 
> > Found init function __vaDriverInit_1_14 libva info: va_openDriver() 
> > returns 0
> > vainfo: VA-API version: 1.14 (libva 2.14.0)
> > vainfo: Driver version: Intel i965 driver for Intel(R) Broxton - 2.4.1
> > vainfo: Supported profile and entrypoints
> >       VAProfileMPEG2Simple            : VAEntrypointVLD
> >       VAProfileMPEG2Main              : VAEntrypointVLD
> >       VAProfileH264ConstrainedBaseline: VAEntrypointVLD
> >       VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
> >       VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
> >       VAProfileH264Main               : VAEntrypointVLD
> >       VAProfileH264Main               : VAEntrypointEncSlice
> >       VAProfileH264Main               : VAEntrypointEncSliceLP
> >       VAProfileH264High               : VAEntrypointVLD
> >       VAProfileH264High               : VAEntrypointEncSlice
> >       VAProfileH264High               : VAEntrypointEncSliceLP
> >       VAProfileH264MultiviewHigh      : VAEntrypointVLD
> >       VAProfileH264MultiviewHigh      : VAEntrypointEncSlice
> >       VAProfileH264StereoHigh         : VAEntrypointVLD
> >       VAProfileH264StereoHigh         : VAEntrypointEncSlice
> >       VAProfileVC1Simple              : VAEntrypointVLD
> >       VAProfileVC1Main                : VAEntrypointVLD
> >       VAProfileVC1Advanced            : VAEntrypointVLD
> >       VAProfileNone                   : VAEntrypointVideoProc
> >       VAProfileJPEGBaseline           : VAEntrypointVLD
> >       VAProfileJPEGBaseline           : VAEntrypointEncPicture
> >       VAProfileVP8Version0_3          : VAEntrypointVLD
> >       VAProfileVP8Version0_3          : VAEntrypointEncSlice
> >       VAProfileHEVCMain               : VAEntrypointVLD
> >       VAProfileHEVCMain               : VAEntrypointEncSlice
> >       VAProfileHEVCMain10             : VAEntrypointVLD
> >       VAProfileVP9Profile0            : VAEntrypointVLD
> >
> > root@mn-el00d08919cf41:~# lshw -disable mmc -c video
> >   *-display
> >        description: VGA compatible controller
> >        product: HD Graphics 500
> >        vendor: Intel Corporation
> >        physical id: 2
> >        bus info: pci@0000:00:02.0
> >        version: 0b
> >        width: 64 bits
> >        clock: 33MHz
> >        capabilities: pciexpress msi pm vga_controller bus_master cap_list rom
> >        configuration: driver=i915 latency=0
> >        resources: irq:129 memory:90000000-90ffffff 
> > memory:80000000-8fffffff ioport:3000(size=64) memory:c0000-dffff
> >
> >
> > Am I missing something?
> >
> > Thanks,
> > Davide
>