Re: Wayland gtk windowing (glimagesink)

Did you call set_window_handle(wl_handle) when on_sync_message triggered?

On Tue, 30 Apr 2024 at 03:04, Gregoire Gentil <[email protected]> wrote:

> Hello Max,
>
> Based on your python code, I have added:
>
> GdkDisplay *display = gdk_display_get_default();
> struct wl_display *wl_handle = gdk_wayland_display_get_wl_display(display);
> GstContext *context =
> gst_context_new("GstWaylandDisplayHandleContextType", True);
> GstStructure *structure = gst_context_writable_structure(context);
> gst_structure_set(structure, "display", G_TYPE_POINTER, wl_handle, NULL);
> gst_element_set_context(data.video_sink, context);
>
> That's the only difference I see. But I'm still receiving the error
> message:
> Error received from element waylandsink0: Application did not provide a
> wayland display handle
> Debugging information: ../ext/wayland/gstwaylandsink.c(1102):
> gst_wayland_sink_set_window_handle ():
> /GstPipeline:pipeline0/GstWaylandSink:waylandsink0:
> waylandsink cannot use an externally-supplied surface without an
> externally-supplied display handle. Consider providing a display handle
> from your application with GstContext
>
> Any idea what I'm missing else?
>
> Grégoire
>
>
> On 4/28/24 23:53, Max Weng wrote:
> > I forgot to mention that handling window operations in Wayland is a bit
> > different from X11.there is a c example for wayland gtk in the gstreamer
> > repo, but I can not find it now.
> > here is my python code, shall be easy convert to C++
> >
> > libgdk = ctypes.CDLL(find_library("libgdk-3"))
> > libgst = ctypes.CDLL(find_library("libgstreamer-1.0"))
> >
> > def get_wayland_window_handle(widget):
> > return
> libgdk.gdk_wayland_window_get_wl_surface(hash(widget.get_window()))
> >
> >
> > def get_default_wayland_display_context():
> > wl_display = libgdk.gdk_wayland_display_get_wl_display(
> > hash(Gdk.Display.get_default())
> > )
> > context = Gst.Context.new("GstWaylandDisplayHandleContextType", True)
> > structure = libgst.gst_context_writable_structure(hash(context))
> > libgst.gst_structure_set(
> > structure,
> > ctypes.c_char_p("display".encode()),
> > hash(GObject.TYPE_POINTER),
> > wl_display,
> > 0,
> > )
> > return context
> >
> > def set_window(self):
> > if self.window is not None:
> > # window already set, when refresh/update, do not create new window
> > return
> > window = Gtk.Window()
> > window.set_title("Video Streaming")
> > self.window = window
> > self.window.connect("destroy", Gtk.main_quit, "WM destroy")
> > window.set_default_size(1200, 800)
> > self.window.show_all()
> > self.window.fullscreen()
> >
> > def set_stream(self, url):
> > command = f"rtspsrc name=rtspsrc location={url}protocols=tcp latency=10
> > max-rtcp-rtp-time-diff=10 ! errorignore ! rtph265depay name=depay !
> > h265parse ! avdec_h265 name=avdec ! videoconvert ! video/x-raw,
> > format=RGBA ! waylandsink name=sink"
> > self.logger.info(f"command {command}")
> > self.pipeline = Gst.parse_launch(command)
> >
> > bus = self.pipeline.get_bus()
> > bus.add_signal_watch()
> > bus.connect("message", self.on_message)
> >
> > if self.drawing_area is not None:
> > self.window.remove(self.drawing_area)
> > self.drawing_area = Gtk.DrawingArea()
> > # set background color
> > self.drawing_area.modify_bg(
> > Gtk.StateType.NORMAL, Gdk.color_parse(self.background_color_str)
> > )
> >
> > glsink = self.pipeline.get_by_name("sink")
> >
> > # Wayland display context wrapped as a GStreamer context.
> > wl_display = get_default_wayland_display_context()
> > glsink.set_context(wl_display)
> >
> > self.window.connect("delete-event", Gtk.main_quit)
> > self.window.add(self.drawing_area)
> > self.drawing_area.realize()
> >
> > self.window.show_all()
> >
> > bus.enable_sync_message_emission()
> > bus.connect("sync-message", self.on_sync_message)
> >
> > def on_sync_message(self, bus: Gst.Bus, message: Gst.Message):
> > if message.get_structure() is None:
> > return 1
> >
> > if message.get_structure().get_name() == "prepare-window-handle":
> > self.logger.info(f"[on_sync_message]
> {message.get_structure().get_name()}")
> > sink = message.src
> > self.logger.info(f"[on_sync_message] sink: {sink}")
> >
> > # Wayland window handle.
> > wl_handle = get_wayland_window_handle(self.drawing_area)
> > sink.set_window_handle(wl_handle)
> >
> > allocation = self.drawing_area.get_allocation()
> > sink.set_render_rectangle(
> > allocation.x, allocation.y, allocation.width, allocation.height
> > )
> >
> > On Mon, 29 Apr 2024 at 12:15, Gregoire Gentil <[email protected]
> > <mailto:[email protected]>> wrote:
> >
> >     Thanks for helping. Still not working.
> >
> >     I have replaced glimagesink by waylandsink. If I don't call
> >     "gst_video_overlay_set_window_handle", the video appears in the
> >     back. So
> >     at least, waylandsink is working. But note that the video is
> appearing
> >     without being part of any window at all.
> >
> >     If I call "gst_video_overlay_set_window_handle", the video doesn't
> >     appear and doesn't play. I also receive the following message:
> >
> >     waylandsink cannot use an externally-supplied surface without an
> >     externally-supplied display handle. Consider providing a display
> handle
> >     from your application with GstContext
> >
> >     Am I missing another API call?
> >
> >     Grégoire
> >
> >
> >     On 4/28/24 20:28, Max Weng wrote:
> >      > I understand your situation better now. :) I've noticed that the
> >     video
> >      > indeed appears in a separate window with other sinks as well. It
> >     seems
> >      > that in my experience, only |waylandsink|consistently integrates
> >      > directly into the GTK window without issues.
> >      >
> >      > On Mon, 29 Apr 2024 at 10:51, Gregoire Gentil
> >     <[email protected] <mailto:[email protected]>
> >      > <mailto:[email protected] <mailto:[email protected]>>> wrote:
> >      >
> >      >     I could have been more precise: the glimageslink pipeline
> >     does work and
> >      >     the video appears in a separated window if I don't link it to
> the
> >      >     window
> >      >     with the code mentioned below. What I want to do is to insert
> >     the video
> >      >     inside the gtk window.
> >      >
> >      >     Also glimagesink has the unique advantage to work on both
> >     wayland and
> >      >     non-wayland systems,
> >      >
> >      >     Grégoire
> >      >
> >      >
> >      >
> >      >
> >      >     On 4/28/24 19:12, Max Weng wrote:
> >      >      > try use waylandsink?
> >      >      >
> >      >      > this work for me "rtspsrc name=rtspsrc location={url}
> >     protocols=tcp
> >      >      > latency=10 max-rtcp-rtp-time-diff=10 ! errorignore !
> >     rtph265depay
> >      >      > name=depay ! h265parse ! avdec_h265 name=avdec !
> >     videoconvert !
> >      >      > video/x-raw, format=RGBA ! waylandsink name=sink"
> >      >      >
> >      >      > On Mon, 29 Apr 2024 at 06:18, Gregoire Gentil via
> >     gstreamer-devel
> >      >      > <[email protected]
> >     <mailto:[email protected]>
> >      >     <mailto:[email protected]
> >     <mailto:[email protected]>>
> >      >      > <mailto:[email protected]
> >     <mailto:[email protected]>
> >      >     <mailto:[email protected]
> >     <mailto:[email protected]>>>> wrote:
> >      >      >
> >      >      >     Hello,
> >      >      >
> >      >      >     I had an application with a gtk+-3.0 window and a
> >     pipeline "... !
> >      >      >     imagesink". The video is rendered inside the gtk
> window.
> >      >      >
> >      >      >     I was doing:
> >      >      >     static void realize_cb(GtkWidget *widget, CustomData
> >     *data) {
> >      >      >     GdkWindow *window = gtk_widget_get_window(widget);
> >      >      >     data->window_handle = GDK_WINDOW_XID(window);
> >      >      >
> >      >      >     and then:
> >      >      >
> >      >      >     static GstBusSyncReply bus_sync_handler(GstBus * bus,
> >      >     GstMessage *
> >      >      >     message, CustomData *data) {
> >      >      >
> >      >
> >
>  gst_video_overlay_set_window_handle(GST_VIDEO_OVERLAY(data->video_sink),
> >      >      >     data->window_handle);
> >      >      >
> >      >      >     Everything was working fine. I want now to support
> wayland
> >      >     system.
> >      >      >
> >      >      >     Pipeline becomes: "... ! glimagesink".
> >      >      >
> >      >      >     in the realize_cb function, I have now:
> >      >      >     GdkWindow *window = gtk_widget_get_window(widget);
> >      >      >     data->window_handle =
> >      >      >     (guintptr)gdk_wayland_window_get_wl_surface(window);
> >      >      >
> >      >      >
> >      >      >     At run-time, I'm getting hundreds of:
> >      >      >     "GStreamer-GL-CRITICAL **: 14:06:48.263: Failed to
> >     flush Wayland
> >      >      >     connection"
> >      >      >
> >      >      >     What am I doing wrong? Many thanks in advance for any
> >     hint,
> >      >      >
> >      >      >     Grégoire
> >      >      >
> >      >
> >
>