Re: gstramer\v4l2loopback userptr mode

Le mercredi 23 août 2023 à 00:38 +0200, Regine Issan a écrit :
> Ps: regarding vivid
> https://www.kernel.org/doc/html/v4.8/media/v4l-drivers/vivid.html#webcam-input
> 
> I see that the frame size is limited.
> 
> Why does the limitation exist ?

To better simulate a camera ?

> 
> What is gstreamer limitation ? 

Hard to say, anything we don't support ?

> 
> What would you recommend if I need bigger sizes ?

There is an "input", I think you get different set of resolutions for different
input. Otherwise, modify the code, its a software driver, no hw involved.

> 
>  
> 
> On Wed, Aug 23, 2023 at 00:20 Regine Issan <[email protected]> wrote:
> > Hello Nicolas !
> > Thank you ,again , for your kind replay.
> > 
> > Would you be so kind to see my comments/questions  below (with “=>comment2 “
> > prefix)
> > 
> > On Tue, Aug 22, 2023 at 21:03 Nicolas Dufresne <[email protected]> wrote:
> > > Le mardi 22 août 2023 à 12:14 +0300, Regine Issan via gstreamer-devel a
> > > écrit :
> > > > Hello  Everyone,
> > > > 
> > > > I have an app that captures frames from a  video device using v4l2.
> > > > the app should support all i-o modes.
> > > > I am mostly interested in userptr mode.
> > > > 
> > > > when i run the code with physical camera it works (with userptr mode).
> > > > when i run the same code with v4l2loopback camera and gstreamer, 
> > > > it fails. calling VIDIOC_REQBUFS with V4L2_MEMEORY_USERPTR returns not
> > > > supported.
> > > > 
> > > > in in link:
> > > > https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/1028
> > > > Nicolas Dufresne pointed me out that v4l2loopback does not support
> > > > userptr.
> > > > I find his anser very helpful and consider my next steps.
> > > > 
> > > > 
> > > > several questions:
> > > > 
> > > > 1.  can i count that gstreamer works with user ptr ?
> > > >      the link  below suggests it sohuld work, correct ?
> > > >                
> > > > https://developer.ridgerun.com/wiki/index.php/GStreamer1.0_and_V4L2_User
> > > > Pt
> > > > 
> > > > 2. 
> > > >      clearly, i am looking  for simple stable solution.
> > > >     as much as I love code, clearly  it is preferable to use existing 
> > > > tools
> > > >     to get fast and stable simulator.
> > > >      if you could please advise ,which of the following options would
> > > > you recommend ?
> > > >       option1: use gstreamer\v4l2sink without v4l2loopback
> > > >                     my question here: which tool can i use instead 
> > > > v4l2loopback ?
> > > 
> > > 
> > > For testing purpose, you can build vivid virtual driver, it sumulates a
> > > camera, and implements USRPTR.
> > 
> > =>comment2:begin
> >      Thanks, that seems the simplest solution.
> >       I will have a look at vivid. 
> > =>comment2:end
> >     
> > 
> > > 
> > > >                    which tool did v4l2sink develpers used to check the
> > > > usertpr mode ? 
> > > 
> > > Its has been added over 15 years ago, so I cannot say for sure. These days
> > > we prefer DMABuf importation instead, though its not yet automatically
> > > using it. Validation is difficult and mistakes will lead to error and
> > > garbled images.
> > > 
> > 
> > 
> > 
> > =>comment2:begin
> >      I read about  DMABuf here
> >  
> >   
> > https://stackoverflow.com/questions/66962795/what-is-the-use-of-mmap-userptr-and-dmabuf-in-video-streaming-using-v4l2-drive
> > 
> >  The consumer reading dmabufs must run
> >   In kernel mode . Correct ?

No, you can mmap() the dmabuf for CPU access. The GstMemory/GstBuffer abstract
this for you.

> >  
> >     Vslidation using userptr and dmabufs
> >     Are equally complicated ? 

DMAbufs object in the kernel have state that helps a lot. It also allow
identifying the exporter. Finally, DMABuf are guarantied page aligned.

> >    
> >      
> > =>comment2:end
> >    
> >     
> > > >         option2:any other gstreamer alternative for
> > > > v4l2sink\v4l2loopback ?
> > > 
> > > I'm not aware of equivalent of v4l2sink.
> > > 
> > > >         option3: changing v4l2loopback to support userptr 
> > > >                       i am not video expert, in fact, quite new.
> > > >                       I wonder whyv4l2loopback did not support userptr
> > > > mode
> > > >                      dou you think this is a hude task ?
> > > >                       can you see Any specific challenge ? I
> > > > 
> > > >                     PS: my userptr works with shared memeory
> > > >                            it should be accessible in user and kernel
> > > > mode 
> > > >                            (as the code worked with physical camera)
> > > 
> > > 
> > > v4l2loopback is an out of tree driver that does not use the videobuf2
> > > helpers, so adding something like userptr is for sure a bit of work and
> > > you have to understand how strides and plane offset calculation works.
> > 
> > =>comment2:begin
> >  
> >  I though the only difference 
> >   Between mmap and userptr 
> >    should  only be 
> >    on requestbufs and cleanup .
> > Mmmm…am I missing anything ?
> >   strides and plane offset ? 

When you import foreign buffers, the foreign chosen padding may differ. Not all
driver can adapt to any foreign padding. The padding is set in S_FMT
width/height, and bytesperline. If you have complicated plane offsets, you have
to use the MPLANE interface.

> > 
> > =>comment2:end
> > 
> > > >         option4: writing a tool that replace v4l2loopback and gstramer 
> > > >         My single goal is to mimic A camera driver
> > > >         that reads images from disk and writes it to v4l2 using userptr.
> > > > 
> > > >       so maybe I should just write an app,
> > > >       that declares the device and writes to v4l2 directly , just like
> > > > driver do ?
> > > >  
> > > 
> > > 
> > > Also seems like quite some coding, but if you go that way, have a look at
> > > libv4l2, you could make an LD_PRELOAD library that simulate a camera. Note
> > > that vivid have a "loop" control, which just like v4l2loopback allow
> > > passing data over. The only tricky bit is that the v4l2src needs to be
> > > configured to match the v4l2sink resolution and format, otherwise it will
> > > give you a test pattern.
> > > 
> > 
> > =>comment2:begin
> > Thanks ! 
> > Perhaps I should  dig into it if vivid won’t work.
> > =>comment2:end
> > 
> > > 
> > > Nicolas