Re: CUDA GstBuffer output buffer with OpenCV GpuMat

I got it working, and I just wanted to close the loop on this in cse
someone is searching around for it in the future.  The only mistake I had
was how I got the data pointer.  After changing that to
GST_VIDEO_FRAME_PLANE_DATA, it all worked!

    cv::cuda::GpuMat gpu_out(
        cv::Size(
            GST_VIDEO_FRAME_WIDTH(&out_frame),
            GST_VIDEO_FRAME_HEIGHT(&out_frame)
        ),
        CV_8UC3,
        GST_VIDEO_FRAME_PLANE_DATA(&out_frame, 0),
        GST_VIDEO_FRAME_PLANE_STRIDE(&out_frame, 0));

On Thu, Jun 8, 2023 at 5:57 PM Richard Pickler <[email protected]> wrote:

> Thanks for the speedy reply.  I've replaced the standard buffer_map with
> the following
>
>
>     GstVideoFrame out_frame;
>     if (!gst_video_frame_map (&out_frame, &debayer->info, outbuf,
>                 (GstMapFlags)(GST_MAP_WRITE | GST_MAP_CUDA))) {
>         //gst_video_frame_unmap (&in_frame);
>         GST_ERROR_OBJECT (debayer, "Failed to map output buffer");
>         std::cout << "Failed to map output buffer" << std::endl;
>         return GST_FLOW_ERROR;
>     }
>
> And the gpuMat initialization:
>
>     cv::cuda::GpuMat gpu_out(
>         cv::Size(
>             GST_VIDEO_FRAME_WIDTH(&out_frame),
>             GST_VIDEO_FRAME_HEIGHT(&out_frame)),
>         CV_8UC3,
>         (char*) out_frame.data,
>         GST_VIDEO_FRAME_PLANE_STRIDE(&out_frame, 0));
>
> But no such luck.  Now I am the proud recipient of black frames.
>
> You're correct that the stride is different.  5760 vs 6144 on my 1920 wide
> image.  That seems to cause the demosaic function to overrun and cause a
> memory exception in cuda.  I then changed the gpumat initialization to:
>
>
>     cv::cuda::GpuMat gpu_out(
>         cv::Size(
>             out_frame.map[0].size / GST_VIDEO_FRAME_HEIGHT(&out_frame) / 3,
>             GST_VIDEO_FRAME_HEIGHT(&out_frame)),
>         CV_8UC3,
>         (char*) out_frame.data,
>         GST_VIDEO_FRAME_PLANE_STRIDE(&out_frame, 0));
>
> Which doesn't crash, but also doesn't produce an image.
>
> (sorry for duplicate.  I forgot to reply-all)
>
> On Thu, Jun 8, 2023 at 2:22 PM Seungha Yang <[email protected]>
> wrote:
>
>> Hi,
>>
>>
>>
>> GstCudaMemory’s stride is likely different from default one.
>>
>> You will need to use gst_video_frame_map() and
>> GST_VIDEO_FRAME_PLANE_STRIDE(frame, 0), or GstCudaMemory.info.stride[0]
>>
>>
>>
>> Regards,
>>
>> Seungha
>>
>>
>>
>> *From:* gstreamer-devel <[email protected]> *On
>> Behalf Of *Richard Pickler via gstreamer-devel
>> *Sent:* Thursday, June 8, 2023 10:05 PM
>> *To:* [email protected]
>> *Cc:* Richard Pickler <[email protected]>
>> *Subject:* CUDA GstBuffer output buffer with OpenCV GpuMat
>>
>>
>>
>> I'm struggling to use the recent cuda improvements in gstreamer, using an
>> OpenCV GpuMat to write to a CUDAMemory outbuf.  I'm was successful in
>> getting an appropriate buffer through decide/propose_allocation, but simply
>> cannot seem to write to a buffer properly.  In my transform function below,
>> I'm able to map the buffer and have the demosaic function run properly, but
>> the output stream comes out garbage.  I placed a simple cv::imwrite in the
>> code, and sure enough, the data comes through the function properly, it's
>> just not in a structure that gstreamer is happy with.
>>
>>
>>
>> Any ideas?  I've been pounding my head against this wall for a couple
>> days now.
>>
>>
>>
>> static GstFlowReturn
>> debayer_transform(GstBaseTransform *base,
>>                   GstBuffer *inbuf, GstBuffer *outbuf)
>> {
>>     Debayer *debayer = DEBAYER(base);
>>     GstCudaBaseTransform *btrans = GST_CUDA_BASE_TRANSFORM (base);
>>
>>     if (gst_buffer_n_memory (outbuf) != 1) {
>>         GST_ERROR_OBJECT (debayer, "Invalid output buffer");
>>         return GST_FLOW_ERROR;
>>     }
>>
>>     GstMemory *mem;
>>     mem = gst_buffer_peek_memory (outbuf, 0);
>>     if (!gst_is_cuda_memory (mem)) {
>>         GST_ERROR_OBJECT (debayer, "Input buffer is not CUDA");
>>         std::cout << "Input buffer is not cuda" << std::endl;
>>         return GST_FLOW_ERROR;
>>     }
>>
>>     GstMapInfo map;
>>     if(!gst_buffer_map(inbuf, &map, GST_MAP_READ))
>>     {
>>         std::cout << "Can't map" << std::endl;
>>         return GST_FLOW_ERROR;
>>     }
>>
>>     GstMapInfo outmap;
>>     if(!gst_buffer_map(outbuf, &outmap, (GstMapFlags)(GST_MAP_WRITE |
>> GST_MAP_CUDA)))
>>     {
>>         std::cout << "Can't map" << std::endl;
>>         return GST_FLOW_ERROR;
>>     }
>>
>>     cv::Mat in(
>>         cv::Size(debayer->info.width, debayer->info.height), CV_8UC1,
>>         (char*) map.data);
>>
>>     cv::cuda::GpuMat gpu_in;
>>
>>     cv::cuda::GpuMat gpu_out(
>>         cv::Size(debayer->info.width, debayer->info.height),
>>         CV_8UC3,
>>         (char*) outmap.data, debayer->info.stride[0]);
>>
>>     gpu_in.upload(in);
>>     cv::cuda::demosaicing(gpu_in, gpu_out, cv::COLOR_BayerRG2BGR);
>>     cv::Mat out(gpu_out);
>>     cv::imwrite("foo.jpg", out);
>>
>>     gst_buffer_unmap(inbuf, &map);
>>     gst_buffer_unmap(outbuf, &outmap);
>>     return GST_FLOW_OK;
>> }
>>
>>
>>
>>
>>
>