Re: Issue with decoding H264 frames using appsrc in GStreamer

Ohh. Thank you but it seems I don't have access to gst_buffer_new_memdup
since I am using 1.18

The other option to not copy I guess is not suitable for me because
encoded_frame will be freed by another component (the encoded data is part
of a bigger object that is free when the object call its destructor)

So finally I opted for this

buffer = gst_buffer_new_allocate (NULL, frame_size, NULL);
    gst_buffer_map (buffer, &map, GST_MAP_WRITE);
    memcpy((guchar *)map.data, encoded_frame,  gst_buffer_get_size( buffer
) );
    gst_buffer_unmap(buffer, &map);


Now I don't see those *broken/invalid nal Type: 0*  messages but
currently I still don't see that the raw file is receiving bytes.
Do you think it is possible to use the decoder with this configuration?
Maybe I need to follow the approach of need-data to feed the decoder or How
Can I check if the decoder is actually processing everything?

On Fri, Jul 7, 2023 at 4:50 PM Michael Gruner <[email protected]>
wrote:

> Hi Israel
>
> This assignment is invalid:
>
> gst_buffer_map(buffer, &map, GST_MAP_WRITE);
> * map.data = encoded_frame;*
>   gst_buffer_unmap(buffer, &map);
>
> You need to copy the data using a plain memcpy or
> by creating the new buffer directly as a copy using gst_buffer_new_memdup.
>
> If you would like to avoid the copy, then you can use
> gst_buffer_new_wrapped or gst_buffer_new_wrapped_full but you need to be
> careful that: 1. The pointer remains valid the while the buffer is alive
> and 2. The pointer is freed when the buffer is unreffed.
>
> Michael
>
> On 7 Jul 2023, at 04:02, Israel Díaz via gstreamer-devel <
> [email protected]> wrote:
>
> 
> Dear GStreamer community,
>
> I am currently facing an issue while attempting to decode H264 frames
> using the appsrc element in GStreamer. I have constructed the following
> pipeline within my application:
>
> appsrc name=mysource ! video/x-h264, alignment=au,
> stream-format=byte-stream ! h264parse ! video/x-h264, alignment=au,
> stream-format=byte-stream ! omxh264dec ! video/x-raw, format=NV16_10LE32,
> width=1920, height=1080, framerate=30/1 ! filesink location=ball.raw
>
> To feed the appsrc with frames, I am reading an H264 file frame by frame
> since I store each frame in a structure along with its size. Then, I push
> the frames into the appsrc using the following function:
>
> void VDRE2GStreamerDecodingDisplaySinkTask::PushBufferToGStreamer(uint8_t*
> encoded_frame, uint32_t frame_size)
> {
>     GstFlowReturn ret;
>     GstMapInfo map;
>     GstBuffer *buffer = gst_buffer_new_and_alloc(frame_size);
>     gst_buffer_map(buffer, &map, GST_MAP_WRITE);
>     map.data = encoded_frame;
>     gst_buffer_unmap(buffer, &map);
>
>     /* Push the buffer into the appsrc */
>     g_signal_emit_by_name(m_appsrc, "push-buffer", buffer, &ret);
>
>     /* Free the buffer now that we are done with it */
>     gst_buffer_unref(buffer);
>
>     if (ret != GST_FLOW_OK) {
>         g_print("PushBufferToGStreamer: Error!\n");
>     }
> }
>
> However, when I execute the pipeline and set GST_DEBUG=4, I encounter the
> following error messages:
>
> Decode Thread: Frame to Process : 1 :s 246
> Decode Thread: Frame to Process : 2 :s 232
> Decode Thread: Frame to Process : 3 :s 854
> Decode Thread: Frame to Process : 4 :s 952
> Decode Thread: Frame to Process : 5 :s 350
> Decode Thread: Frame to Process : 6 :s 1038
> Decode Thread: Frame to Process : 7 :s 1447
> Decode Thread: Frame to Process : 8 :s 1316
> Decode Thread: Frame to Process : 9 :s 1297
> Decode Thread: Frame to Process : 10 :s 1205
> 0:00:02.370465675  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 29 will be dropped
> 0:00:02.370549896  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 1140 will be dropped
> Decode Thread: Frame to Process : 11 :s 1222
> Decode Thread: Frame to Process : 12 :s 1189
> Decode Thread: Frame to Process : 13 :s 1094
> Decode Thread: Frame to Process : 14 :s 930
> Decode Thread: Frame to Process : 15 :s 1000
> Decode Thread: Frame to Process : 16 :s 888
> Decode Thread: Frame to Process : 17 :s 901
> Decode Thread: Frame to Process : 18 :s 788
> Decode Thread: Frame to Process : 19 :s 833
> Decode Thread: Frame to Process : 20 :s 764
> Decode Thread: Frame to Process : 21 :s 727
> Decode Thread: Frame to Process : 22 :s 616
> Decode Thread: Frame to Process : 23 :s 728
> Decode Thread: Frame to Process : 24 :s 622
> Decode Thread: Frame to Process : 25 :s 609
> Decode Thread: Frame to Process : 26 :s 657
> Decode Thread: Frame to Process : 27 :s 651
> Decode Thread: Frame to Process : 28 :s 621
> Decode Thread: Frame to Process : 29 :s 534
> Decode Thread: Frame to Process : 30 :s 2989
> 0:00:03.033182378  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 29 will be dropped
> 0:00:03.033258709  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 117 will be dropped
> 0:00:03.033309439  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 29 will be dropped
> 0:00:03.033369800  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 1468 will be dropped
> Decode Thread: Frame to Process : 31 :s 411
> Decode Thread: Frame to Process : 32 :s 465
> Decode Thread: Frame to Process : 33 :s 514
> Decode Thread: Frame to Process : 34 :s 520
> Decode Thread: Frame to Process : 35 :s 519
> Decode Thread: Frame to Process : 36 :s 476
> Decode Thread: Frame to Process : 37 :s 448
> Decode Thread: Frame to Process : 38 :s 544
> Decode Thread: Frame to Process : 39 :s 523
> Decode Thread: Frame to Process : 40 :s 454
> Decode Thread: Frame to Process : 41 :s 561
> Decode Thread: Frame to Process : 42 :s 409
> Decode Thread: Frame to Process : 43 :s 506
> Decode Thread: Frame to Process : 44 :s 330
> Decode Thread: Frame to Process : 45 :s 2180
> 0:00:03.530095934  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 29 will be dropped
> 0:00:03.530192075  6548 0xffff8c01ff60 WARN               h264parse
> gsth264parse.c:1562:gst_h264_parse_handle_frame:<myparser> broken/invalid
> nal Type: 0 Unknown, Size: 117 will be dropped
> 0:00:03.530251666  654
>
>
> So then I analyzed the encoded_data to check if there are valid NAL units:
>
> have analyzed the encoded_frame data to determine if it contains valid NAL
> units. Here's a summary of your findings for the first five data sets:
>
> Encoded data values in hexadecimal: 0, size: 3241
> 00 00 00 01 27 7A 00 28 B6 D0 0C E8 07 80 22 7E 27 01 6A 02 04 04 80 00 00
> 03 00 80 00 00 1E 19 A8 00 3D 09 00 01 6E 37 FF FE 05 00 00 00 01 28 EE 3C
>
> Encoded data values in hexadecimal: 1, size: 246
> 00 00 00 01 21 E2 01 08 42 DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03
> 00 00 03 00 00 03 00 00 09 36 B5 84 05 B0 C9 8C DA C9 6F 9D F0 00
>
> Encoded data values in hexadecimal: 2, size: 232
> 00 00 00 01 21 E4 02 08 42 DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03
> 00 00 03 00 00 03 00 00 03 00 16 D0 00 00 01 21 00 1F E3 90 08 21
>
> Encoded data values in hexadecimal: 3, size: 854
> 00 00 00 01 21 E6 03 08 8B 7F 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03
> 00 00 03 00 00 03 00 00 03 00 06 2C 00 00 01 21 00 1F E3 98 0C 22
>
> Encoded data values in hexadecimal: 4, size: 952
> 00 00 00 01 21 E8 04 0E DF 00 00 03 00 00 03 00 00 03 00 00 03 00 00 03 00
> 00 03 00 00 03 00 00 03 00 00 97 80 00 00 01 21 00 1F E3 A0 10 3B
>
> It seems that the analyzed data contains valid NAL units, as expected. I
> have attempted a similar test using a file source and launching the
> pipeline with gst-launch as follows:
>
> gst-launch-1.0 -e filesrc location=ball2.h264 ! video/x-h264,
> alignment=au, width=1920, height=1080, framerate=30/1 ! h264parse !
> omxh264dec ! video/x-raw, format=NV16_10LE32, width=1920, height=1080,
> framerate=30/1 ! filesink location=ball_gst.raw
>
> During this test, I did not encounter any issues or error messages with
> GST_DEBUG=4. Additionally, the raw file was populated with bytes, unlike
> the appsrc version where the content always appears empty (presumably
> because it is not being decoded at any point).
>
> I even tried with a identity dump=true element between filesrc and
> h264parse and I also have the same bytes I am injecting to appsrc So I
> don't know what can be the difference
>
> Thank you in advance.
> Best regards.
> Israel.
>
>