[PATCH 13/18] avcodec/h264dec: pick up the view list wherever the subset SPS appears

In mp4 and Matroska the subset SPS is carried in band rather than in extradata,
so the views a stream contains are not always known by the time get_format() is
first called -- which is where a caller is expected to select them, and where the
ffmpeg CLI view specifiers do so. Go through get_format() again when the view list
first becomes available, so that the selection can still be made.

This deliberately does not reinitialise the decoding context: that would flush the
DPB and change base view output. Only the exported view list changes, so the
format returned must be the one already in use.

Encoders also place the subset SPS between the base view slices and the dependent
view slices of the access unit it belongs to; the ones used for mp4 and Matroska
do. Taken in NAL order the views would still be unknown when the base view slices
of the first access unit activate the parameter sets, and that access unit's
dependent slices would be dropped, costing the first dependent view picture of
the stream and leaving the pictures that referenced it without a reference until
the following anchor. Parse the subset SPSs of a packet before anything else in
it: they have their own id space, ps.subset_sps_list, and decode_seq_parameter_set()
with subset set writes nowhere else, so the base view cannot be affected.

Signed-off-by: Dom Cobley <[email protected]>
---
 libavcodec/h264_slice.c | 16 ++++++++++++++++
 libavcodec/h264dec.c    | 14 ++++++++++++++
 2 files changed, 30 insertions(+)

diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
index 5b2749e3b5..8650e1bd01 100644
--- a/libavcodec/h264_slice.c
+++ b/libavcodec/h264_slice.c
@@ -1351,6 +1351,22 @@ static int h264_init_ps(H264Context *h, const H264SliceContext *sl, int first_sl
             ret = export_multiview(h, mvc_sps);
             if (ret < 0)
                 return ret;
+
+            /* Give the caller a chance to select views now that they are known,
+             * by going through get_format() again. Deliberately not a context
+             * reinit: that would flush the DPB and change base view output. The
+             * format itself cannot change here, only the exported view list. */
+            if (h->context_initialized && mvc_sps) {
+                ret = get_pixel_format(h, 1);
+                if (ret < 0)
+                    return ret;
+                if (ret != h->avctx->pix_fmt) {
+                    av_log(h->avctx, AV_LOG_ERROR,
+                           "Pixel format changed while exporting the view list\n");
+                    return AVERROR(ENOSYS);
+                }
+            }
+
             ret = setup_multiview(h, mvc_sps);
             if (ret < 0)
                 return ret;
diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c
index fc296447ed..392bab863e 100644
--- a/libavcodec/h264dec.c
+++ b/libavcodec/h264dec.c
@@ -666,6 +666,20 @@ static int decode_nal_units(H264Context *h, AVBufferRef *buf_ref,
         return ret;
     }
 
+    /* An encoder is free to put the subset SPS between the base view slices and
+     * the dependent view slices of the access unit it belongs to, and the ones in
+     * mp4 and Matroska do exactly that. Picking it up in NAL order would leave
+     * the views unknown while the first access unit is decoded, and its dependent
+     * slices would be dropped below. Parse them all up front instead: subset SPSs
+     * have their own id space (ps.subset_sps_list) and cannot affect the base
+     * view, so hoisting them changes nothing else. */
+    for (i = 0; i < h->pkt.nb_nals; i++)
+        if (h->pkt.nals[i].type == H264_NAL_SUB_SPS) {
+            /* on a copy: the NAL is parsed again in NAL order below */
+            GetBitContext tmp_gb = h->pkt.nals[i].gb;
+            ff_h264_decode_subset_seq_parameter_set(&tmp_gb, avctx, &h->ps);
+        }
+
     if (avctx->active_thread_type & FF_THREAD_FRAME)
         nals_needed = get_last_needed_nal(h);
     if (nals_needed < 0)
-- 
2.53.0

_______________________________________________
ffmpeg-devel mailing list -- [email protected]
To unsubscribe send an email to [email protected]