Re: [Cellar] Depth offsets for subtitles in case of 3D MVC tracks in MKV files

2016-02-28 23:12 GMT+01:00 madshi <[email protected]>:
> 2016-02-14 15:50 GMT+01:00 Steve Lhomme <[email protected]>:
>> 2016-01-26 14:39 GMT+01:00 Nithin Mathew Kurien <[email protected]>:
>>> Hi,
>>>
>>> It's like this. Suppose the MVC video track has 8 frames. Also suppose it
>>> has 5 Offset Metadata Sequences embedded in it at the time of encoding.
>>> (Sequence entries are in pixels, positive if in front of the screen, and
>>> negative if behind.)
>>>
>>> Frame  1  2  3  4  5  6  7  8
>>> Seq1    5  1 -8 -2 -9  1 -3 -1
>>> Seq2    8  1  0  8 -4 -5  4 -1
>>> Seq3   -2  8 -3 -5  7  5 -5 -4
>>> Seq4    9  0  0 -5 -2  1 -4 -1
>>> Seq5   -9  3 -2 -4 -6  0 -3  6
>>>
>>> If the M2TS container has 2 subtitle tracks P1, P2 and 1 menu track I1, then
>>> a mapping can be defined in the M2TS container like this:
>>> P1 : Seq1
>>> P2 : Seq3
>>> I1  : Seq4
>>
>> OK, so it seems it's both in the header and in the track (the GOP is
>> part of the stream, I think).
>
> Yes. As Nithin Mathew explained (thanks!), there are SEI 3D depth info
> blocks inside of the video stream. We don't need to do anything about them.
> But there's additional data in the Blu-Ray playlist file(s) which
> assigns specific
> subtitle tracks to an "array index" in the SEI 3D depth info block.
>
> In the meanwhile nevcairiel and I have already added full support for all
> this 3D depth offset stuff to LAV Splitter/Video decoder and madVR, when
> playing back original Blu-Rays. But for Matroska there's no full solution
> possible yet, because the information in the playlist cannot be properly
> stored yet.
>
> It would be great if we could get an official solution for this soon, because
> we'd like to implement it "now", and we have the MakeMKV devs on board
> for this now, too.
>
> Basically I think what we need is one additional item for each subtitle
> track which simply assigns one SEI 3D depth array index to the subtitle
> track. E.g. something like "stereo_subtitle_offset_id", or whatever. The
> name is not important to me. Possible values I think are between 0..31
> for 3D Blu-Rays.

That seems to be heavily relying on the Blu-ray specs. I'd favor a
solution that is more generic and doesn't have to go through "codec
specific" lookup tables.

Otherwise if the codec used for Blu-Rays subtitles is already
specific, we could just use the CodecState to store the SEI value when
it changes.
https://matroska.org/technical/specs/index.html#CodecState

That means custom made text subtitles will not work properly with 3D
Blu-ray content because it doesn't use the same codec.

If we want the general case to work we may introduce an element at the
same level as CodecState that defines the Z-order for the Block. That
means it would have to translate the lookup index in the SEI to the
actual z-order value (if I understand correctly how it works). We need
to define if it's like the CodecState, ie it only changes from scene
to scene, not Block by Block. So it would be a State that needs to be
remembered when needed and only written when it changes.

What is the range of depth allowed in Blu-Rays ? Are they discrete
value or floating values ?

> One additional piece of information would be which array indexes of the
> SEI 3D depth info block are assigned to IG (interactive graphics). This
> could be used by media players or video renderers to decide at which
> 3D depth to draw the user interface (GUI/OSD). FWIW, it could be none,
> one or multiple array indexes. So the information field in the MKV header
> would have to be some sort of list/array. Logically this info would
> probably belong to the video track. This is less important than the
> subtitle assignment, though. But would still be nice to have.
>
> Best regards, madshi.



-- 
Steve Lhomme
Matroska association Chairman
_______________________________________________
Matroska-devel mailing list
[email protected]
http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel
Read Matroska-Devel on GMane: http://dir.gmane.org/gmane.comp.multimedia.matroska.devel