Re: EBML IRL survey

2015-10-22 15:58 GMT+02:00 Dave Rice <[email protected]>:
> Hi all,
>
> For those who are very curious, I've started to do some analysis of Matroska
> files (running through about 100,000 of user-submitted ones in archive.org).
> Currently I'm only about 1% of the way through this process. From the
> gathered data I'm mining out certain characteristics to see what is being
> done or has been done with ebml/mkv muxers. Eventually this will lead to
> testing of the in-development ebml/matroska implementation checker,
> MediaConch. Here's a few histograms of the initial findings, please let me
> know if you suggest any particular tests to add to this process.
>
> Total EBML Header Size in bytes (histogram of # of occurrences and the
> value)
>  905 24
>  878 40
>  133 47
>
> Inclusion of EBMLVersion and EBMLReadVersion is about 50/50. About half of
> the sample set don't specifically use the Element and the other half set it
> to 1.
>
> Same for EBMLMaxIDSize and EBMLMaxDataSize. Not used about half the time,
> the other times it is set to 4 and 8. I haven't seen any non-default values
> for these.
>
> Histogram of matroska versions:
>  443 1
>  827 2
>  646 4
>
> What happened to matroska version 3? Is 4 considered 'released'?

It's not frozen, but since we always keep backward & forward
compatibility, if you want to use a feature of 4 you can already use
it. The current mkvmerge uses 4.

> So far, there is only one EBML and one Segment as the level 0 elements of
> the files. No VOID at level 0 (yet), never (yet) more than one level 0
> element after the EBML Header.
>
> Matroska timescale histogram
>    3    "not written"
> 1913 1000000

This is due to some buggy hardware players that would not work
properly without that value (IIRC).

> Histogram of muxingApp
>    3
>    1 Haali DirectShow Matroska Muxer 1.10.262.12
>    1 IDMmkvlib0.1
>    8 Lavf51.12.1
>    1 Lavf52.31.0
>    1 Lavf52.84.0
>    1 Lavf53.24.0
>    1 Lavf53.24.2
>    3 Lavf53.31.100
>    9 Lavf53.32.100
>    1 Lavf54.29.104
>    9 Lavf54.63.104
>   23 Lavf55.12.0
>   13 Lavf55.19.104
>    6 Lavf55.40.100
>    1 Lavf55.43.100
>    1 Lavf55.48.100
>    2 Lavf55.51.101
>    2 Lavf55.7.100
>    5 Lavf56.1.0
>    3 Lavf56.4.101
>   36 SUPER(C)
>    4 WonderShare Matroska Muxer
>    6 libDivXMediaFormat 3.4.1.0004
>    3 libebml v0.6.3 + libmatroska v0.6.3
>  132 libebml v0.7.7 + libmatroska v0.8.0
>  695 libebml v0.7.7 + libmatroska v0.8.1
>    3 libebml v0.7.8 + libmatroska v0.8.1
>   62 libebml v0.7.9 + libmatroska v0.8.1
>    6 libebml v0.8.0 + libmatroska v0.9.0
>   18 libebml v1.0.0 + libmatroska v1.0.0
>    5 libebml v1.2.0 + libmatroska v1.1.0
>    1 libebml v1.2.1 + libmatroska v1.1.1
>   17 libebml v1.2.2 + libmatroska v1.3.0
>  171 libebml v1.2.3 + libmatroska v1.3.0
>   30 libebml v1.3.0 + libmatroska v1.4.0
>  562 libebml v1.3.0 + libmatroska v1.4.1
>   37 libebml v1.3.1 + libmatroska v1.4.2
>   33 libmkv 0.6.5

Is libmkv the Google one ? Otherwise you may be missing some YouTube
samples for more accuracy.

> Histogram of writingApp
>    4
>   14 AVSMatroskaFile
>    6 DivXMKVMux 3.4.1.0004
>    3 HandBrake 0.10.0 2014112200
>    8 HandBrake 0.10.1 2015030800
>    1 HandBrake 0.9.6
>    8 HandBrake 0.9.8
>   24 HandBrake 0.9.9
>   12 HandBrake 6548svn 2014112401
>    1 HandBrake 6967svn 2015030601
>    3 HandBrake svn3404
>    3 Helix Producer MKV Output Plugin - MatroskaMuxer 1.0
>    1 IDMmkvlib0.1
>    8 Lavf51.12.1
>    1 Lavf52.31.0
>    1 Lavf52.84.0
>    1 Lavf53.24.0
>    1 Lavf53.24.2
>    3 Lavf53.31.100
>    9 Lavf53.32.100
>    1 Lavf54.29.104
>    9 Lavf54.63.104
>   13 Lavf55.19.104
>    6 Lavf55.40.100
>    1 Lavf55.43.100
>    1 Lavf55.48.100
>    2 Lavf55.51.101
>    2 Lavf55.7.100
>    4 Lavf56.1.0
>    3 Lavf56.4.101
>   36 SUPER(C)
>    1 gdsmux
>  104 mkvmerge v1.7.0 ('What Do You Take Me For') built on Apr 28 2006
> 17:20:19
>   28 mkvmerge v2.0.0 ('After The Rain Has Fallen') built on Feb  7 2007
> 19:06:51
>   62 mkvmerge v2.1.0 ('Another Place To Fall') built on Oct  9 2007 04:36:00
>   79 mkvmerge v2.2.0 ('Turn It On Again') built on Mar  4 2008 12:58:26
>    5 mkvmerge v2.2.0 ('Turn It On Again') built on Mar  4 2008 13:20:25
>   58 mkvmerge v2.3.0 ('Freak U') built on Sep  8 2008 18:32:16
>   10 mkvmerge v2.4.0 ('Fumbling Towards Ecstasy') built on Oct 11 2008
> 20:13:15
>    4 mkvmerge v2.4.1 ('Use Me') built on Dec  5 2008 15:43:32
>    2 mkvmerge v2.5.1 ('He Wasn't There') built on Feb 22 2009 18:19:47
>    7 mkvmerge v2.5.2 ('Stranger in your Soul') built on Feb 28 2009 18:58:17
>    4 mkvmerge v2.5.3 ('Boogie') built on Mar  7 2009 15:00:41
>   50 mkvmerge v2.6.0 ('Kelly watch the Stars') built on Mar 24 2009 15:23:17
>  101 mkvmerge v2.7.0 ('Do It Again') built on Apr 15 2009 00:27:10
>    7 mkvmerge v2.8.0 ('The Tree') built on May 14 2009 09:28:39
>  139 mkvmerge v2.9.0 ('Moanin'') built on May 22 2009 17:46:31
>    2 mkvmerge v2.9.5 ('Tu es le seul') built on Jun  7 2009 11:07:07
>    2 mkvmerge v2.9.5 ('Tu es le seul') built on Jun  8 2009 14:34:11
>    7 mkvmerge v2.9.7 ('Tenderness') built on Jul  1 2009 18:43:35
>    1 mkvmerge v2.9.7 ('Tenderness') built on Jul 13 2009 11:47:01
>  144 mkvmerge v2.9.8 ('C'est le bon') built on Aug 13 2009 12:49:06
>    8 mkvmerge v2.9.9 ('Tutu') built on Nov 25 2009 21:13:32
>    3 mkvmerge v3.1.0 ('Happy up here') built on Jan 19 2010 12:09:24
>   51 mkvmerge v3.2.0 ('Beginnings') built on Feb 12 2010 16:46:17
>    6 mkvmerge v3.3.0 ('Language') built on Mar 24 2010 14:59:24
>   15 mkvmerge v4.0.0 ('The Stars were mine') built on Jun  6 2010 16:18:42
>    2 mkvmerge v4.3.0 ('Escape from the Island') built on Sep  5 2010
> 10:30:51
>    1 mkvmerge v4.3.0 ('Escape from the Island') built on Sep 28 2010
> 23:42:10
>    1 mkvmerge v4.6.0 ('Still Crazy After All These Years') built on Mar 10
> 2011 02:50:32
>    3 mkvmerge v4.7.0 ('Just Like You Imagined') built on Apr 21 2011
> 01:13:14
>    1 mkvmerge v4.8.0 ('I Got The...') built on May 24 2011 03:12:58
>    1 mkvmerge v4.9.1 ('Ich will') built on Jul 11 2011 23:53:15
>   16 mkvmerge v5.0.1 ('Es ist Sommer') built on Oct  9 2011 11:55:43
>    1 mkvmerge v5.1.0 ('And so it goes') built on Feb  1 2012 11:32:58
>    8 mkvmerge v5.2.0 ('I can't explain') built on Dec 18 2011 18:12:03
>    2 mkvmerge v5.3.0 ('I could have danced') built on Feb  9 2012 10:38:07
>    2 mkvmerge v5.5.0 ('Healer') built on Apr  6 2012 21:43:24
>   10 mkvmerge v5.6.0 ('Kenya Kane') built on May 27 2012 16:44:04
>  130 mkvmerge v5.7.0 ('The Whirlwind') built on Jul  8 2012 20:08:51
>    1 mkvmerge v5.7.0 ('The Whirlwind') built on Jul 29 2012 15:12:35
>   18 mkvmerge v5.8.0 ('No Sleep / Pillow') built on Sep  2 2012 15:37:04
>    2 mkvmerge v5.9.0 ('On The Loose') built on Dec  9 2012 15:37:01
>   13 mkvmerge v6.0.0 ('Coming Up For Air') built on Jan 20 2013 09:52:00
>    1 mkvmerge v6.0.0 ('Coming Up For Air') built on Jan 20 2013 12:38:35
>    2 mkvmerge v6.1.0 ('Old Devil') built on Mar  2 2013 14:32:37
>    3 mkvmerge v6.2.0 ('Promised Land') built on Apr 28 2013 12:22:01
>    2 mkvmerge v6.3.0 ('You can't stop me!') built on Jul 10 2013 08:40:50
>    7 mkvmerge v6.3.0 ('You can't stop me!') built on Jun 28 2013 20:09:41
>    4 mkvmerge v6.4.1 ('Omega Point') built on Sep 16 2013 22:05:10
>  358 mkvmerge v6.5.0 ('Isn't she lovely') built on Oct 20 2013 12:50:05
>   12 mkvmerge v6.6.0 ('The Edge Of The In Between') built on Dec  1 2013
> 17:55:00
>    1 mkvmerge v6.6.0 ('The Edge Of The In Between') built on Dec  1 2013
> 18:33:13
>    4 mkvmerge v6.7.0 ('Back to the Ground') 32bit built on Jan  8 2014
> 15:03:17
>    2 mkvmerge v6.7.0 ('Back to the Ground') 64bit built on Jan  8 2014
> 15:10:52
>   58 mkvmerge v6.8.0 ('Theme for Great Cities') 32bit built on Mar  2 2014
> 21:27:07
>    2 mkvmerge v6.9.0 ('On Duende') 64bit built on Apr 18 2014 12:57:54
>    6 mkvmerge v6.9.1 ('Blue Panther') 64bit built on Apr 18 2014 18:23:38
>    3 mkvmerge v7.0.0 ('Where We Going') 32bit built on Jun  9 2014 15:08:34
>    1 mkvmerge v7.1.0 ('Good Love') 64bit built on Jul 27 2014 13:06:55
>   31 mkvmerge v7.2.0 ('On Every Street') 32bit built on Sep 13 2014 15:42:11
>    4 mkvmerge v7.2.0 ('On Every Street') 64bit built on Sep 13 2014 15:52:10
>    7 mkvmerge v7.3.0 ('Nouages') 32bit built on Oct 22 2014 18:44:01
>   34 mkvmerge v7.3.0 ('Nouages') 64bit built on Oct 22 2014 18:53:34
>   35 mkvmerge v7.4.0 ('Circles') 64bit built on Dec 12 2014 12:19:56
>    1 mkvmerge v7.5.0 ('Glass Culture') 64bit built on Jan  4 2015 16:48:44
>    1 mkvmerge v7.7.0 ('Six Voices') 32bit built on Feb 28 2015 23:23:00
>   10 mkvmerge v7.7.0 ('Six Voices') 64bit built on Feb 28 2015 23:39:33
>    1 mkvmerge v7.8.0 ('River Man') 64bit built on Mar 27 2015 16:31:37
>   10 mkvmerge v7.9.0 ('Birds') 32bit
>   12 mkvmerge v8.1.0 ('Psychedelic Postcard') 32bit
>    2 mkvmerge v8.1.0 ('Psychedelic Postcard') 64bit

I think it doesn't have the one used on YouTube (gdsmux ?). I'll dig
through my old files to see if I can find older things. At least the
files with RV9 in them which were popular before H264 because
dominant.

> Best Regards,
> Dave Rice
>
> _______________________________________________
> Matroska-devel mailing list
> [email protected]
> http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel
> Read Matroska-Devel on GMane:
> http://dir.gmane.org/gmane.comp.multimedia.matroska.devel



-- 
Steve Lhomme
Matroska association Chairman
_______________________________________________
Matroska-devel mailing list
[email protected]
http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel
Read Matroska-Devel on GMane: http://dir.gmane.org/gmane.comp.multimedia.matroska.devel