[Helix-client-dev] CR: Bug 11047 H263+ clips cause SIGABORT due to memory corruption

Date: 2010-09-24
Project: RealPlayer for Android Smartphones

Synopsis: H263+ clips cause SIGABORT due to memory corruption

Overview: In my test, some H263+ clips cause SIGABORT during playback. The reason is that in slice structure mode, the micro block index iMBA read from the slice header exceed the valid range and corrupt the memory.  The clips for testing are created by the SUPER video converter, which uses FFmpeg.  The comparison among Helix h263 decoder, FFmpeg h263 decoder and the ITU H263 document show that FFmpeg does not strictly follow the picture header structure format of the ITU H263 for extended type.  In FFmpeg, the SQUANT field is right after the SSS field (ituh263dec.cpp: line 1072). It also ignores the MBA value.  In ITU 263,  the fields after SSS are "ELNUM RLNUM" (page 162); the SQUANT is after fields "MBA SEPB2" (page 96). In the CR, the decoder return PIA_S_UNSUPPORTED instead of continuing when iMBA exceeds valid range. 

Reference: 
  ITU H263 doc: http://www.itu.int/rec/dologin_pub.asp?lang=e&id=T-REC-H.263-200501-I!!PDF-E&type=items
  FFmpeg ituh263dec.cpp: http://git.ffmpeg.org/?p=ffmpeg;a=blob;f=libavcodec/ituh263dec.c;h=8b5d9391b2e196b6767848660ecbc93497d337b8;hb=HEAD

Changed files:
datatype-restricted/rm/video/codec/g2mp4combo/dec/d3mvdec.cpp

Image Size and Heap Use impact (Client -Only):
None

Platforms and Profiles Affected:
Platform: : head, 361, and 362
Profile: helix-client-android

Distribution Libraries Affected:
NA

Distribution library impact and planned action:
NA

Platforms and Profiles Build Verified:
Platform: hxclient_3_6_1_atlas
Profile:  helix-client-android

Platforms and Profiles Functionality verified:
Platform: hxclient_3_6_1_atlas
Profile: helix-client-android


Copyright assignment: I am a RealNetworks employee or contractor

_______________________________________________
Helix-client-dev mailing list
[email protected]
http://lists.helixcommunity.org/mailman/listinfo/helix-client-dev

? Makefile
? a
? adecencg2mp4.mak
? adecencg2mp4.upp
? adecg2mp4.mak
? adecg2mp4.upp
? android-dbg
? cdecg2mp4.mak
? cdecg2mp4.upp
? dbg
? dmp4lib.mak
? dmp4lib.upp
? h263.diff
? mp4vlib.mak
? mp4vlib.upp
? rel
? ribosome_logs
? umakefil.upp
Index: dec/d3mvdec.cpp
===================================================================
RCS file: /cvsroot/rarvcode-mpeg4combo/codec/g2mp4combo/dec/d3mvdec.cpp,v
retrieving revision 1.3.18.2
diff -u -w -r1.3.18.2 d3mvdec.cpp
--- dec/d3mvdec.cpp	5 Aug 2010 16:42:54 -0000	1.3.18.2
+++ dec/d3mvdec.cpp	24 Sep 2010 23:39:25 -0000
@@ -3339,6 +3339,10 @@
 					else
 					{
 						m_iPQuant = m_iGQuant = iSQUANT;
+						if(iSliceMBA >= m_iTotalNumberOfMBs)
+						{
+						    return PIA_S_UNSUPPORTED; 
+						}
 						iMBA = iSliceMBA;
 					}
 
@@ -3388,6 +3392,13 @@
 					}
 
 					bFirstSlice	= FALSE;
+					/* Some H263+ encoder (ffmpeg eg.) does not exactly follow H263+ in sliced structure mode, the index iSliceMBA
+					 * can exceed the maximum number of MBs, and corrupt the memory.
+					 */
+					if(iSliceMBA >= m_iTotalNumberOfMBs)
+					{
+					    return PIA_S_UNSUPPORTED; 
+					}					
 					iMBA = iSliceMBA;
 
 					// Set iLeft to one if this is the first MB for the slice.