bug in thread count calculation

Lukasz Marek <[email protected]> Tue, 25 Nov 2014 15:31:37 +0100
Newsgroups gmane.comp.video.xvid.devel
Message-ID <CAOvrpSQJQ7OAriAy+6hqoPvk2j6fz+wbMi_ox+NCM+YKqTQ2Qw@mail.gmail.com>
Hello,

During some tests of ffmpeg encoders (it uses libxvidcore) I noticed there
is not allocated memory in use.
Bad line is encoder.c:466

bug is in following code

#ifndef HAVE_PTHREAD
                int t = MAX(1, create->num_threads);
#else
                int t = MIN(create->num_threads, (int)
(pEnc->mbParam.mb_height>>1)); /* at least two rows per thread */
#endif

For height <=16 pEnc->mbParam.mb_height is 1. (it is calculated earlier as
(height +15) /16) so it produces t = 0.
It is later used in

pEnc->smpData = xvid_malloc(t*sizeof(SMPData), CACHE_LINE);

Im not sure what is correct fix, but one of follows may help

#ifndef HAVE_PTHREAD
                int t = MAX(1, create->num_threads);
#else
                int t = MIN(create->num_threads, (int)
(pEnc->mbParam.mb_height>>1)); /* at least two rows per thread */
                t = MAX(1, t);
#endif

or

#ifndef HAVE_PTHREAD
                int t = MAX(1, create->num_threads);
#else
                int t = MIN(create->num_threads, (int)
(pEnc->mbParam.height>>1)); /* at least two rows per thread */
#endif

or

#ifndef HAVE_PTHREAD
                int t = MAX(1, create->num_threads);
#else
                int t = MIN(create->num_threads, (int)
(pEnc->mbParam.mb_height>>1)); /* at least two rows per thread */
#endif
                if (!t)
                    goto xvid_err_nosmp;

Regards,
Lukasz Marek