Doubts on DCT

Rakesh Singh <[email protected]> Sun, 14 Jul 2013 11:03:31 +0530
Newsgroups gmane.comp.video.xvid.devel
Message-ID <CAJkJVMDUcaYcoKtBq7gGQbW5-DOeqhk2Do1QMXiNFaRDS4zs5w@mail.gmail.com>
Hi All

I have gone through the original Loeffler's paper. There he said his DCT
output should be scaled by 1/2*sqrt(2).

What I am unable to understand at this point of time is that where is this
scaling done in XVID code.

OK what I have seen from the code is that

1. from the first pass pass
   pIn[0] = SHIFTL(mm0, FPASS);
    pIn[4] = SHIFTL(mm1, FPASS);

in the second pass
    pIn[0*8] = (int16_t)SHIFTR(mm0, FPASS+3);
    pIn[4*8] = (int16_t)SHIFTR(mm1, FPASS+3);

Is it really necessary to perform the 1st operation? What if I do

    pIn[0*8] = (int16_t)SHIFTR(mm0, 3);
    pIn[4*8] = (int16_t)SHIFTR(mm1, 3);

in the second pass where FPASS is zero in the first pass.

How does  multiplication by sqrt(2) is done ? Please anyone would explain
it.

With Regards and thanks in anticipation.