Re: Re[2]: III_requantize() accuracy

Rob Leslie <[email protected]>
Newsgroups gmane.comp.audio.mad.devel
Message-ID <[email protected]>
On Mar 24, 2004, at 5:36 PM, Grigory A. wrote:
>> Keeping in mind that 2^(-36.75) = 2^(-36) × 2^(-3/4), you will find 
>> that III_requantize(210, -147) calculates:
>>
>>               210^(4/3)    ⇒ 0x04e0393e × 2^12
>>    0x04e0393e × 2^(-36+12) ⇒ 0x00000005
>>                 2^(-3/4)   ⇒ 0x09837f05
>>    0x00000005 × 0x09837f05 ⇒ 0x00000002
>
> Yes I am agree. Only one difference in my calculation scheme I firs 
> multiply on root - in this case 2^(-3/4) and next rounding and 
> shifting. This small changes give more precise result.

You make a good point. The accuracy can indeed be improved by 
multiplying the root first, before losing significant digits from the 
shift.

This probably merits a change for the next release.

Thanks,

-- 
Rob Leslie
[email protected]
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.