Re: y4mdenoise version 4 beta 1 is checked in!

Hallo

Steven Boswell II wrote:
> --- On Sat, 9/4/10, Bernhard Praschinger<[email protected]>
> wrote:
>> I did test y4mdenoise with a Full Resolution PAL video. For some
>> reason y4mdenoise crashes every time after denoising up to 63
>> frames. The process uses than about 1GB of ram (Is that normal ? I
>> have 4GB of Ram)
>
> Yikes!  No, that's definitely not normal.  About the only reason
> y4mdenoise would fail is if it ran out of memory, and it's supposed
> to purge most of its structures regularly.
When I look at xosview I see that the memory usage gets up and than down 
a little. But always up.

> I've got a DVD-sized y4mdenoise process running right now, and it
> seems to be maxing out at 300 MB of memory.
That sound reasonable.

> What are the exact parameters you're passing to y4mdenoise?  In
> particular, what are you passing for "-f"?  That's the only parameter
> that could be making it use more memory.
For testing reasons I left everything to default. So the command looked 
like that: ... | y4mdenoise | ....

For the testing now I have added -v 2 and -f 10.

The strange thing is that it does not change anything.

Do you check for the Chroma ? The video I have uses 420paldv ? which is 
decoded by libdv (YUY2 (4:2:2) )

> As a stupid test, you could look for the word "Purge" (case
> sensitive) in newdenoise.cc and comment out the "if (frame %
> denoiser.frames == 0)" above it, to make sure it purges every frame.
>
> Otherwise, I'm not sure what else to check...except maybe you could
> make a build with assertions (i.e. remove "-DNDEBUG" from
> y4mdenoise/Makefile.*) and see if it finds something odd.
I have removed the -DNDEBUG,
and the CFLAGS are changed to: g -pthread -Wall -Wunused
and CXXFLAGS to: -g -pthread

Didn't make a difference.

I did add to every occurance of frame % denoiser.frames a line like 
that: mjpeg_warn("Purgen Line 660");

Where the 660 is the line number where the purge is called. And I see 
that it is only called at two specific lines.
I have even removed the if statement, and the Purge is not called in 
line 660 and 995 (might be old) and line 767, 835
It is called in line 1174 and 1274 (maybe I missed the exact line).

For testing I have also removed the if so it purges every frame. But 
that didn't make a difference for the memory usage.

Compiled it, the output lookes now like that:
++ WARN: [y4mdenoise] Purgen Line 1274
Frame 4: 0.0%+99.9%+0.0% not-moved, 0.0%+0.0% moved, 0.0%+0.1% new
2 moved-regions, 1 pixel-sorters
Frame 4: 0.0%+99.9%+0.0% not-moved, 0.0%+0.0% moved, 0.0%+0.1% new
2 moved-regions, 1 pixel-sorters
++ WARN: [y4mdenoise] Purgen Line 1174
Frame 5: 0.0%+0.4%+0.6% not-moved, 48.4%+10.3% moved, 25.8%+14.4% new
2 moved-regions, 1763 pixel-sorters
Frame 5: 0.0%+24.5%+1.9% not-moved, 25.0%+7.2% moved, 26.5%+14.9% new
2 moved-regions, 2619 pixel-sorters
++ WARN: [y4mdenoise] Purgen Line 1274
Frame 5: 0.0%+99.9%+0.0% not-moved, 0.0%+0.0% moved, 0.0%+0.1% new
2 moved-regions, 1 pixel-sorters
Frame 5: 0.0%+99.9%+0.0% not-moved, 0.0%+0.0% moved, 0.0%+0.1% new
2 moved-regions, 1 pixel-sorters
++ WARN: [y4mdenoise] Purgen Line 1174
Frame 6: 0.0%+0.0%+0.3% not-moved, 50.2%+9.4% moved, 24.6%+15.5% new
2 moved-regions, 1639 pixel-sorters
Frame 6: 0.0%+54.0%+1.1% not-moved, 2.7%+5.3% moved, 21.3%+15.6% new
2 moved-regions, 2326 pixel-sorters
++ WARN: [y4mdenoise] Purgen Line 1274
Frame 6: 0.0%+99.9%+0.0% not-moved, 0.0%+0.0% moved, 0.0%+0.1% new
2 moved-regions, 1 pixel-sorters

I did run it once with valgrind. The command looked like that:
lav2yuv -f 3 jmj.eli -C 420jpeg | valgrind y4mdenoise -v 2 -f 6 | \ yuvplay

The last output looked like that:
HEAP SUMMARY:
   in use at exit: 713,769,360 bytes in 2,486 blocks
total heap usage: 10,974 allocs, 8,488 frees, 2,992,325,008 bytes allocated

LEAK SUMMARY:
definitely lost: 0 bytes in 0 blocks
indirectly lost: 0 bytes in 0 blocks
possibly lost: 645,070,752 bytes in 2,344 blocks
still reachable: 68,698,608 bytes in 142 blocks

But I am a bit lost with the output of valgrind when I add the -v option.
The Summary tells me that (I hope it tells you something I didn't see, 
and sorry for the line wrap):
==15461== Use of uninitialised value of size 8
==15461==    at 0x40F5D9: SearchBorder<short, int>::MoveRight(Status_t&) 
(SearchBorder.hh:1133)
==15461==    by 0x426D55: MotionSearcher<unsigned char, 1, int, short, 
int, (short)4, (short)2, unsigned short, Pixel<unsigned char, 1, int>, 
ReferencePixel<unsigned short, unsigned char, 1, Pixel<unsigned char, 1, 
int> >, ReferenceFrame<ReferencePixel<unsigned short, unsigned char, 1, 
Pixel<unsigned char, 1, int> >, short, int> >::AddFrame(Status_t&, 
Pixel<unsigned char, 1, int> const*) (MotionSearcher.hh:1720)
==15461==    by 0x404266: newdenoise_interlaced_frame_intensity(unsigned 
char const*, unsigned char*) (newdenoise.cc:1222)
==15461==    by 0x4072F7: newdenoise_interlaced_frame (newdenoise.cc:1392)
==15461==    by 0x40235F: main (main.c:215)

And the 2nd error:
==15461== Conditional jump or move depends on uninitialised value(s)
==15461==    at 0x40F487: SearchBorder<short, 
int>::RemoveMotionVectorMatch(SearchBorder<short, int>::MovedRegion*) 
(SearchBorder.hh:1093)
==15461==    by 0x40F6FD: SearchBorder<short, int>::MoveRight(Status_t&) 
(SearchBorder.hh:1168)
==15461==    by 0x426D55: MotionSearcher<unsigned char, 1, int, short, 
int, (short)4, (short)2, unsigned short, Pixel<unsigned char, 1, int>, 
ReferencePixel<unsigned short, unsigned char, 1, Pixel<unsigned char, 1, 
int> >, ReferenceFrame<ReferencePixel<unsigned short, unsigned char, 1, 
Pixel<unsigned char, 1, int> >, short, int> >::AddFrame(Status_t&, 
Pixel<unsigned char, 1, int> const*) (MotionSearcher.hh:1720)
==15461==    by 0x404266: newdenoise_interlaced_frame_intensity(unsigned 
char const*, unsigned char*) (newdenoise.cc:1222)
==15461==    by 0x4072F7: newdenoise_interlaced_frame (newdenoise.cc:1392)
==15461==    by 0x40235F: main (main.c:215)

auf hoffentlich bald,

Berni the Chaos of Woodquarter

Email: [email protected]
www: http://www.lysator.liu.se/~gz/bernhard

------------------------------------------------------------------------------
This SF.net Dev2Dev email is sponsored by:

Show off your parallel programming skills.
Enter the Intel(R) Threading Challenge 2010.
http://p.sf.net/sfu/intel-thread-sfd