Re: y4mdenoise artifact...any ideas?

	Would have chimed in earlier but was on vacation for a couple weeks...

On Thu, 9 Jul 2009, Steven Boswell II wrote:

> ...one nasty artifact that I'm having trouble fixing.  I'm hoping someone 
> might have an idea what to do about it.

	I'll give it a try and see what folks think.

> The problem happens in fuzzy, dark areas -- they'll start jumping around like
> mad.  I see it in the backgrounds of cel animation, I see it in the dim
> corners of the stage in rock concerts, and I see it in scenes obscured by
>  fog.  I've even seen it in the face of a craggy old sailor.

	What has been called "grey splotches" and other similar names.
	not too polite ;-)) in the past.  Technically it's the "code 100 
	problem".

	In the case of y4mdenoise you're seeing, in my opinion, a combination
	of the "code 100" problem mixed with ghosting (misplaced pixels from
	other frames).

	Basically the problem is that differences of 1% or less in luma values
	are not noticeable to the eye.  Exceed that threshold and the
	differences become visible. 

	If you have a pixel (or group but I'll use pixel for simplicity) with
	a Y' value of 200 and next to it you have a pixel of 202 then you're
	not going to notice the difference since it's only 1%

	However, if you've a pixel of 50 and next to it is a 52 that's 4%
	difference and will be noticeable.

	Combine that with not getting a good match for dark areas ("ghosting"
	pixels from other areas/frames) and you'll end up with dancing 
	splotches.

> ...possibly I need a new metric for ranking the desirability of found matches
> ... idea I had was to penalize matches based on the size of the motion vector
> But this didn't seem to do anything in practice.

	I think a luma based matching scheme, see below, might work better.

> Another idea was to gamma-correct the image on input to increase the effective
> resolution of darker areas, and to reverse that correction on output.

	Crushing the blacks will solve the problem but also lose any detail
	in darker areas.  Once you raise the black level (gamma correct) the
	data's lost - there's nothing to reverse later on.

> But I'm still reading the big book on digital video, and don't totally 
> understand the consequences of this idea.  One thing that seems clear, is

	If it's the Poynton book then pages 12 and 13 are the ones that 
	describe the problem ("code 100").

> that 8-bit-per-component input is probably too coarse to respond well to such
> gamma correction, and I'd probably have to wait until we could get all

	Doubt it.  If you move the black level up from 16 to 32 you'll 
	fix the problem of splotches for luma values up to 32 but codes between
	16 and 32 have then become nonexistent - there's no way to get them
	back.

> So does anyone have any good ideas?

	Good?  I'll let others make the value judgements but I do have a couple
	ideas to toss out ;-)

	1) Use a luma based threshold for matching.  Work a lot harder for
	   groups of pixels with an average (center weighted, straight avg,
	   try various types of averaging?) luma less than 100, work less hard
	   for values above 100.   Maybe have verious levels - 0-50 work extremely
	   hard, 50-75 a little less hard, 75-100 not too hard, above 150 not
	   any extra work, and so on.

	   Night scenes and dark/dim movies will denoise slowly but brighter
	   areas/movies shouldn't be affected to much

	2) median filter darker areas w/o detail - maybe some edge detection
	   and find areas that could be heavily median filtered to reduce 
	   black/grey level differences (this would, I think, also help find
	   suitable matches for dark areas).

	Have fun!

	Cheers,
	Steven Schultz


------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time, 
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge