Multiprocessing and y4mdenoise

I've been meaning to write a more multiprocessing version of y4mdenoise, but I keep finding general performance improvements.  I think I've finally run out, though.  (I'll check in the latest batch of them pretty soon.)

I have two basic methods.  One involves dividing the frame into "slices" that get denoised separately, with some interaction between slices for regions that cross slice boundaries.  The upside is that this scales up to arbitrary numbers of hardware-threads, which should be a big boon when I start throwing 1920x1080 high-definition video at it.  The downside is that y4mdenoise will no longer be deterministic -- the handling of regions that overlap slice boundaries will be subject to race conditions.  Something tells me this will lead to visible artifacts at slice boundaries.  I don't know how often this would happen, or if increasing the match-size-throttle would make it happen less often, but this problem could potentially render this method useless.

The other method involves a few internal worker-threads for the remaining big bottleneck that I haven't otherwise solved -- the flood-filling of pixel-group matches.  The problem here is how to coordinate the worker-threads with the main threads.  Experience with multi-threaded programming tells me that mutexes are WAY too slow to be used this much -- if I try to use several hundred thousand mutex locks/unlocks within a single frame, I'll just beat the operating system to death.  I'd prefer the model I used on the Cell processor when doing SPU programming, i.e. have a worker thread check for something to do (using atomic operations), and it it doesn't find anything, sleep for a short time, long enough to prevent hammering the data bus, but short enough to prevent the program from stalling.  Something like 50 microseconds.  It looks like either usleep() or nanosleep() should be available on all of the platforms we support -- nanosleep() has
 apparently been part of POSIX since 1993.  The upside of this method is that y4mdenoise remains deterministic, and that it'll work well with the Cell processor's SPUs; the downside is that it doesn't scale up to arbitrary numbers of hardware-threads.

So...
[1] Any comments on these two methods?
[2] Can I rely on usleep() or nanosleep() being available on all our platforms?

I intend to try the parallel-flood-filling method first, but am willing to give everyone else a chance to change my mind. :-)

-Steven Boswell

------------------------------------------------------------------------------
Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT 
is a gathering of tech-side developers & brand creativity professionals. Meet
the minds behind Google Creative Lab, Visual Complexity, Processing, & 
iPhoneDevCamp as they present alongside digital heavyweights like Barbarian 
Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com

_______________________________________________
Mjpeg-developer mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mjpeg-developer