Re: mulithread rework

Radek Czyz ([email protected]) wrote:
> Yes, updating own complete_count_self (which means updating someone 
> else's complete_count_above) can be changed to a proper signal for that 
> thread below to continue.

The whole thing could be replaced with a much cleaner thread pool
implementation. A consumer would push new bunches of available blocks
to the work queue and let the pool consumers pop them from the queue
when they are ready to process new data.

That would avoid all that "spinlocking" on the complete_count_above
vars. But that means using conditions and mutexes to guarantee the
queue thread safety. So in the end you add OS work which can cost
a lot.

The thread pool doesn't require spawning threads at each FrameCodeP
or FrameCodeB.

So to make the threadpool worth using, you have to ensure that
the cost of the thread queue synchronisation is <<< compared to the
task unit of each consumer.

From a theoritical point of view, a task unit could be a single block,
but in reality better task units should be 10 or more blocks on the same
row. Not very important as this could be a tuned value anyway.

You can have a look at some thread pool code in c and c++:
 - http://ed.gomez.free.fr/vrac/threadpool.c /h
 - http://ed.gomez.free.fr/vrac/ThreadPool.cpp /hpp

That implementation lacks a "global join" waiting for all consumers to
finish. The flush function just flushes the queue till it has no tasks
left but leaves before the consumers finished. It is left as an exercise
of 5 min to the reader :-)

PS: the simple pthreads defines in XviD aren't good enough
    for these thread pools to work on Win32, you have to use 
    Pthread-win32 for condition variables.

-- 
Edouard Gomez