Re: mulithread rework

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, Apr 24, 2008 at 07:01:10AM +1000, Con Kolivas wrote:
> On Thu, 24 Apr 2008 01:02:57 Edouard Gomez wrote:
> > Quoting Con Kolivas <[email protected]>:
[...]
> > > So are there any other potential gains? There are minor improvements
> > > to actual FPS encoded if all the threads are generated in advance and
> > > not generated/destroyed on each frame being encoded because thread
> > > creation is not exactly instantaneous, and allows CPU balancing to
> > > move them around in advance rather than always starting on the same
> > > CPU as the parent process. However, even this does not amount to much
> > > improvement (I can't give you a firm figure but eyeballing the FPS was
> > > not impressive).
> >
> > Yeah that was my idea with the threadpools, thread creation cost is
> > reduced to a minimum.
> >
> > If it can help you investigate the threading pool + task queue, i have some
> > free code online that can serve as a good starting point.
> >
> > See:
> > http://ed.gomez.free.fr/vrac/threadpool.c
> > http://ed.gomez.free.fr/vrac/threadpool.h
> >
> > The API is quite easy to understand, but be aware that the flush operation
> > doesn't sync on thread task completion, it just makes sure the task queue
> > is empty before returning. It's a 5min exercice (left to the reader) to add
> > a sync on tasks completion instead.
> 
> I already said I tried thread pools (see eyeballing comment). They're 
> unimpressive in their performance gains. The way work is currently divided up 
> is simply not that scalable. This is not limited by the locking/threading 
> model (which is what I originally thought). Changing the threading and 
> locking model reduces cpu cost but does not improve the throughput.The work 
> needs to be divided up differently or else nothing can make it scale any 
> more.

If anyone is interrested in seeing how scaleable multithreaded encoding
based on independant slices is (or is not ...) and by how much it worsens
the quality per bitrate. FFmpegs mpeg1/2/4/h263 encoders support
multithreaded encoding based on independant slices.

PS: for a realistic test with ffmpeg one should tweak the parameters
as our defaults are no good quality/bitrate wise.

PS2: I dont have a multi cpu system, so i actually dont know how bad
ffmpeg scales ...

[...]
- -- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

it is not once nor twice but times without number that the same ideas make
their appearance in the world. -- Aristotle
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFID99tYR7HhwQLD6sRAsGMAJkBNaFEoyHj8Tb724tfjKs8tOBEYACeNjfu
TDWmdOWNsPPciSC5AbjYho8=
=CCRv
-----END PGP SIGNATURE-----