Re: Multiprocess

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Anders,

Anders Bo Rasmussen a écrit :
> Hi Denis
> 
> I was thinking on something like the N+M threads in the faq. But with a
> restriction on the buffer size - e.g. on 32MB for each thread. And I
> would only have one process reading files and only one writing the output file.
> This would give a large speedup as long as the files are not larger than 32MB
> compressed. If the files are only a little larger it will speedup some. If the
> files are e.g. 2GB the speedup will be insignificant. 
> 
> Also a buffer in the writing process on e.g. 128MB would help to even
> out the gaps that happens in reading/compression.
> 
> Most files that are larger than 32MB is in my case files that are not
> compressed anyway because of their extention (.gz,.jpg,.cr2, etc) so I
> think all this would give a significant speedup. 
> 
> For files larger than 32MB something like 
> http://www.zlib.net/pigz/
> could be used?

You remind me a feature request about pigz, and fetching the reference
to it:
https://sourceforge.net/tracker/index.php?func=detail&aid=2908554&group_id=65612&atid=511615
I noticed that there had been very interesting feedback since the last
time I discussed about it there. So if some improvement has to be done
for parallel computing it will be done thanks pigz I hope.

> 
> /Anders

Thanks for reminding me this point!

Regards,
Denis.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFMoIRkpC5CI8gYGlIRAtNpAKChjHYotWZTSv6dfn+uzk7O7x89xwCgxRzq
eZFJGx9mRZp1KFOxfIYeajg=
=xZBg
-----END PGP SIGNATURE-----

------------------------------------------------------------------------------
Start uncovering the many advantages of virtual appliances
and start using them to simplify application deployment and
accelerate your shift to cloud computing.
http://p.sf.net/sfu/novell-sfdev2dev