Re: Multithreading not using most of my resources

Peter,

--- Al Tobey <[email protected]> wrote:

> On Mon, 2006-12-11 at 15:03 -0500, Lobsingerp,Peter [CIS] wrote:

> > Also, how well do threads scale? My initial attempts are using 100
> > threads but this may need to rise to 1000. As it stands it is
> taking
> > forever to complete, but this is most likely caused by not having
> access
> > to all CPUs that are available.
> 
> You'll need a LOT of RAM.   Depending on your data sharing needs, it
> may
> be way more efficient to use fork() [...]

In the spirit of benefiting from fork() copy-on-write memory saving
(and depending on your IPC performance needs), you may wish to try the
forks CPAN module:

http://search.cpan.org/~rybskej/forks-0.20/lib/forks.pm

It is an ithread drop-in replacement, using a mix of fork() and socket
programming to provide an alternate, memory-efficient Perl threads API.

Also, assuming moderate use of threads::shared resources, forks.pm's
use of multiple processes should help you take full advantage of all
your CPU resources without needing to rewrite any code in your
ithread-enabled application.


That being said...I do wonder about the low overall use of CPU
resources with native ithreads given that you are doing (what sounds
like) relatively CPU-intensive numerical analysis.  Can you give an
example of your overall thread application architecture is designed, in
how you distribute resources for processing and aggregrate results?

Regards,
Eric