Re: Multithreading not using most of my resources

Lobsingerp,Peter [CIS] wrote:
> Hello,
> 
> I am running a script that needs to check its results with a lot of
> random data (Monte Carlo methods). I am trying to use ithreads to do
> this. However, on an 8 CPU server, Perl never uses more than 200% CPU. I
> am running Perl 5.8.5 on Red Hat EL. Is there some configuration that
> could allow Perl to use more CPU while using ithreads?

wo/ knowing more about your app, its difficult to guess at any cause.
In theory, yes, ithreads should run concurrently on all your CPUs (since it
sits atop pthreads).

Do you have a lot of threads::shared data ? esp. w/ lock()'s around them ?
threads::shared does a lot of coarse grain locking (pretty much anytime you access
any threads::shared variable, it grabs a lock on a global shared interpretter context),
which may be causing contention issues.

Also, have you installed the latest threads and threads::shared pkgs
from CPAN ? (they are now "dual life" modules, available w/ CORE, but
with updates available on CPAN).

> 
> Also, how well do threads scale? My initial attempts are using 100
> threads but this may need to rise to 1000. As it stands it is taking
> forever to complete, but this is most likely caused by not having access
> to all CPUs that are available.
> 

How many modules are you using, and how much memory per thread are
you using ? 100 should be feasible, albeit slow to start, but
1000 may start thrashing.

Assuming you've updated to the latest threads module, you'll
also need to adjust the per-thread stack size. See

http://search.cpan.org/~jdhedden/threads-1.53/threads.pm#THREAD_STACK_SIZE

and

http://www.perlmonks.com/?node_id=533058

for details.

HTH,
Dean Arnold
Presicient Corp.