unexpected stall when join()ing

[email protected] ((James Taylor))
Newsgroups perl.ithreads
Organization PerlDiscuss.com
Message-ID <[email protected]>
Hello,

(I sent this message on Friday to [email protected], but it doesn't
seem to have made it there.  I'm now sending this through the perldiscuss
website.  Apologies if this gets posted twice.)

I have written a small webbot that uses threads.  Due to the memory leak
reported by the "Scalars leaked" error message, I have coded the programme
to allow a fixed number of threads to execute, then quit, then respawn
itself.  (I just read that this memory leak may have been fixed in 5.8.1,
so I may be using an outdated version.)   This way works fine, as the
state of the bot is saved (to files) between runs and restarting the bot
only takes a few seconds.  (Ideally, though, this bot would be running as
a daemon-like process.)

The problem I am experiencing occurrs during the final "clean-up" phase. 
After having run a number of threads, my programme essentially just waits
around for the remaining children to complete their job before it (the
parent) quits.  During this time, and more specifically when few ( < 10 )
children still remain, the still-running children (and occasionally the
parent, too, ) stall.   I have written a minimalist programme that
(sometimes) experiences the same behaviour and have included it below.  I
am using both 'threads' and 'threads::shared' and don't know which of
these (if any) is related to my problem.  Any help would be greatly
appreciated.

Quick English explanation of programme:
Create a total of 1000 threads, but never create any new ones if 50 are
already running.  (i.e. try to save some resources)  Each thread just
sleeps a random (integer) number of seconds (0 to 9), and then returns
with the number of seconds slept.   After the 1000 threads have been
created, we wait for them to finish before the main programme exits. 
Diagnostic messages are printed to screen.

################################################################

#!/usr/bin/perl -w

use strict;

use threads;
use threads::shared;

$| = 1;

sub func( $ );

my @threads = ();
my $num_threads = 0;
# these numbers produce the error fairly frequently on my machine
#   you may need to change this on yours
my $total_threads = 1000;
my $max_simultaneous_threads = 50;
my @finished_threads : shared = ();
my $thread_id;
my $result;




for my $i (0..$total_threads) {
 {
   # join any finished threads
   lock(@finished_threads);
   while (@finished_threads) {
     $thread_id = shift @finished_threads;
     $result = $threads[$thread_id]->join;
     $num_threads--;
     print "Thread $thread_id finished: $result.\n";
   }
 }


 while ($num_threads >= $max_simultaneous_threads) {
   # don't continue until we join at least some threads
   {
     lock(@finished_threads);
     while (@finished_threads) {
   $thread_id = shift @finished_threads;
   $result = $threads[$thread_id]->join;
   $num_threads--;
   print "Thread $thread_id finished: $result.\n";
     }
   }
   if ($num_threads >= $max_simultaneous_threads) {
     print "Yielding ($num_threads)...\n";
     threads->yield;
   }
 }

 # ok, create a new thread now
 $threads[$i] = threads->new(\&func, $i);
 $num_threads++;
}


while ($num_threads > 0) {
 # just wait around until all threads join back
 {
   lock(@finished_threads);
   while (@finished_threads) {
     $thread_id = shift @finished_threads;
     $result = $threads[$thread_id]->join;
     $num_threads--;
     print "Thread $thread_id finished: $result.\n";
   }
 }
 if ($num_threads > 0) {
   print "Yielding at end ($num_threads)...\n";
   threads->yield;
 }
}


# this subroutine is what is called as a new thread each time
sub func( $ ) {
 my ($num) = @_;
 my $return = int(rand(10));
 sleep($return);

 {
   lock(@finished_threads);
   push @finished_threads, $num;
   return $return;
 }
}

###########################################################

Thank you for any assistance you may provide.

Cheers,
James
P.S. I used this "online" email address in order to protect myself from
spam.  If desired, I can reply personnally to postings from my "real"
email address.
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.