unexpected stall when join()ing
[email protected] ((James Taylor))
| Newsgroups | perl.ithreads |
|---|---|
| Organization | PerlDiscuss.com |
| Message-ID | <[email protected]> |
Hello, (I sent this message on Friday to [email protected], but it doesn't seem to have made it there. I'm now sending this through the perldiscuss website. Apologies if this gets posted twice.) I have written a small webbot that uses threads. Due to the memory leak reported by the "Scalars leaked" error message, I have coded the programme to allow a fixed number of threads to execute, then quit, then respawn itself. (I just read that this memory leak may have been fixed in 5.8.1, so I may be using an outdated version.) This way works fine, as the state of the bot is saved (to files) between runs and restarting the bot only takes a few seconds. (Ideally, though, this bot would be running as a daemon-like process.) The problem I am experiencing occurrs during the final "clean-up" phase. After having run a number of threads, my programme essentially just waits around for the remaining children to complete their job before it (the parent) quits. During this time, and more specifically when few ( < 10 ) children still remain, the still-running children (and occasionally the parent, too, ) stall. I have written a minimalist programme that (sometimes) experiences the same behaviour and have included it below. I am using both 'threads' and 'threads::shared' and don't know which of these (if any) is related to my problem. Any help would be greatly appreciated. Quick English explanation of programme: Create a total of 1000 threads, but never create any new ones if 50 are already running. (i.e. try to save some resources) Each thread just sleeps a random (integer) number of seconds (0 to 9), and then returns with the number of seconds slept. After the 1000 threads have been created, we wait for them to finish before the main programme exits. Diagnostic messages are printed to screen. ################################################################ #!/usr/bin/perl -w use strict; use threads; use threads::shared; $| = 1; sub func( $ ); my @threads = (); my $num_threads = 0; # these numbers produce the error fairly frequently on my machine # you may need to change this on yours my $total_threads = 1000; my $max_simultaneous_threads = 50; my @finished_threads : shared = (); my $thread_id; my $result; for my $i (0..$total_threads) { { # join any finished threads lock(@finished_threads); while (@finished_threads) { $thread_id = shift @finished_threads; $result = $threads[$thread_id]->join; $num_threads--; print "Thread $thread_id finished: $result.\n"; } } while ($num_threads >= $max_simultaneous_threads) { # don't continue until we join at least some threads { lock(@finished_threads); while (@finished_threads) { $thread_id = shift @finished_threads; $result = $threads[$thread_id]->join; $num_threads--; print "Thread $thread_id finished: $result.\n"; } } if ($num_threads >= $max_simultaneous_threads) { print "Yielding ($num_threads)...\n"; threads->yield; } } # ok, create a new thread now $threads[$i] = threads->new(\&func, $i); $num_threads++; } while ($num_threads > 0) { # just wait around until all threads join back { lock(@finished_threads); while (@finished_threads) { $thread_id = shift @finished_threads; $result = $threads[$thread_id]->join; $num_threads--; print "Thread $thread_id finished: $result.\n"; } } if ($num_threads > 0) { print "Yielding at end ($num_threads)...\n"; threads->yield; } } # this subroutine is what is called as a new thread each time sub func( $ ) { my ($num) = @_; my $return = int(rand(10)); sleep($return); { lock(@finished_threads); push @finished_threads, $num; return $return; } } ########################################################### Thank you for any assistance you may provide. Cheers, James P.S. I used this "online" email address in order to protect myself from spam. If desired, I can reply personnally to postings from my "real" email address.