RE: iThreads troubles

Okay, I'm elated at the progress this morning.  Once again I can thank
you guys enough for pointing me in the right direction.  Take a look at
this quick and dirty example of my use of threads and Thread::Queue

It worked fairly well however my memory usage was on a steady climb as
well as my processor hitting that 100% mark.

This is simulation of what My program is trying to do using threads with
a set number of resources but and unknown number of jobs.

use strict;
use threads;
use Thread::Queue;

my ($thread,$resource,$wanPending);
my $wanQ    = new Thread::Queue;
my $modemQ  = new Thread::Queue;

# WAN Resources
my $wan1 = 'O:';
my $wan2 = 'P:';
my $wan3 = 'Q:';
my $wan4 = 'R:';
my $wan5 = 'S:';

# Modem Resources
my $modem1 = 'U:';
my $modem2 = 'V:';
my $modem3 = 'W:';
my $modem4 = 'X:';
my $modem5 = 'Y:';

# Build the Queue
$wanQ   ->enqueue($wan1,$wan2,$wan3,$wan4,$wan5);
$modemQ ->enqueue($modem1,$modem2,$modem3,$modem4,$modem5);

# Number of Jobs to kick off...
my @wanList = (1..50);

# Create a new Thread with whats in the queue.  
foreach my $numFilesToDo (@wanList) {
    
    $wanPending = $wanQ->pending;
    print $wanPending." wan resources left\n";
    
    # If we have a resource available in the queue
    # then create a thread to doJob
    if ($wanQ->pending != 0) {

        $resource = $wanQ->dequeue;
        
        $thread = threads->create("doJob","$numFilesToDo","$resource");
        $thread->detach();

    } else {
        sleep(10);
    }

}

sub doJob {
    
    my ($jobId,$resource) = @_;
    my $random = int( rand (10)) + 2;
    print " :: Job ID number $jobId has resource $resource in use\n";
    
    # simulate performing job
    sleep($random);
    
    # Requeue it after we are done
    $wanQ->enqueue($resource);
    
    return;
}



-----Original Message-----
From: Mike Pomraning [mailto:[email protected]] 
Sent: Wednesday, April 19, 2006 9:27 AM
To: Daniel Rychlik
Cc: [email protected]; [email protected]
Subject: RE: iThreads troubles

On Wed, 19 Apr 2006, Daniel Rychlik wrote:

> Some of you have asked for code examples.
> 
> One of issues that I am having is when the thread enters into
> getAvailableResource routine its not seeing that the
> $Resources{'WAN1'}->{INUSE} = 1; has been made unavailable.  The idea
is
> for the next thread to pick up the next unavailable resource.  I tried
> to sleep(1) between thread creation, and also used threads::shared
> share(%Resources); which also failed.  

Dan,

A few quick remarks/suggestions.

First, please post the smallest possible code that reproduces your
symptoms:
pare down your problematic program, stuff it with stubs until you've got
a
cogent proof-of-problem.

Second, &share()ing your hash clears it [0], so populate it /after/ you
mark
it shared.  Additionally, the hash members themselves have to be
explicitly
shared, IIUC. [1]

Third, your 'INUSE' check looks to have a race.  You'll need to &lock(),
use
a semaphore, etc.  As others have suggested, Thread::Queue makes these
kinds
of problems a lot easier.  You could, perhaps, manage the "wan" and
"modem"
resources themselves in a queue -- pop off the next available one, use
it,
then re-queue it for another job later on.  TMTOWTDI.

[0] "When share is used on arrays, hashes, array refs or hash refs, any
    data they contain will be lost."
 
http://search.cpan.org/~jdhedden/threads-shared-1.01/shared.pm#BUGS_AND_
LIMITATIONS

[1] "This places restrictions on what may be assigned to shared array
and
    hash elements: only simple values or references to shared variables
are
    allowed..."
 
http://www.perldoc.com/perl5.8.4/pod/perlthrtut.html#Shared-And-Unshared
-Data

Good luck,
Mike
--
cat: .signature: Message too long