No gain in speed with threads

[email protected] ("Blanchette, Marco") Wed, 31 Dec 2008 08:32:59 -0600
Newsgroups perl.ithreads
Message-ID <C580DEBB.C184%[email protected]>
Dear all,

I am trying to speed up a very long procedure that I need to run on multipl=
e files and though that I could multithread different jobs on different fil=
es across multiple CPUs. For some reason that I don't really get, I only ac=
hieve very small time gain. I have included my script which essentially rep=
eat the same function, extractSeq() on multiple files using a maximum of fo=
ur threads.

I would really appreciate if I could finally understand how to use threads =
to speed up some of my lengthy scripts.

Thanks

Marco

#!/usr/local/bin/perl -w

use strict;
use Bio::SeqIO;
use threads;
use Getopt::Std;

our $opt_p;

init();
my @thr;
for (my $i=3D0;$i<=3D$#ARGV;$i++){
  push @thr, threads->new(\&extractSeq, $ARGV[$i]);
  if (scalar(@thr) =3D=3D $opt_p || $i =3D=3D $#ARGV){
    print "Running ",scalar(@thr)," parallel jobs\n";
    $_->join for @thr;
    undef @thr;
  }
}

sub extractSeq {
  my $file=3Dshift;

  my ($dir,$pre,$suf) =3D ($file=3D~/(^.+\/|^)(.+)\.(.+$)/);
  my $out_name =3D "$pre"."_CleanSeq.$suf";

  my $seqin =3D Bio::SeqIO->new(-file =3D> $file,
             -format =3D>'fasta');

  my $seq_out =3D Bio::SeqIO->new(-file =3D> ">$out_name",
                  -format =3D> 'fasta');

  while (my $seq =3D $seqin->next_seq){
    if ($seq->seq =3D~ /AGATC/){
      $seq->seq($seq->subseq(1,$-[0]+5));

      $seq_out->write_seq($seq);
    }
  }
    return(0);
}


sub init {
  getopts("p:");
  unless (@ARGV) {
    print("extractseq.pl [-p 4] seq_1.fa [seq_2.fa ...]\n\n",
      "Take the sequences from the Solexa sequences in Fasta format and\n",
      "\t1)Find the B primer\n",
      "\t2)Extract the sequences before the B primer leaving 5 nt of B prim=
er\n\n",
      "-p\tNumber of processors to be used to process the files when more t=
han one files are passed to the command line\n",
      "\tDefault 4\n\n");
    exit(1);
  }
  $opt_p =3D 4 unless $opt_p;
  return(0);
}

--
Marco Blanchette, Ph.D.
Assistant Investigator
Stowers Institute for Medical Research
1000 East 50th St.

Kansas City, MO 64110

Tel: 816-926-4071
Cell: 816-726-8419
Fax: 816-926-2018