xmltv/grab/na_dtv tv_grab_na_dtv,1.20,1.21

Update of /cvsroot/xmltv/xmltv/grab/na_dtv
In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv11901

Modified Files:
	tv_grab_na_dtv 
Log Message:
add option for number of child processes: --processes N

IMPORTANT: the default is now to run only 1 process (to avoid issues in Windows). Use --processes N to run more (previously was fixed at 8). Obviously this is not backward compatible but it means that Windoze users can at least *use* this grabber (they can't currently).

Index: tv_grab_na_dtv
===================================================================
RCS file: /cvsroot/xmltv/xmltv/grab/na_dtv/tv_grab_na_dtv,v
retrieving revision 1.20
retrieving revision 1.21
diff -C2 -d -r1.20 -r1.21
*** tv_grab_na_dtv	23 May 2014 15:38:18 -0000	1.20
--- tv_grab_na_dtv	23 May 2014 16:55:46 -0000	1.21
***************
*** 11,21 ****
  tv_grab_na_dtv --help
  
! tv_grab_na_dtv --configure [--config-file FILE] [--root-url URL] 
  
! tv_grab_na_dtv [--config-file FILE] [--root-url URL] 
!                  [--days N] [--offset N] [--channel xmltvid,xmltvid,...]
                   [--output FILE] [--quiet] [--debug]
  
! tv_grab_na_dtv --list-channels [--config-file FILE] [--root-url URL]
                   [--output FILE] [--quiet] [--debug]
  
--- 11,21 ----
  tv_grab_na_dtv --help
  
! tv_grab_na_dtv --configure [--config-file FILE] 
  
! tv_grab_na_dtv [--config-file FILE] 
!                  [--days N] [--offset N] [--processes N]
                   [--output FILE] [--quiet] [--debug]
  
! tv_grab_na_dtv --list-channels [--config-file FILE]
                   [--output FILE] [--quiet] [--debug]
  
***************
*** 46,49 ****
--- 46,57 ----
  B<--offset N> Start grabbing at today + N days.
  
+ B<--processes N> Nunber of processes to run to fetch program details.
+ 8 is a good number to try. You could try more with plenty of CPU and
+ bandwidth. More processes will reduce the time it takes to fetch your
+ listings. But be warned, the benefit might not be as much as you think,
+ and the more processes you initiate the more you are making it obvious
+ you are scraping and more likely to get banned by the source site. 
+ A 'fast' website scraper is an oxymoron!
+ 
  B<--quiet> Only print error-messages on STDERR.
  
***************
*** 106,111 ****
  
  # This is the number of concurrent processes for scraping and parsing
! # program details.  You could try more with plenty of CPU and bandwidth.
! my $MAX_PROCESSES = 8;
  
  my $VERBOSE = 1;
--- 114,127 ----
  
  # This is the number of concurrent processes for scraping and parsing
! # program details. 8 is a good number to try. You could try more with
! # plenty of CPU and bandwidth. More processes will reduce the time it
! # takes to fetch your listings.
! # But be warned, the benefit might not be as much as you think,
! # and the more processes you initiate the more you are making it obvious
! # you are scraping and more likely to get banned by the source site. 
! # A 'fast' website scraper is an oxymoron!
! #
! # Set default to 1 so the xmltv.exe does NOT use perl fork
! my $MAX_PROCESSES = 1;
  
  my $VERBOSE = 1;
***************
*** 163,166 ****
--- 179,190 ----
    binmode $fhq;
  
+   if ( $MAX_PROCESSES == 1 ) {
+     
+     # Don't create any child processes - do it ourself
+     $proc_number = 0;
+     &child_logic();
+     
+   } else {
+     
    # Create the children.
    for ( $proc_number = 0 ; $proc_number < $MAX_PROCESSES ; ++$proc_number ) {
***************
*** 210,213 ****
--- 234,239 ----
    }
  
+   }
+ 
    print STDERR "Done.  Writing results and cleaning up.\n" if ($VERBOSE);
  
***************
*** 338,341 ****
--- 364,369 ----
          '$Id$',
        description => "North America using www.directv.com",
+       extra_options    => [qw/procs=i processes=i/],      # allow 'procs' as a synonym for 'processes'
+       defaults         => {'procs'=>0, 'processes'=>'0'}
      }
    );
***************
*** 343,346 ****
--- 371,378 ----
    # If we get here, then we are generating data normally.
  
+   # Get max( $MAX_PROCESSES, --procs, --processes )
+   $MAX_PROCESSES = ($MAX_PROCESSES, $opt->{procs})[$MAX_PROCESSES < $opt->{procs}];
+   $MAX_PROCESSES = ($MAX_PROCESSES, $opt->{processes})[$MAX_PROCESSES < $opt->{processes}];
+   
    $VERBOSE = !$opt->{quiet};
    $DEBUG   = $opt->{debug};
***************
*** 561,565 ****
      last unless ($readlen);
  
!     #print STDERR "Process $proc_number Queue entry:$line\n" if ($VERBOSE);
      # Queue line format (%-6s %10s)
      if ( $line =~ /^([0-9-]+)\s+(.*)$/ ) {
--- 593,597 ----
      last unless ($readlen);
  
!     print STDERR "Process $proc_number Queue entry:$line" if ($DEBUG);
      # Queue line format (%-6s %10s)
      if ( $line =~ /^([0-9-]+)\s+(.*)$/ ) {


------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs