xmltv/grab Get_nice.pm,1.23,1.24

Update of /cvsroot/xmltv/xmltv/grab
In directory sfp-cvs-1.v30.ch3.sourceforge.com:/tmp/cvs-serv15346

Modified Files:
	Get_nice.pm 
Log Message:
Add "minimum delay" option to reduce chance of scraper detection

Index: Get_nice.pm
===================================================================
RCS file: /cvsroot/xmltv/xmltv/grab/Get_nice.pm,v
retrieving revision 1.23
retrieving revision 1.24
diff -C2 -d -r1.23 -r1.24
*** Get_nice.pm	13 Feb 2010 19:26:29 -0000	1.23
--- Get_nice.pm	19 Oct 2013 09:14:13 -0000	1.24
***************
*** 10,13 ****
--- 10,21 ----
  # maximum delay - on average the sleep will be half that.
  #
+ #This random delay will be between 0 and 5 ($Delay) seconds. This means
+ # some sites will complain you're grabbing too fast (since 20% of your 
+ # grabs will be less than 1 second apart). To introduce a minimum delay 
+ # set $XMLTV::Get_nice::MinDelay to a value in seconds.
+ # This will be added to $Delay to derive the actual delay used. 
+ # E.g. Delay = 5 and MinDelay = 3, then the actual delay will be 
+ # between 3 and 8 seconds,
+ #
  # get_nice() is the function to call, however
  # XMLTV::Get_nice::get_nice_aux() is the one to cache with
***************
*** 24,27 ****
--- 32,36 ----
  use XMLTV;
  our $Delay = 5; # in seconds
+ our $MinDelay = 0; # in seconds
  our $FailOnError = 1; # Fail on fetch error
  
***************
*** 69,73 ****
          # nice to the server.
          #
!         my $next_get_time = $last_get_time + (rand $Delay);
          my $sleep_time = $next_get_time - time();
          sleep $sleep_time if $sleep_time > 0;
--- 78,82 ----
          # nice to the server.
          #
!         my $next_get_time = $last_get_time + (rand $Delay) + $MinDelay;
          my $sleep_time = $next_get_time - time();
          sleep $sleep_time if $sleep_time > 0;


------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from 
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk