Re: ithreads and sockets

At 21:08 +0000 8/30/03, Steve Schein wrote:
>The challenge:  Using ithreads for TCP and UDP sockets
>- client:
>Multiple (should scale to thousands) Java apps. running mostly on PC's
>- program:
>Large Perl program connected to a MySQL database and Apache server.  The
>program is intended to allow for login requests on a TCP socket, receive
>"stayalive" messages from clients on a dedicated UDP socket and send UDP
>messages on a dedicated UDP socket, the peer address of which will
>change based on criteria establish in the main program

I'm afraid these prerequisites will disqualify using Perl ithreads 
for your application in a production setting.  This is caused by the 
fact that each Perl ithread gets a _copy_ of all data-structures in 
each thread.  In practice, this means that even with a small number 
of modules loaded, the memory footprint soon becomes very large. 
Observe this output from Benchmark::Thread::Size:

   #   (ref)    noexport      export                                   
   0    1727 ± 4     +96 ± 6    +100 ± 4
   1    2097 ± 6    +141 ± 4    +145 ± 4
   2    2394 ± 6    +168 ± 4    +173   
   5    3284 ± 2    +244 ± 2    +257 ± 4
  10    4775 ± 2    +376 ± 6    +391 ± 2
  20    7753 ±12    +635 ±10    +662 ± 4
  50   16723 ±120   +1361 ±14   +1440 ±14
100   31578 ±18   +2696 ±22   +2822 ±22

==== noexport ===============================================
use threads::shared ();
==== export ================================================
use threads::shared;
========================================================

The first column shows the number of threads created.  The second 
column shows the memory footprint of the process without any 
additional code.  Any other columns show the amount of extra memory 
used for the indicated piece of code.  Since memort footprints are 
rarely exactly the same when using threads, the deviation from the 
average is also shown.

With just loading threads::shared, you're increasing your memory 
footprint by between 100Kbyte (for one thread) and 27KByte (if you're 
running 100 threads).  Please also note that exporting 4 subroutines 
has a significant memorty impact: about 250 bytes per subroutine per 
thread.  And that's just for a code reference!  Let alone for any 
significant amount of data.  Take a look at the following benchmark, 
in which no shared variables are used.  Instead, in the first case a 
package lexical with 1000 bytes is created.  And the second case 
contains a package lexical with 100000 bytes:

   #   (ref)    K1            K100
   0    1727 ± 4      +4 ± 4    +200   
   1    2097 ± 6      +4 ± 4    +401 ± 2
   2    2395 ± 4      +8 ± 6    +600   
   5    3287 ± 4     +12       +1202 ± 2
  10    4772 ± 4     +29 ± 2   +2204 ± 2
  20    7753 ± 6     +49 ± 2   +4207 ± 4
  50   16691 ±12    +102 ±10  +10215 ± 4
100   31584 ±16    +217 ±18  +20230 ±12

==== K1 ===================================================
my $var = 'a' x 1000;
==== K100 =================================================
my $var = 'a' x 100000;
=========================================================

You'll notice that each thread takes about twice the size of the 
variable in memory.  So that means if you have 100 threads with a 
single 100K variable, it's going to cost you 20 MB of RAM extra. 
Unshared RAM, I might add.  This clearly demonstrates that the data 
is copied to each thread.


>I've already done testing with the TCP and UDP socket functionality and
>that works fine over the Internet but I currently run these as parallel
>processes which is not ideal for my program.  I've even played with
>ithreads a bit to get familiar with the environment, but I'm not exactly
>certain how to proceed with building an overall structure to meet my
>needs.  I'm not looking for this mailing list to provide designs, I'm
>mainly looking for guidance on how to use Perl ithreads to accomplish my
>goals from a structural perspective.

If you're going to need thousands of threads, you're going to need a 
lot of RAM.  And the copying of data becomes a considerable CPU 
burden as well.  But I don't need a lot of data, you say.  Well, you 
do need modules, and modules contain subroutines and variables.  And 
those are all data.  Observe the output of this little program:

use threads ();
my $coderef = sub { 'foo' };
print "before: coderef = $coderef\n";
threads->new( sub {print "thread: coderef = $coderef\n"} )->join;
print " after: coderef = $coderef\n";

which outputs:

before: coderef = CODE(0x107c88)
thread: coderef = CODE(0x20dbac)
  after: coderef = CODE(0x107c88)

Note that the coderef inside the thread is _different_ from the one 
in the main program.  Named subroutines are basically only hashes 
with the name of the subroutine as the key and the coderef as the 
value.  So you're copying data for each existing subroutine!


>I want to keep this message as short as possible as not to scare away
>any prospective contributors, but I am very happy at any point to get
>into more detail on all aspects of the program.

I think you'll be much better off with an approach that uses fork() 
as you appear to have been doing so far.  Forking a process is _way_ 
faster than spawning a Perl ithread and uses _way_ fewer resources. 
The OS's Copy On Write system makes for much better memory usage than 
Perl ithreads, because the copying of data basically counteracts 
anything the OS can do to save memory.

If you still want to use Perl ithreads, and are not worried about 
performance per se, you might want to have a look at the forks.pm 
module I wrote.  It basically has the same interface as Perl 
ithreads, but uses fork() to start threads (hence its name).  It also 
uses a TCP connection for each thread that has shared variables, so 
that may turn into a lot of sockets in your case.


>My sincerest thanks in advance for sharing your wisdom on the subject.
>I think it is safe to say there aren't too many ithreads gurus in the
>world, and I'm happy to have found some congregated together sipping
>fine brandy and chatting thoughtfully on the subject.

I'm afraid there isn't too much chatting going on here lately.  But 
the list does get monitored by some people.  ;-)

In any case, I hope this helps.


Liz