Forked Processes Won't Migrate, Perl & ForkManager

Hi all

I've built an OpenMosix cluster on Slackware Linux.  From any node, I 
can run mosmon and see the load for each node.  I built this to aid in 
the processing of a large amount of data, but have run into a problem.

I wrote a perl script to read the file and use a foreach loop to process 
each record.  I have the program structured to print a status indicater 
to the screen, then call a subroutine to do the actual processing, 
passing it the variable containing the data.  That process then chews on 
the data, writes to a mysql database and a text file.

I tried running this with the cluster enabled and there was no migration 
- the node the process was running on was maxxed out and the other ones 
were idle.  "Hey, I bet I need to re-write it to use threads!" I think 
and proceed to do so.  (When will I learn to read FAQs *first*??? ;) )  
Anyways, the threaded implementation of my program didn't distribute the 
work among the nodes and after reading the FAQ, I re-wrote the program 
again to use the ForkManager package referenced in the FAQ.  (MUCH 
easier than using threads, plus I don't have to download and re-compile 
Perl.)

After doing this and still obtaining the same results, I downloaded a 
stock analyzer program I found on the 'net, one possible use of which is 
to test openmosix clusters.  All the nodes went to 100% CPU usage and 
stayed there until the program was done, so I have to assume that the 
cluster is set up correctly.

After this, I figured that the routine responsible for chewing on the 
data must be exiting too quickly for openmosix to recognize the high 
load and to distribute (takes about 5 seconds on an AMD 2500+), so I 
incorporated a crude blocking scheme and passed 1000 records a shot to 
be processed.  This make no difference either.

I assumed that there must be a problem using a mysql connection on the 
local machine, so I changed the program to connect to mysql on a 
separate machine.  No change.

I then thought that it must be writing to a file on the local machine.  
After trying to enable MFS and continually getting an 'unknown file 
system type' error, I gave up and changed it to write to a text file on 
another machine using a TCP connection.  I set up a daemon on the remote 
machine that does nothing but listen for connections and spit whatever 
it receives into a text file.  When I start the program, the only node 
working on the data is still the whatever one I start the process on.

I'm frankly out of ideas of what to try.  Processing on the amount of 
data I need to will take on the order of months and I would very much 
like to speed this up, but don't know what else to try.  Any advice is 
appreciated.

Jim


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV