Re: Forked Processes Won't Migrate, Perl & ForkManager

The latest stable, built on the 2.4.26 kernel, with matching userland tools.

Jim

Moshe Bar wrote:

>Which version of openMosix is that, James?
>
>Moshe
>
> 
>
>-----Original Message-----
>From: openmosix-general-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>[mailto:openmosix-general-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org] On Behalf Of James
>Wagner
>Sent: Monday, July 24, 2006 12:16 PM
>To: openmosix-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>Subject: [openMosix-general] Forked Processes Won't Migrate,Perl &
>ForkManager
>
>Hi all
>
>I've built an OpenMosix cluster on Slackware Linux.  From any node, I can
>run mosmon and see the load for each node.  I built this to aid in the
>processing of a large amount of data, but have run into a problem.
>
>I wrote a perl script to read the file and use a foreach loop to process
>each record.  I have the program structured to print a status indicater to
>the screen, then call a subroutine to do the actual processing, passing it
>the variable containing the data.  That process then chews on the data,
>writes to a mysql database and a text file.
>
>I tried running this with the cluster enabled and there was no migration
>- the node the process was running on was maxxed out and the other ones were
>idle.  "Hey, I bet I need to re-write it to use threads!" I think and
>proceed to do so.  (When will I learn to read FAQs *first*??? ;) ) Anyways,
>the threaded implementation of my program didn't distribute the work among
>the nodes and after reading the FAQ, I re-wrote the program again to use the
>ForkManager package referenced in the FAQ.  (MUCH easier than using threads,
>plus I don't have to download and re-compile
>Perl.)
>
>After doing this and still obtaining the same results, I downloaded a stock
>analyzer program I found on the 'net, one possible use of which is to test
>openmosix clusters.  All the nodes went to 100% CPU usage and stayed there
>until the program was done, so I have to assume that the cluster is set up
>correctly.
>
>After this, I figured that the routine responsible for chewing on the data
>must be exiting too quickly for openmosix to recognize the high load and to
>distribute (takes about 5 seconds on an AMD 2500+), so I incorporated a
>crude blocking scheme and passed 1000 records a shot to be processed.  This
>make no difference either.
>
>I assumed that there must be a problem using a mysql connection on the local
>machine, so I changed the program to connect to mysql on a separate machine.
>No change.
>
>I then thought that it must be writing to a file on the local machine.  
>After trying to enable MFS and continually getting an 'unknown file system
>type' error, I gave up and changed it to write to a text file on another
>machine using a TCP connection.  I set up a daemon on the remote machine
>that does nothing but listen for connections and spit whatever it receives
>into a text file.  When I start the program, the only node working on the
>data is still the whatever one I start the process on.
>
>I'm frankly out of ideas of what to try.  Processing on the amount of data I
>need to will take on the order of months and I would very much like to speed
>this up, but don't know what else to try.  Any advice is appreciated.
>
>Jim
>
>
>-------------------------------------------------------------------------
>Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's
>Techsay panel and you'll get the chance to share your opinions on IT &
>business topics through brief surveys -- and earn cash
>http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>_______________________________________________
>openMosix-general mailing list
>openMosix-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
>https://lists.sourceforge.net/lists/listinfo/openmosix-general
>
>  
>

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys -- and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV