Re: large number of short jobs?

Yanming Di wrote:

> I am new to openmosix cluster environment. I am running some
> scientific simulations, where I need run a large number of short jobs.
> Simply put, I have different parameter settings for my simulation, for
> each parameter setting, I need to repeat a short program 10000 times.
> What I want is, for that 10000 jobs, with the same parameter setting,
> I want to run them on the same node sequentially. But for different
> parameter settings, I want to run them on different nodes parallelly.
> So basically, if I have 20 computers, I will log on each one of them,
> and start a simulation of 10000 runs under a specific parameter
> setting on each machine.
> Now I have a cluster of 20 computers. I organize my jobs using a bash
> script. If I simply submit all the jobs from one computer, that
> computer become very heavily loaded. Is there a easy solution for
> situations like this.
>
Have a short shell script which does your 10000 repeats. Make sure it
locks itself ('echo 1 > /proc/$$/lock').

Have another shell script which loops over the sets of parameters you
want to test. For each set of parameters, either

a) send it to a random machine, with 'mosrun -j'
b) have some sort of loop counter, and send each job to a specific
machine with 'mosrun -<node number>'

-- 
Dr Mark Mackey
Director, Development
Cresset Biomolecular Discovery Ltd

_______________________________________________
openMosix-general mailing list
openMosix-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/openmosix-general