Re: large number of short jobs?

On 19:03 Wed 14 Jun     , Yanming Di wrote:
> Now I have a cluster of 20 computers. I organize my jobs using a bash script. If I simply submit all the jobs from one computer, that computer become very heavily loaded. Is there a easy solution for situations like this.

Well, if you get those 10000 jobs to run as one sequential process
(rather than creating 10000 small and short-lived processes) then oM
should distribute them among your 20 nodes pretty evenly. 

If not I'd suggest you to use a DRM system (Distributed Resource
Management, e.g. Sun Grid Engine). Such a system can start the
short-lived jobs directly on the target machine. 

That ain't as beautiful as the transparent oM approach, but oM has a
problem with short-lived processes: a process can only be migrated
after it's creation. If it doesn't run at all it's migration will be
more expensive than the local execution.

Regards
-Andreas

_______________________________________________
openMosix-general mailing list
openMosix-general-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/openmosix-general

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEkUDRxQzSAzXqsyYRAlNqAJ92Icw3gCvvWBXztB+sFrEX4CB0GQCdFRKO
mu6TT7DL4/M7oJ1jwXuq8tM=
=92B3
-----END PGP SIGNATURE-----