Re: HPC with Beoqulf

On Mon, Jan 17, 2011 at 9:42 AM, Carsten Aulbert
<[email protected]> wrote:
> Hi Patrick,
>
> On Monday 17 January 2011 09:24:47 Patrick Schmid wrote:
>> It has to be a HPC cause it's a given challenge: "Build a Linux HPC
>> with ethernet"
>>
>
> Good, and I can confirm it works and scales quite well (at least the beast we
> manage here ;))
>
>>
>> The goal is to split up the load of computing a job to multiple nodes.
>>
>> So you have a script / job you need to compute, and need a fast way to
>> to that. That's typically it ;)
>> My imagination is, that you upload this script to a master node, which
>> splits it up and distributes the parts to its nodes.
>>
>> hope you can understand :)
>>
>> What do you think? Still possible to do that with Beowulf?
>
> Sure, but it will depend how "social" your users will be, i.e. if they will
> partition the resources without fighting each other.
>
> You can use "dsh" (distributed shell) to start jobs manually on a set of

clustershell (in debian sid) provide many features like file copy,
nice agregated output and it scales much more better.

> cluster nodes (or loop over ssh connections), or you can go the way to use a
> "real" job scheduler (PBS, OGE, Torque, Condor, ...) depending of what you
> really want/need...

Slurm is a really good choice too and it is easy to configure. The
community is active and open. It is provided in Debian too.

>
> You just need to grow with the expectations I guess, start small and become
> bigger of time if need be.
>
> Cheers
>
> Carsten
>
>
> --
> To UNSUBSCRIBE, email to [email protected]
> with a subject of "unsubscribe". Trouble? Contact [email protected]
> Archive: http://lists.debian.org/[email protected]
>
>


cheers,

-- 
Stéphan Gorget