Re: [lustre-devel] [LSF/MM/BPF TOPIC] [DRAFT] Lustre client upstreaming

On Mon, 2025-02-03 at 20:10 +0000, Day, Timothy wrote:
> > Unfortunately cloud is not very conductive to the way boilpot
> > operates,
> > the whole idea is to instantiate a gazillion of virtual machines
> > that
> > are run on a single physical host to overcommit the cpu (a lot!)
> > 
> > so I have this 2T RAM AMD box and I instantiate 240 virtual
> > machines on
> > it, each gets 15G RAM and 15 CPU cores (this is the important part,
> > if
> > you do not have cpu overcommit, nothing works)
> 
> You can do a similar thing in the cloud with bare metal instances.
> Normally,
> you can't do nested virtualization (i.e. QEMU/KVM inside EC2). But a
> bare
> metal instance avoids that issue. That's how I run ktest [1], which
> uses
> QEMU/KVM. Something like m7a.metal-48xl has 192 CPU and 768G of
> memory, so similar to the size you mention. What ratio of overcommit
> do you have? For RAM, it seems to be 2:1. What about for CPU?

don't really need memory overcommit (in fact it's somewhat
counterproductive), but since VMs typically don't use all RAM I wing it
and run somewhat more VMs than what memory permits.

as for CPU - the more overcommit is the better (my box has 96 cores).

if this is to be deployed in the cloud at will, some robust
orchestration is needed host-side - I create 240 libvirt driven VMs
with their own storage in LVM, dhcp-driven autoconf, nfs export host-
side with the right distro - just once per box lifetime and compiled
lustre every time I run testing (so a resh checkout of master-next
usually).
Then configure crashdumping and an inotifywatch-based script to catch
cores and do some light processign and ship results to the central data
collector. (might be more efficient to do using in-vm crashdumping
instead?)

at $11/hour the m7a.metal-48xl would take $264 to run for just one day,
a week is an eye-watering $1848, so running this for every patch is not
super economical I'd say.
_______________________________________________
lustre-devel mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-devel-lustre.org