Re: [PATCH v2 0/4] vLLM and the vLLM production stack

On Wed, Oct 08, 2025 at 01:46:15PM -0400, Chuck Lever wrote:
> On 10/4/25 1:14 PM, Chuck Lever wrote:
> >> But the rest of the changes are legit, please feel
> >> free to cherry pick what you see useful from here:
> >>
> >> https://github.com/linux-kdevops/kdevops/tree/ci-testing/
> >> mcgrof/20251004-cloud-bill
> > Great! I will have a look at those.
> 
> Have some comments/requests on these, not sure where to post them.
> Oldest to newest:
> 
> - aws: prevent SSH key conflicts across multiple kdevops directories
> 
> This one was posted before, and my comment still stands: this is badly
> needed IMO, but it should work for all cloud providers, not just aws,
> and the new Kconfig options should go in the existing kconfig ssh menu
> for terraform, probably.
> 
> Do you want me to work on adapting this one

Yes please!

> - Add static GPU Kconfig support for AWS
> 
> Wondering if my dynamic instance type menu already brings in these new
> GPU-enabled instance types.

What do you mean?

make cloud-config
Cloud Provider Configuration Summary
============================================================

✓ Lambda Labs: 0/1 instances available, 1 regions, pricing varies
  Kconfig files generated successfully

⚠ AWS: Dynamic configuration not yet implemented

So I suspect we don't yet have this? To be clear, I didn't intend
to add static instance types, I was just trying to add something
temporary so I can get AWS going with GPU instances while you
land dynamic AWS support upstream. And so my static patches can be
ignored.

> - Add make cloud-bill target for AWS cost tracking
> 
> Nit: I'd like to see provider-specific scripts go into
> 
>   terraform/<provider>/scripts/

Sure, any chance you can adapt that to your preference?

> I'm sorry that I had to drop the pricing information from my dynamic
> menu patches.

I really don't care for that stuff, I just want functionality
at this point.

> - terraform/aws: use default VPC to avoid VPC limit issues
> 
> I think we can make this work, and IIRC some of the other providers
> also provision default VPCs. Making it switchable (use the default,
> or create one for me) makes sense. We might consider following the
> precedent that OCI has set here (use an existing VPC).

Sure whatever you recommend, I just know I'm out of VPCs in my account
and so can't add more.

> There are some other resources that have similar limits.
> 
> 
> - terraform/aws: fix EBS volume availability zone mismatch
> 
> Fair catch, but why not use the AZ that the instance is in rather
> than the AZ that the subnet is in?

Sure.

> - terraform/aws: enable public IP assignment for instances
> - terraform/aws: prefer subnets with public IP auto-assignment
> 
> As above, might need some work, but these two look do-able. Probably
> should be squashed into "terraform/aws: use default VPC to avoid VPC
> limit issues"

Sure, I'm in hopes you can adapt to your preference.

> - terraform/aws: fix GPU AMI selection in terraform templates
> 
> No comment on this one. I need to first go and merge in your original
> GPU AMI patches. I'd like to see that integrated into the existing AWS
> Kconfig compute menu.

That'd be nice.

> - ansible: map GPU instance configurations to terraform instance types
> 
> - defconfigs: fix GPU instance choice configuration
> 
> Wondering if these two are still necessary with my dynamic menu patches.

I am not aware of your dynamic menu patches! Please just merge :)

> - slack-billing: add AWS cost notifications to Slack
> 
> Clever, but isn't this something that should be configured via the
> cloud console? Not really sure.

You mean AWS console? No, we didn't find anything. They just want your
money.

> - kconfig: fix Slack notification configuration syntax errors
> 
> Squash-me.

Indeed.

> - defconfigs: add AWS P5.4xlarge GPU instance support
> 
> No comment.

I'm hoping you can just take this and adapt it as you see fit as
I've just been waiting for your overhaul to get AWS GPU instances
going. Right now I can't use them and the above branch was an attempt
to get it going with static instances, which is clearly not right.

Any chance you might be able to take what you think deserves to get
upstream?

  Luis