Re: [PATCH v2 0/4] vLLM and the vLLM production stack

On 10/4/25 12:38 PM, Luis Chamberlain wrote:
> This adds initial vLLM and vLLM production stack support on kdevops.
> 
> This v2 series augments vLLM support for real CPUs on bare metal using
> the DECLARE_HOSTS and also goes tested against a real GPU on the cloud,
> showing that essentially now anyone can use the vLLM production stack on
> any cloud provider we support in a flash. All we need are the instances
> which have GPUs added, and for that we expect growth soon using dynamic
> kconfig support.

As an update/road-map on that:

I think Lambda has GPU support already, and AWS has enough dynamic menu
support now that GPU-enabled instance types are available there with the
default menus in the git tree. Please let me know if that's missing
something.

I haven't done the follow-up work yet to integrate GPU-enabled AMIs into
the AWS Compute menu. That seems like it should be the top priority. I
need to go back and look at what you did to generate those in your
prototype, to close those gaps.

When that is complete, my next steps are to ask Claude to "copy" the
scripts from terraform/aws/scripts to the other three major cloud
providers that kdevops supports... an NFS bake-a-thon is this coming
week, so there will be some delay.

In the medium term, adding support for enabling RDMA fabrics in these
environments is on my to-do list. I believe that will allow testing
things like GPU direct with NVMe-o-F devices.


-- 
Chuck Lever