Re: [PATCH v2 0/4] vLLM and the vLLM production stack

On Sat, Oct 04, 2025 at 09:38:10AM -0700, Luis Chamberlain wrote:
> This adds initial vLLM and vLLM production stack support on kdevops.
> 
> This v2 series augments vLLM support for real CPUs on bare metal using
> the DECLARE_HOSTS and also goes tested against a real GPU on the cloud,
> showing that essentially now anyone can use the vLLM production stack on
> any cloud provider we support in a flash. All we need are the instances
> which have GPUs added, and for that we expect growth soon using dynamic
> kconfig support.
> 
> Demo results of the temporary quick benchmark for all cases, GPUs, CPUs,
> and VMs are here:
> 
> https://github.com/mcgrof/demo-vllm-benchmark
> 
> We will expand support soon for synthetic engines, so we can stress test
> vLLM routing without the overhead of any real hardware. We then need to
> expand the scope of testing using the vLLM benchmarks and graphing them.
> 
> One of the benefits of all this is we can support *upstream kernel*
> changes and automatic testing of vLLM for compute in any complex way
> we can think of. Upstream kernels are not a requriement, we just support
> that. We also support AB testing since we already provide support for
> that, meaning folks can do AB testing with two different kernels.
> 
> This should be enough to kick the tires, and scale real production AI
> workloads on kdevops.
> 
> Since the first v1 patch already passed CI testing on kdevops, I'm
> posting this just as formality and will soon be merging this.

Merged.

  Luis