Re: [PATCH v2 02/10] scripts: add Lambda Labs Python API library

On Thu, Aug 28, 2025 at 04:13:12PM -0400, Chuck Lever wrote:
> On 8/28/25 4:03 PM, Luis Chamberlain wrote:
> > On Thu, Aug 28, 2025 at 04:00:09PM -0400, Chuck Lever wrote:
> >> On 8/28/25 3:33 PM, Luis Chamberlain wrote:
> >>> On Thu, Aug 28, 2025 at 02:59:18PM -0400, Chuck Lever wrote:
> >>>> On 8/27/25 5:28 PM, Luis Chamberlain wrote:
> >>>>> Add a Python library for interacting with the Lambda Labs cloud API.
> >>>>> This provides core functionality to query instance types, regions,
> >>>>> availability, and manage cloud resources programmatically.
> >>>>>
> >>>>> The API library handles:
> >>>>> - Instance type enumeration and capacity checking
> >>>>> - Region availability queries
> >>>>> - SSH key management operations
> >>>>> - Error handling and retries for API calls
> >>>>> - Parsing and normalizing API responses
> >>>>>
> >>>>> This forms the foundation for dynamic Kconfig generation and terraform
> >>>>> integration, but doesn't enable any features yet.
> >>>>
> >>>> Thanks for splitting this up. Much much easier for humble humans to
> >>>> understand now. There are some really interesting ideas in the series;
> >>>> kudos to you and Claude.
> >>>>
> >>>> The other cloud providers have their own house-brewed command line
> >>>> utilities that are typically packaged by distributions (eg, oci, aws,
> >>>> and so on), as well as their own Ansible collections.
> >>>>
> >>>> I assume that Lambda does not have those (yet).
> >>>
> >>> Right. It sad to not see such a thing, which meant it was actually
> >>> harder to do this inference work, and yet it shows its possible. Which
> >>> also means, technically in theory, it might even be possible to do the
> >>> same with other cloud providers without a CLI tool.
> >>>
> >>>> Can the patch
> >>>> description mention that? 
> >>>
> >>> Sure.
> >>>
> >>>> Basically that's the whole purpose for this
> >>>> patch and the two following, IIUC.
> >>>>
> >>>> I would actually encourage this script to be restructured and renamed so
> >>>> that it works like the tooling available from other cloud providers.
> >>>
> >>> So you mean we write Lambda's CLI tool? Sure we can try that, but who do
> >>> we follow? And do we need to do that now? Or can I do that later?
> >>>
> >>>> Separate the API calls and retrieval of information (which is rather
> >>>> general purpose) from the translation of that information to Kconfig
> >>>> menus (which is specific to kdevops).
> >>>
> >>> You mean, to re-use all the knowledge we used for Kconfig dynamic
> >>> kconfig to a create a specific tool, which can then be itself leveraged
> >>> for both the dynamic kconfig and also a user tool?
> >>
> >> I think so: Write a command line tool that makes API queries and spits
> >> out JSON. Write a second tool that takes the JSON and turns it into
> >> Kconfig menus. Re-use the first tool for other tasks. That is how I
> >> was thinking of building dynamic menus for the other providers; their
> >> existing command line tools are tool #1 above.
> > 
> > Yeah I think this is really *it*!
> > 
> >> But, we can do it later. I won't hold you up with crazy ideas.
> > 
> > Its not crazy at all, proper architecture trumps all. But if we didn't
> > start evaluating this patch series with this inference cloud thing for
> > dynamic kconfig thing we wouldn't have ended up here. And so I think we
> > have a solid path.
> > 
> > I'd like to extend your suggestion further: I am not sure if we need a
> > different CLI tool per cloud provider. Wouldn't it be nice if we had
> > unified way to query such things and output json for each cloud
> > provider?
> 
> Basically I'm re-using the tools each provider gives us because, well,
> I'm a lazy sod. It gives some immediate traction.
> 
> But I can see that doing it that way means we have a distinct JSON to
> Kconfig translation for each provider.
> 
> If handling the API queries is as easy as the Python script here looks,
> then the only hard part to building a single tool will be turning the
> provider-specific JSON into something common (and YAML output might be
> easier for Ansible and Python to deal with than JSON, IMO).

I see the challenges ahead, yeah screw it. Let's for now diverge and
later we can evaluate. I don't think we *need* a common tool other than
a utopian dream of sharing dynamic kconfig generation scripts. That can
wait until we have each cloud provider set up.

  Luis