[PATCH v3 00/10] terraform: add Lambda Labs cloud provider support

This v3 takes up the idea shared by Chuck to implement a CLI tool
and make the dynamic kconfig use it instead. We also add documentation
about this. This should hopefully pave the way for other cloud provider
support to leverage this more easily, AIs can just read the docs and
go to town.

Luis Chamberlain (10):
  gitignore: add entries for Lambda Labs dynamic configuration
  scripts: add Lambda Labs Python API library
  scripts: add Lambda Labs testing and debugging utilities
  scripts: add Lambda Labs credentials management
  scripts: add Lambda Labs SSH key management utilities
  kconfig: add dynamic cloud provider configuration infrastructure
  terraform/lambdalabs: add Kconfig structure for Lambda Labs
  terraform/lambdalabs: add terraform provider implementation
  ansible/terraform: integrate Lambda Labs into build system
  kconfigs: enable Lambda Labs cloud provider in menus

 .gitignore                                    |   9 +
 PROMPTS.md                                    |  56 ++
 defconfigs/lambdalabs                         |  15 +
 defconfigs/lambdalabs-gpu-1x-a10              |   9 +
 defconfigs/lambdalabs-gpu-1x-a100             |   8 +
 defconfigs/lambdalabs-gpu-1x-h100             |   8 +
 defconfigs/lambdalabs-gpu-8x-a100             |   8 +
 defconfigs/lambdalabs-gpu-8x-h100             |   8 +
 defconfigs/lambdalabs-shared-key              |  11 +
 defconfigs/lambdalabs-smart                   |  10 +
 docs/dynamic-cloud-kconfig.md                 | 461 +++++++++++++
 docs/lambda-cli.1                             | 245 +++++++
 kconfigs/Kconfig.bringup                      |   5 +
 playbooks/roles/gen_tfvars/defaults/main.yml  |  23 +
 .../templates/lambdalabs/terraform.tfvars.j2  |  18 +
 playbooks/roles/terraform/tasks/main.yml      |  85 +++
 scripts/cloud_list_all.sh                     | 152 +++++
 scripts/dynamic-cloud-kconfig.Makefile        |  44 ++
 scripts/dynamic-kconfig.Makefile              |   2 +
 scripts/generate_cloud_configs.py             | 113 ++++
 scripts/lambda-cli                            | 639 ++++++++++++++++++
 scripts/lambdalabs_api.py                     | 556 +++++++++++++++
 scripts/lambdalabs_credentials.py             | 242 +++++++
 scripts/lambdalabs_infer_region.py            |  61 ++
 scripts/lambdalabs_smart_inference.py         |  62 ++
 scripts/lambdalabs_ssh_key_name.py            | 135 ++++
 scripts/lambdalabs_ssh_keys.py                | 358 ++++++++++
 scripts/ssh_config_file_name.py               |  79 +++
 scripts/terraform.Makefile                    | 108 ++-
 scripts/update_ssh_config_lambdalabs.py       | 110 +++
 terraform/Kconfig.providers                   |  10 +
 terraform/Kconfig.ssh                         |  37 +-
 terraform/lambdalabs/Kconfig                  |  33 +
 terraform/lambdalabs/README.md                | 349 ++++++++++
 terraform/lambdalabs/SET_API_KEY.sh           |  20 +
 .../lambdalabs/ansible_provision_cmd.tpl      |   1 +
 terraform/lambdalabs/extract_api_key.py       |  40 ++
 terraform/lambdalabs/kconfigs/Kconfig.compute |  48 ++
 .../lambdalabs/kconfigs/Kconfig.identity      |  76 +++
 .../lambdalabs/kconfigs/Kconfig.location      |  73 ++
 terraform/lambdalabs/kconfigs/Kconfig.smart   |  25 +
 terraform/lambdalabs/kconfigs/Kconfig.storage |  12 +
 terraform/lambdalabs/main.tf                  | 154 +++++
 terraform/lambdalabs/output.tf                |  51 ++
 terraform/lambdalabs/provider.tf              |  19 +
 terraform/lambdalabs/shared.tf                |   1 +
 terraform/lambdalabs/vars.tf                  |  65 ++
 terraform/shared.tf                           |  14 +-
 48 files changed, 4656 insertions(+), 12 deletions(-)
 create mode 100644 defconfigs/lambdalabs
 create mode 100644 defconfigs/lambdalabs-gpu-1x-a10
 create mode 100644 defconfigs/lambdalabs-gpu-1x-a100
 create mode 100644 defconfigs/lambdalabs-gpu-1x-h100
 create mode 100644 defconfigs/lambdalabs-gpu-8x-a100
 create mode 100644 defconfigs/lambdalabs-gpu-8x-h100
 create mode 100644 defconfigs/lambdalabs-shared-key
 create mode 100644 defconfigs/lambdalabs-smart
 create mode 100644 docs/dynamic-cloud-kconfig.md
 create mode 100644 docs/lambda-cli.1
 create mode 100644 playbooks/roles/gen_tfvars/templates/lambdalabs/terraform.tfvars.j2
 create mode 100755 scripts/cloud_list_all.sh
 create mode 100644 scripts/dynamic-cloud-kconfig.Makefile
 create mode 100755 scripts/generate_cloud_configs.py
 create mode 100755 scripts/lambda-cli
 create mode 100755 scripts/lambdalabs_api.py
 create mode 100755 scripts/lambdalabs_credentials.py
 create mode 100755 scripts/lambdalabs_infer_region.py
 create mode 100755 scripts/lambdalabs_smart_inference.py
 create mode 100755 scripts/lambdalabs_ssh_key_name.py
 create mode 100755 scripts/lambdalabs_ssh_keys.py
 create mode 100755 scripts/ssh_config_file_name.py
 create mode 100755 scripts/update_ssh_config_lambdalabs.py
 create mode 100644 terraform/lambdalabs/Kconfig
 create mode 100644 terraform/lambdalabs/README.md
 create mode 100644 terraform/lambdalabs/SET_API_KEY.sh
 create mode 120000 terraform/lambdalabs/ansible_provision_cmd.tpl
 create mode 100755 terraform/lambdalabs/extract_api_key.py
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.compute
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.identity
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.location
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.smart
 create mode 100644 terraform/lambdalabs/kconfigs/Kconfig.storage
 create mode 100644 terraform/lambdalabs/main.tf
 create mode 100644 terraform/lambdalabs/output.tf
 create mode 100644 terraform/lambdalabs/provider.tf
 create mode 120000 terraform/lambdalabs/shared.tf
 create mode 100644 terraform/lambdalabs/vars.tf

-- 
2.50.1