Re: [PATCH 4/8] terraform: Add DataCrunch GPU cloud provider integration

On Sat, Dec 6, 2025, at 11:56 AM, Luis Chamberlain wrote:
> Add comprehensive DataCrunch cloud provider support for GPU instances
> ranging from Blackwell B300 to Tesla V100.
>
> Features intelligent GPU selection with three strategies: wildcard
> variant selection (ANY_1H100), tier-based fallback (H100_OR_LESS,
> B300_OR_LESS) for automatic fallback through GPU tiers when top options
> unavailable, and explicit instance types for production workloads.
>
> GPU tier hierarchy supports 10 tiers with automatic fallback to
> maximize provisioning success while capping costs. Tier-based selection
> recommended for most users (~$1.99/hr cap for H100 tier).
>
> Add capacity checking infrastructure to validate instance availability
> before provisioning. API credential management uses OAuth2 with secure
> token retrieval from ~/.datacrunch/credentials.
>
> Special handling for local provider development with dev_overrides in
> ~/.terraformrc, skipping force_init in Ansible terraform module.
>
> Add ML environment setup for DataCrunch instances: package updates, ML
> dependencies, PyTorch in virtualenv, NVIDIA driver reload with proper
> module unload sequence (nvidia_uvm, nvidia_drm, nvidia_modeset, nvidia)
> to avoid PyTorch/driver mismatch, Claude Code installation, MOTD with
> PyTorch activation instructions, and bashrc auto-activation.
>
> Add persistent volume support with KEEP=1 configuration option. Volume
> mappings cached in ~/.cache/kdevops/datacrunch/ enable fast
> reprovisioning (seconds vs minutes) while managing ~$10/month storage
> costs. Volume lifecycle wrapper scripts automate volume ID tracking
> across instance destroy/recreate cycles.
>
> Add defconfig files for tier-based selection, specific GPUs, and
> multi-GPU configurations. Comprehensive documentation focuses on
> capacity challenges, selection strategies, and best practices.
>
> When a DataCrunch instance type fails to deploy due to capacity issues,
> the terraform bringup will retry with a different instance from the
> same tier group. An exclude option can be used multiple times to skip
> instance types that have already failed.
>
> Generated-by: Claude AI
> Signed-off-by: Luis Chamberlain <[email protected]>

Merged into origin/main.

-- 
Chuck Lever