[PATCH v3 0/3] declared hosts support
Luis Chamberlain <[email protected]>
| Newsgroups | dev.linux.lists.kdevops |
|---|---|
| Message-ID | <[email protected]> |
This v3 now goes tested, and I simplfied the workflow so we just put a stop gap at the data partition on the role instead of adding tons of check on the tasks on the create_data_partition role in different tasks. This demos the DECLARE_HOSTS feature with minio warp, a new simple workflow we can test for S3 with minio. Luis Chamberlain (3): gen_hosts: use kdevops_workflow_name directly for template selection declared_hosts: add support for pre-existing infrastructure minio: add MinIO Warp S3 benchmarking with declared hosts support .gitignore | 2 + Makefile | 7 + defconfigs/minio-warp | 40 + defconfigs/minio-warp-ab | 41 + defconfigs/minio-warp-btrfs | 35 + defconfigs/minio-warp-declared-hosts | 56 ++ defconfigs/minio-warp-multifs | 74 ++ defconfigs/minio-warp-storage | 65 ++ defconfigs/minio-warp-xfs | 53 ++ defconfigs/minio-warp-xfs-16k | 53 ++ defconfigs/minio-warp-xfs-lbs | 65 ++ kconfigs/Kconfig.bringup | 8 + kconfigs/Kconfig.declared_hosts | 71 ++ kconfigs/workflows/Kconfig | 51 ++ kconfigs/workflows/Kconfig.data_partition | 13 +- playbooks/create_data_partition.yml | 2 + playbooks/minio.yml | 53 ++ playbooks/roles/ai_setup/tasks/main.yml | 40 +- .../create_data_partition/defaults/main.yml | 1 + playbooks/roles/devconfig/defaults/main.yml | 2 + playbooks/roles/devconfig/tasks/main.yml | 25 + playbooks/roles/gen_hosts/defaults/main.yml | 1 + playbooks/roles/gen_hosts/tasks/main.yml | 315 ++----- playbooks/roles/gen_hosts/templates/hosts.j2 | 242 +---- .../roles/gen_hosts/templates/workflows/ai.j2 | 99 ++ .../gen_hosts/templates/workflows/blktests.j2 | 58 ++ .../gen_hosts/templates/workflows/cxl.j2 | 7 + .../templates/workflows/declared-hosts.j2 | 239 +++++ .../templates/workflows/fio-tests.j2 | 38 + .../gen_hosts/templates/workflows/fstests.j2 | 72 ++ .../gen_hosts/templates/workflows/gitr.j2 | 41 + .../gen_hosts/templates/workflows/linux.j2 | 110 +++ .../gen_hosts/templates/workflows/ltp.j2 | 41 + .../gen_hosts/templates/workflows/minio.j2 | 173 ++++ .../gen_hosts/templates/workflows/mix.j2 | 62 ++ .../gen_hosts/templates/workflows/mmtests.j2 | 77 ++ .../gen_hosts/templates/workflows/nfstest.j2 | 41 + .../gen_hosts/templates/workflows/pynfs.j2 | 7 + .../templates/workflows/reboot-limit.j2 | 33 + .../templates/workflows/selftests.j2 | 53 ++ .../gen_hosts/templates/workflows/sysbench.j2 | 53 ++ playbooks/roles/gen_nodes/tasks/main.yml | 116 +++ playbooks/roles/minio_destroy/tasks/main.yml | 34 + playbooks/roles/minio_install/tasks/main.yml | 61 ++ playbooks/roles/minio_results/tasks/main.yml | 86 ++ playbooks/roles/minio_setup/defaults/main.yml | 16 + playbooks/roles/minio_setup/tasks/main.yml | 100 ++ .../roles/minio_uninstall/tasks/main.yml | 17 + playbooks/roles/minio_warp_run/tasks/main.yml | 249 +++++ .../templates/warp_config.json.j2 | 14 + workflows/Makefile | 4 + workflows/ai/Makefile | 3 - workflows/blktests/Makefile | 3 - workflows/cxl/Makefile | 2 - workflows/demos/reboot-limit/Kconfig | 5 + workflows/fio-tests/Makefile | 3 - workflows/fstests/Makefile | 3 - workflows/gitr/Makefile | 3 - workflows/linux/Makefile | 1 - workflows/ltp/Makefile | 3 - workflows/minio/Kconfig | 23 + workflows/minio/Kconfig.docker | 66 ++ workflows/minio/Kconfig.storage | 364 ++++++++ workflows/minio/Kconfig.warp | 141 +++ workflows/minio/Makefile | 76 ++ .../minio/scripts/analyze_warp_results.py | 858 ++++++++++++++++++ .../minio/scripts/generate_warp_report.py | 404 +++++++++ .../minio/scripts/run_benchmark_suite.sh | 116 +++ workflows/mmtests/Makefile | 3 - workflows/nfstest/Makefile | 3 - workflows/pynfs/Makefile | 3 - workflows/selftests/Makefile | 3 - workflows/sysbench/Makefile | 3 - 73 files changed, 4751 insertions(+), 554 deletions(-) create mode 100644 defconfigs/minio-warp create mode 100644 defconfigs/minio-warp-ab create mode 100644 defconfigs/minio-warp-btrfs create mode 100644 defconfigs/minio-warp-declared-hosts create mode 100644 defconfigs/minio-warp-multifs create mode 100644 defconfigs/minio-warp-storage create mode 100644 defconfigs/minio-warp-xfs create mode 100644 defconfigs/minio-warp-xfs-16k create mode 100644 defconfigs/minio-warp-xfs-lbs create mode 100644 kconfigs/Kconfig.declared_hosts create mode 100644 playbooks/minio.yml create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ai.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/blktests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/cxl.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/declared-hosts.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fio-tests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/fstests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/gitr.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/linux.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/ltp.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/minio.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mix.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/mmtests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/nfstest.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/pynfs.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/reboot-limit.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/selftests.j2 create mode 100644 playbooks/roles/gen_hosts/templates/workflows/sysbench.j2 create mode 100644 playbooks/roles/minio_destroy/tasks/main.yml create mode 100644 playbooks/roles/minio_install/tasks/main.yml create mode 100644 playbooks/roles/minio_results/tasks/main.yml create mode 100644 playbooks/roles/minio_setup/defaults/main.yml create mode 100644 playbooks/roles/minio_setup/tasks/main.yml create mode 100644 playbooks/roles/minio_uninstall/tasks/main.yml create mode 100644 playbooks/roles/minio_warp_run/tasks/main.yml create mode 100644 playbooks/roles/minio_warp_run/templates/warp_config.json.j2 create mode 100644 workflows/minio/Kconfig create mode 100644 workflows/minio/Kconfig.docker create mode 100644 workflows/minio/Kconfig.storage create mode 100644 workflows/minio/Kconfig.warp create mode 100644 workflows/minio/Makefile create mode 100755 workflows/minio/scripts/analyze_warp_results.py create mode 100755 workflows/minio/scripts/generate_warp_report.py create mode 100755 workflows/minio/scripts/run_benchmark_suite.sh -- 2.50.1