Re: [PATCH 1/3] fio-tests: add multi-filesystem testing support

On Fri, Nov 21, 2025 at 09:07:17PM +0100, Daniel Gomez wrote:
> On 20/11/2025 04.15, Luis Chamberlain wrote:
> > This merges the long-pending fio-tests filesystem support patch that adds
> > comprehensive filesystem-specific performance testing capabilities to
> > kdevops. The implementation allows testing filesystem optimizations,
> > block size configurations, and I/O patterns against actual mounted
> > filesystems rather than just raw block devices.
> > 
> > The implementation follows the proven mmtests architecture patterns with
> > modular Kconfig files and tag-based ansible task organization, avoiding
> > the proliferation of separate playbook files that would make maintenance
> > more complex.
> > 
> > Key filesystem testing features include XFS support with configurable
> > block sizes from 4K to 64K with various sector sizes and modern features
> > like reflink and rmapbt. The ext4 support provides both standard and
> > bigalloc configurations with different cluster sizes. For btrfs, modern
> > features including no-holes, free-space-tree, and compression options
> > are available.
> > 
> > The multi-filesystem section-based testing enables comprehensive
> > performance comparison across different filesystem configurations by
> > creating separate VMs for each configuration. This includes support for
> > XFS block size comparisons, comprehensive XFS block size analysis, and
> > cross-filesystem comparisons between XFS, ext4, and btrfs.
> > 
> > Node generation for multi-filesystem testing uses dynamic detection
> > based on enabled sections, creating separate VM nodes for each enabled
> > section with proper Ansible groups for each filesystem configuration.
> > A/B testing support is included across all configurations.
> > 
> > Results collection and analysis is handled through specialized tooling
> > with performance overview across filesystems, block size performance
> > heatmaps, IO depth scaling analysis, and statistical summaries with CSV
> > exports.
> > 
> > The patch has been updated to work with the current codebase which now
> > uses workflow-specific template includes for host file generation rather
> > than embedding all workflow templates in a single hosts.j2 file. The
> > fio-tests specific template has been enhanced with multi-filesystem
> > support while maintaining backward compatibility with single filesystem
> > testing.
> > 
> > Generated-by: Claude AI
> > Signed-off-by: Luis Chamberlain <[email protected]>
> > ---
> >  .github/workflows/fio-tests.yml               |  98 +++
> >  CLAUDE.md                                     | 401 ++++++++++++
> >  PROMPTS.md                                    | 344 ++++++++++
> >  defconfigs/fio-tests-fs-btrfs-zstd            |  25 +
> >  defconfigs/fio-tests-fs-ext4-bigalloc         |  24 +
> >  defconfigs/fio-tests-fs-ranges                |  24 +
> >  defconfigs/fio-tests-fs-xfs                   |  74 +++
> >  defconfigs/fio-tests-fs-xfs-4k-vs-16k         |  57 ++
> >  defconfigs/fio-tests-fs-xfs-all-blocksizes    |  63 ++
> >  defconfigs/fio-tests-fs-xfs-all-fsbs          |  57 ++
> >  defconfigs/fio-tests-fs-xfs-vs-ext4-vs-btrfs  |  57 ++
> >  defconfigs/fio-tests-quick                    |  74 +++
> >  playbooks/fio-tests-graph-host.yml            |  76 +++
> >  playbooks/fio-tests-graph.yml                 | 168 +++--
> >  playbooks/fio-tests-multi-fs-compare.yml      | 140 ++++
> >  .../fio-tests/fio-multi-fs-compare.py         | 434 +++++++++++++
> >  .../tasks/install-deps/debian/main.yml        |   1 +
> >  .../tasks/install-deps/redhat/main.yml        |   1 +
> >  .../tasks/install-deps/suse/main.yml          |   1 +
> >  playbooks/roles/fio-tests/tasks/main.yaml     | 430 ++++++++++---
> >  .../roles/fio-tests/templates/fio-job.ini.j2  |  31 +-
> >  playbooks/roles/gen_hosts/tasks/main.yml      |  60 ++
> >  .../templates/workflows/fio-tests.j2          |  66 ++
> >  playbooks/roles/gen_nodes/tasks/main.yml      | 100 ++-
> >  workflows/fio-tests/Kconfig                   | 370 ++++++++---
> >  workflows/fio-tests/Kconfig.btrfs             |  87 +++
> >  workflows/fio-tests/Kconfig.ext4              | 114 ++++
> >  workflows/fio-tests/Kconfig.fs                |  75 +++
> >  workflows/fio-tests/Kconfig.xfs               | 170 +++++
> >  workflows/fio-tests/Makefile                  |  65 +-
> >  .../scripts/generate_comparison_graphs.py     | 605 ++++++++++++++++++
> >  .../generate_comprehensive_analysis.py        | 297 +++++++++
> >  workflows/fio-tests/sections.conf             |  47 ++
> >  33 files changed, 4350 insertions(+), 286 deletions(-)
> >  create mode 100644 .github/workflows/fio-tests.yml
> >  create mode 100644 defconfigs/fio-tests-fs-btrfs-zstd
> >  create mode 100644 defconfigs/fio-tests-fs-ext4-bigalloc
> >  create mode 100644 defconfigs/fio-tests-fs-ranges
> >  create mode 100644 defconfigs/fio-tests-fs-xfs
> >  create mode 100644 defconfigs/fio-tests-fs-xfs-4k-vs-16k
> >  create mode 100644 defconfigs/fio-tests-fs-xfs-all-blocksizes
> >  create mode 100644 defconfigs/fio-tests-fs-xfs-all-fsbs
> >  create mode 100644 defconfigs/fio-tests-fs-xfs-vs-ext4-vs-btrfs
> >  create mode 100644 defconfigs/fio-tests-quick
> >  create mode 100644 playbooks/fio-tests-graph-host.yml
> >  create mode 100644 playbooks/fio-tests-multi-fs-compare.yml
> >  create mode 100644 playbooks/python/workflows/fio-tests/fio-multi-fs-compare.py
> >  create mode 100644 workflows/fio-tests/Kconfig.btrfs
> >  create mode 100644 workflows/fio-tests/Kconfig.ext4
> >  create mode 100644 workflows/fio-tests/Kconfig.fs
> >  create mode 100644 workflows/fio-tests/Kconfig.xfs
> >  create mode 100755 workflows/fio-tests/scripts/generate_comparison_graphs.py
> >  create mode 100644 workflows/fio-tests/scripts/generate_comprehensive_analysis.py
> >  create mode 100644 workflows/fio-tests/sections.conf
> > 
> > diff --git a/.github/workflows/fio-tests.yml b/.github/workflows/fio-tests.yml
> > new file mode 100644
> > index 00000000..0a7c0234
> > --- /dev/null
> > +++ b/.github/workflows/fio-tests.yml
> 
> This is not following the current CI modular approach.
> It should be way easier to support a new workflow.
> 
> tree .github/workflows
> .github/workflows
> ├── config-tests.yml
> └── kdevops.yml
> 
> 1 directory, 2 files
> 
> Check kdevops.yml, it has this modular and reusable design. Steps are split
> in actions:
> 
> tree .github/actions
> .github/actions
> ├── archive
> │   └── action.yml
> ├── bringup
> │   └── action.yml
> ├── build-test
> │   └── action.yml
> ├── cleanup
> │   └── action.yml
> ├── configure
> │   └── action.yml
> ├── linux
> │   └── action.yml
> └── test
>     └── action.yml
> 
> 8 directories, 7 files
> 
> I know the documentation is still lacking, but Claude should be able to parse
> the directory structure and understand what's needed.

Indeed! I just copy and pasted your feedback and it did the magic. After
some CI testing I will post patches.

  Luis