Re: [PATCH v4 8/9] KVM: selftests: Add rule to generate default tests for KVM selftests runner

On Tue, Mar 31, 2026, Vipin Sharma wrote:
> Add 'tests_install' rule in the Makefile.kvm to auto generate default
> testcases for KVM selftests runner. Preserve the hierarchy of test
> executables for autogenerated files. Remove these testcases on
> invocation of 'make clean'.
> 
> Autogeneration of default test files allows runner to execute default
> testcases easily. These default testcases don't need to be checked in as
> they are just executing the test without any command line options.
> 
> Signed-off-by: Vipin Sharma <[email protected]>
> ---
>  tools/testing/selftests/kvm/.gitignore   |  1 +
>  tools/testing/selftests/kvm/Makefile.kvm | 26 +++++++++++++++++++++++-
>  2 files changed, 26 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index 95af97b1ff9e..548d435bde2f 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -7,6 +7,7 @@
>  !*.S
>  !*.sh
>  !*.test
> +default.test
>  !.gitignore
>  !config
>  !settings
> diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
> index 6471fa214a9f..fb9439cb5f3d 100644
> --- a/tools/testing/selftests/kvm/Makefile.kvm
> +++ b/tools/testing/selftests/kvm/Makefile.kvm
> @@ -1,7 +1,7 @@
>  # SPDX-License-Identifier: GPL-2.0-only
>  include ../../../build/Build.include
>  
> -all:
> +all: tests_install
>  
>  LIBKVM += lib/assert.c
>  LIBKVM += lib/elf.c
> @@ -330,11 +330,15 @@ $(SPLIT_TEST_GEN_PROGS): $(OUTPUT)/%: $(OUTPUT)/%.o $(OUTPUT)/$(ARCH)/%.o
>  $(SPLIT_TEST_GEN_OBJ): $(OUTPUT)/$(ARCH)/%.o: $(ARCH)/%.c
>  	$(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@
>  
> +# Default testcases for KVM selftests runner will be generated in this directory.
> +DEFAULT_TESTCASES = testcases_default_gen

That's a cumbersome name.  It was also written by Yoda :-) How about default_testcases?

> +
>  EXTRA_CLEAN += $(GEN_HDRS) \
>  	       $(LIBKVM_OBJS) \
>  	       $(SPLIT_TEST_GEN_OBJ) \
>  	       $(TEST_DEP_FILES) \
>  	       $(TEST_GEN_OBJ) \
> +	       $(OUTPUT)/$(DEFAULT_TESTCASES) \
>  	       cscope.*
>  
>  $(LIBKVM_C_OBJ): $(OUTPUT)/%.o: %.c $(GEN_HDRS)
> @@ -363,3 +367,23 @@ cscope:
>  	find . -name '*.c' \
>  		-exec realpath --relative-base=$(PWD) {} \;) | sort -u > cscope.files
>  	cscope -b
> +
> +# Generate runner testcases in DEFAULT_TESTCASES directory.
> +# $(OUTPUT) is either CWD or specified in the make command.
> +tests_install: list_progs = $(patsubst $(OUTPUT)/%,%,$(TEST_GEN_PROGS))
> +tests_install:

This is very much *not* an install.  In fact, install is broken, because the
runner isn't added to TEST_FILES.

Another issue with install is that it flattens the directory structures, i.e.
drops the $ARCH/ subdirectories.  That causes issues for the default.test testcases
due to them using partially qualified paths.

And coming back to this with fresh eyes, I don't love generating the testcases
for the "normal" build.  It necessitates adding default.test to .gitignore,
effectively requires copying the runner to the output directory, and is pure
noise for folks that don't want to utilize the runner.

Rather than generate testcases for the default build, what if we make this 100%
opt-in, and take a hard dependency on install?  That should obviate the need for
ignoring default.test, because if someone is silly enough to install testcases
in the source tree, they get to deal with the noise.

And we can commit to having the default testcases provide completely unqualified
paths, so that the Just Work with the flattened output.  If someone wants to run
with an unflattened tree, then we should update the runner itself to handle
multiple paths, a la the actual PATH variable.

The other nice thing is that we can make the runner opt-in, without requiring the
user to run multiple make commands, by chaining testcases => install => all.  If
we try to generate testcases as an optional step to the normal build, then we'll
end up being able to run install without having generated the testcases, which
is "fine", but probably not what we want for people that are trying to use the
runner.

The obvious downside is that it requires doing an install to get the default
testcases, but IMO that's an acceptable tradeoff.

> +	$(foreach tc, $(TEST_PROGS), \
> +		$(shell mkdir -p $(OUTPUT)/$(DEFAULT_TESTCASES)/$(patsubst %.sh,%,$(tc))))
> +	$(foreach tc, $(TEST_PROGS), \
> +		$(shell echo $(tc) > $(patsubst %.sh,$(OUTPUT)/$(DEFAULT_TESTCASES)/%/default.test,$(tc))))
> +
> +	$(foreach tc, $(list_progs), \
> +		$(shell mkdir -p $(OUTPUT)/$(DEFAULT_TESTCASES)/$(tc)))
> +	$(foreach tc, $(list_progs), \
> +		$(shell echo $(tc) > $(patsubst %,$(OUTPUT)/$(DEFAULT_TESTCASES)/%/default.test,$(tc))))
> +
> +	@if [ ! -d $(OUTPUT)/runner ]; then             \

This is very wrong, as this will fail to pick up any changes made to the runner.

All in all, this as fixup?

diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index 83aa2fe01bac..91d2b21d396f 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -8,7 +8,6 @@
 !*.S
 !*.sh
 !*.test
-default.test
 !.gitignore
 !config
 !settings
diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index c5821d495ea6..fc9dc41fdbeb 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -1,8 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 include ../../../build/Build.include
 
-all: tests_install
-
 LIBKVM += lib/assert.c
 LIBKVM += lib/elf.c
 LIBKVM += lib/guest_modes.c
@@ -347,15 +345,13 @@ $(SPLIT_TEST_GEN_PROGS): $(OUTPUT)/%: $(OUTPUT)/%.o $(OUTPUT)/$(ARCH)/%.o
 $(SPLIT_TEST_GEN_OBJ): $(OUTPUT)/$(ARCH)/%.o: $(ARCH)/%.c
        $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@
 
-# Default testcases for KVM selftests runner will be generated in this directory.
-DEFAULT_TESTCASES = testcases_default_gen
+TEST_FILES := $(selfdir)/kvm/runner
 
 EXTRA_CLEAN += $(GEN_HDRS) \
               $(LIBKVM_OBJS) \
               $(SPLIT_TEST_GEN_OBJ) \
               $(TEST_DEP_FILES) \
               $(TEST_GEN_OBJ) \
-              $(OUTPUT)/$(DEFAULT_TESTCASES) \
               cscope.*
 
 $(LIBKVM_C_OBJ): $(OUTPUT)/%.o: %.c $(GEN_HDRS)
@@ -386,21 +382,18 @@ cscope:
        cscope -b
 
 # Generate runner testcases in DEFAULT_TESTCASES directory.
-# $(OUTPUT) is either CWD or specified in the make command.
-tests_install: list_progs = $(patsubst $(OUTPUT)/%,%,$(TEST_GEN_PROGS))
-tests_install:
-       $(foreach tc, $(TEST_PROGS), \
-               $(shell mkdir -p $(OUTPUT)/$(DEFAULT_TESTCASES)/$(patsubst %.sh,%,$(tc))))
-       $(foreach tc, $(TEST_PROGS), \
-               $(shell echo $(tc) > $(patsubst %.sh,$(OUTPUT)/$(DEFAULT_TESTCASES)/%/default.test,$(tc))))
+DEFAULT_TESTCASES = default_testcases
+
+testcases: list_progs = $(notdir $(patsubst $(OUTPUT)/%,%,$(TEST_GEN_PROGS)))
+testcases: install
+       $(foreach tc, $(notdir $(TEST_PROGS)), \
+               $(shell mkdir -p $(INSTALL_PATH)/$(DEFAULT_TESTCASES)/$(notdir $(patsubst %.sh,%,$(tc)))))
+       $(foreach tc, $(notdir $(TEST_PROGS)), \
+               $(shell echo $(tc) > $(patsubst %.sh,$(INSTALL_PATH)/$(DEFAULT_TESTCASES)/%/default.test,$(tc))))
 
        $(foreach tc, $(list_progs), \
-               $(shell mkdir -p $(OUTPUT)/$(DEFAULT_TESTCASES)/$(tc)))
+               $(shell mkdir -p $(INSTALL_PATH)/$(DEFAULT_TESTCASES)/$(tc)))
        $(foreach tc, $(list_progs), \
-               $(shell echo $(tc) > $(patsubst %,$(OUTPUT)/$(DEFAULT_TESTCASES)/%/default.test,$(tc))))
-
-       @if [ ! -d $(OUTPUT)/runner ]; then             \
-               cp -r $(selfdir)/kvm/runner $(OUTPUT);  \
-       fi
+               $(shell echo $(tc) > $(patsubst %,$(INSTALL_PATH)/$(DEFAULT_TESTCASES)/%/default.test,$(tc))))
 
        @:

> +		cp -r $(selfdir)/kvm/runner $(OUTPUT);  \
> +	fi
> +
> +	@:
> -- 
> 2.53.0.1118.gaef5881109-goog
> 

-- 
kvm-riscv mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kvm-riscv