Re: [kvm-unit-tests PATCH] scripts/arch-run: Mark migration tests as SKIP if ncat is not available

On 21/12/2021 18.25, Paolo Bonzini wrote:
> On 12/21/21 11:12, Thomas Huth wrote:
>> On 21/12/2021 10.58, Paolo Bonzini wrote:
>>> On 12/21/21 10:21, Thomas Huth wrote:
>>>> Instead of failing the tests, we should rather skip them if ncat is
>>>> not available.
>>>> While we're at it, also mention ncat in the README.md file as a
>>>> requirement for the migration tests.
>>>>
>>>> Resolves: https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/issues/4
>>>> Signed-off-by: Thomas Huth <[email protected]>
>>>
>>> I would rather remove the migration tests.  There's really no reason for 
>>> them, the KVM selftests in the Linux tree are much better: they can find 
>>> migration bugs deterministically and they are really really easy to 
>>> debug. The only disadvantage is that they are harder to write.
>>
>> I disagree: We're testing migration with QEMU here - the KVM selftests 
>> don't include QEMU, so we'd lose some test coverage here.
>> And at least the powerpc/sprs.c test helped to find some nasty bugs in the 
>> past already.
> 
> I understand that this is testing QEMU, but I'm not sure that testing QEMU 
> should be part of kvm-unit-tests.

It's the combination of QEMU (migration handling) + KVM in the kernel 
(register saving and restoring) that we are testing here. If you say that 
QEMU should not be involved at all, we could also say that all 
kvm-unit-tests should be converted to KVM selftests instead...

>  Migrating an instance of QEMU that runs 
> kvm-unit-tests would be done more easily in avocado-vt or avocado-qemu.

But we don't have the environment for compiling small, Linux-independent 
test kernels there, do we? So unless there is a way to write and compile 
small test kernels in that framework, I think kvm-unit-tests is the better 
fit for these kind of tests.

  Thomas