Re: [kvm-unit-tests PATCH] scripts/arch-run: Mark migration tests as SKIP if ncat is not available

On Tue, Dec 21, 2021 at 06:25:30PM +0100, Paolo Bonzini wrote:
> On 12/21/21 11:12, Thomas Huth wrote:
> > On 21/12/2021 10.58, Paolo Bonzini wrote:
> > > On 12/21/21 10:21, Thomas Huth wrote:
> > > > Instead of failing the tests, we should rather skip them if ncat is
> > > > not available.
> > > > While we're at it, also mention ncat in the README.md file as a
> > > > requirement for the migration tests.
> > > > 
> > > > Resolves: https://gitlab.com/kvm-unit-tests/kvm-unit-tests/-/issues/4
> > > > Signed-off-by: Thomas Huth <[email protected]>
> > > 
> > > I would rather remove the migration tests.  There's really no reason
> > > for them, the KVM selftests in the Linux tree are much better: they
> > > can find migration bugs deterministically and they are really really
> > > easy to debug. The only disadvantage is that they are harder to
> > > write.
> > 
> > I disagree: We're testing migration with QEMU here - the KVM selftests
> > don't include QEMU, so we'd lose some test coverage here.
> > And at least the powerpc/sprs.c test helped to find some nasty bugs in
> > the past already.
> 
> I understand that this is testing QEMU, but I'm not sure that testing QEMU
> should be part of kvm-unit-tests.  Migrating an instance of QEMU that runs
> kvm-unit-tests would be done more easily in avocado-vt or avocado-qemu.
>

Migrating is easier with avocado*, but if we want to migrate kut unit
tests, and the unit tests want to ensure the guest is in a specific state
at the time of the migration, then we'll still need the getchar() stuff.
And, if we need the getchar() stuff, then I think we also need a
lightweight way to test migration, which is currently the ncat-based
run_migration bash function. IOW, I vote we keep the code we have, but I'm
also in favor of people building new test harnesses for the kut *.flat
files which can better exercise QEMU or whatever.

Thanks,
drew