Re: [PATCH RFC] check: add new option "--loop <n>" which runs each test multiple times

On Tue, May 05, 2026 at 03:34:09PM +0200, Theodore Tso wrote:
> On Wed, Apr 15, 2026 at 05:32:48PM -0400, Theodore Ts'o wrote:
> > Teach the check script a new option --loop, which re-run each test
> > multiple times.  This works very similarly to to -L, which will retry
> > a particular test after it first fails, except that the test is rerun
> > unconditionally.
> 
> Ping?  Does anyone have a preference between adding a new option,
> --loop, or changing the heaviour of the -i option?

Hi Ted,

Personally, I feel that once we have --loop, the existing -i starts to lose
its value. --loop is far more effective at catching specific, flaky bugs due
to its stashing logic.

Furthermore, if we eventually add something like --loop-while-successful, then
-I also becomes redundant. It might be worth considering a more unified naming
convention in the long run, perhaps renaming -L to --loop-on-fail and keeping
everything under the --loop-* family. This would be much clearer for other users
than the current mix of single-letter flags.

Welcome any further feedback. Also, feel free to weigh in if you have concerns
about the potential impact of renaming these option names.

Thanks,
Zorro

> 
> > This differs from the "-i <n>" option, which iterates each set of
> > tests <n> times instead of each test.  The -i option is problematic in
> > two ways.  First, it doesn't save the test artifacts from each test run.
> > This is unfortunate because when the developer is trying to debug a
> > flaky test failure, running "check -i 100" will run a test 100 times,
> > but if only the 42nd test fails, the NNN.out.bad file for that failing
> > test run is not preserved.  The second difference between --loop and
> > -i is the result.xml file is rewritten after each test set, so we do
> > not have the cumulative statistics of the 100 test runs in the junit
> > XML file.
> 
> 						- Ted
>