Re: Unreliable test suites?

On Wed, Oct 14, 2015 at 12:58 PM,  <[email protected]> wrote:
> I'm doing some work on gdb and want to run the testsuites to confirm that I didn't mess it up.
>
> The problem is that a number of them seem to be quite unreliable.  I've seen test runs where gdb.btrace/step.exp and/or stepi.exp have a pile of failures, but then when I rerun either just those tests, or the whole suite, they pass.
>
> Since I haven't a clue how the reverse execution stuff works, I don't know if this is expected.  It seems strange.  I also don't know what to do about it if it's not supposed to be like that. For now, I'm just running things a couple of times, and if they pass once, I call it good enough.

Those particular tests don't fail for me, even with check-parallel,
but then it could depend on the target.
OTOH several tests *are* flaky, especially under load.

What I normally do is for the failures,
run just those tests one at a time (to reduce load induced failures).
And do that in the before and after trees.
If they fail in the before tree too, then don't worry about it.