testsuite layout conventions

Mike Frysinger via Binutils <[email protected]> Wed, 6 Jan 2021 00:12:22 -0500
Newsgroups	gmane.comp.gnu.binutils,gmane.comp.gdb.devel
Message-ID	<20210106051222.GB7494@vapier>
do we have docs/guidelines for testsuite/ dir layouts ?  we seem to
(inconsistently) use a lot of redundant layouts largely due to our
use of dejagnu.

dejagnu takes a tool-centric view of the world.  this makes sense
when you have a flat testsuite dir (e.g. if gas & binutils & ld &
gdb all shared a testsuite/ dir), but i'm not sure it makes that
much sense when they're all sep.  i think dejagnu is the single
reason for the layouts we have and it seems a bit unnecessary.

when you set tool=<tool> in its site.exp, it will only search for
dirs with <tool> prefixes under testsuite/.  i.e. it applies the
glob like testsuite/<tool>*/**.

a quick survey of binutils/gdb/gcc dirs that have tests:
* binutils     / testsuite / binutils-<group>  / <tests>
* gas          / testsuite / gas / <group>     / <tests>
* gdb          / testsuite / gdb.<group>       / <tests>
* gold         / testsuite / <tests>
* ld           / testsuite / ld-<group>        / <tests>
* libctf       / testsuite / libctf-<group>    / <tests>
* libiberty    / testsuite / <tests>
* sim          / testsuite / sim / <group>     / <tests>
* gcc          / testsuite / <tool>.<group>    / <tests>
* libatomic    / testsuite / libatomic.<group> / <tests>
* libffi       / testsuite / libffi.<group>    / <tests>
* libgomp      / testsuite / libgomp.<group>   / <tests>
* libgo        / testsuite / libgo.<group>     / <tests>
* libitm       / testsuite / libitm.<group>    / <tests>
* libphobos    / testsuite / libphobos.<group> / <tests>
* libstdc++-v3 / testsuite / libstdc++-<group> / <tests>
* libvtv       / testsuite / libvtv.<group>    / <tests>

the notable standouts are gold & libiberty.  they don't use dejagnu
while everyone else does.  so i'll ignore them.

libstdc++-v3 has additional test subdirs beyond the tool=libstdc++
prefix.  not sure how that works exactly.  so i'll ignore them.

gcc is also a bit funky, but i guess that's what you get for such a
large subdir.  i don't think we want to use that as precedence :).

for all the rest, you can see that we have three styles:
	<tool>.<group>/
	<tool>-<group>/
	<tool>/<group>/

the odd thing is that it seems most projects don't actually set
tool in their site.exp.  libstdc++, gdb, and sim do.  so i guess
either the others ones used to, or they just adopted a similar
naming convention at some point because others were ?

i've just finished cleaning up the sim/testsuite/ tree.  i'm left
with sim/testsuite/sim/<arch>/<tests>.  i spend a lot of time in
sim/<arch>/ and sim/testsuite/sim/<arch>/ comparing and trying to
collapse common logic.  the extra sim/ dir irks me ;).  it ends up
being easy to collapse by deleting 'set tool sim' from site.exp,
then just doing:
  cd testsuite/
  git mv sim/* ./
which leaves me with sim/testsuite/<arch>/.

based on the fact that no one seems to use dejagnu's tool setting
anymore, and we don't seem to have general guidance, and every
project is slightly different, changing sim to drop the "sim" tool
prefix is fine.
-mike
signature.asc (application/pgp-signature, 833 B)