proposal for splitting translation tests into a separate project

Sometimes we make a Fedora release in which translations crash anaconda, 
and that sucks, and I'd like us not to do that. I have the beginnings of 
an idea of how not to do that and I'd like some feedback.

We have a handful of tests against translations in anaconda, but it's 
not just anaconda that's translated. Given that errors in translations 
in any library used by anaconda can crash anaconda, I feel that it's 
important to test the translations in all of the code we maintain to 
make sure that mistakes from translators aren't going to make anything 
blow up.

The transifex/zanata model is nice in that the code repository isn't 
crudded up with constant translation commits, but its biggest pitfall is 
that translations, and mistakes in translations, can be changed right up 
until the moment we create a release tarball. For that reason I'd like 
the tests to be run both as part of the jenkins CI and just before a 
release is built. The input would be the source tarball: the output of 
make dist for anaconda, make archive for blivet, etc.

This would not be as simple as moving tests/gettext/ into its own repo 
and adding a script or two. Most of the tests in tests/gettext/ test 
translatable strings rather than translated strings, and tests against 
translatable strings are mostly anaconda-specific.

The tests I would like to move out of anaconda and run against every 
project are:
   - gettext/gettext_warnings.sh, which mostly catches busted format 
strings (including errors on our end where the string needs named 
parameters)
   - the translation portion of glade/check_markup.py
       (I could go either on this, since it should really only be 
anaconda that has markup strings, but I think it should at least be 
something that is run at package build time)
   - a new test in reaction to 1283599 in which the translator screwed 
up the Plural-Forms header in lt.po in blivet.

Which leave these staying behind:
   - click.py, which ensures "clickable" anaconda info bars have 
something to click
   - contexts.py, which ensures that strings with keyboard accelerators 
have a translation context. Stuff from outside anaconda shouldn't be 
adding keyboard accelerators
   - gettext_potfiles.py, which ensures we keep POTFILES.in up to date. 
anaconda is the only one using autotools so the only one with a POTFILES.in
   - style_guide.py, which checks that we say "host name" instead of 
"hostname" and stuff like that. I dunno, maybe every project should get 
that treatment? Argue your case.

Discussion question: should this stuff go in a for-real package in 
Fedora? The pros would be easier release-time testing (could be an 
automatic part of rc-release or similar), integration with make ci, and 
Chris wouldn't need to configure any new stuff in jenkins. Cons would be 
that there's another package in Fedora and that testing translations in 
make ci would be adding yet more things out of our control.

As far as how all of this looks, I was thinking just a directory of 
checkers (maybe convert gettext_warnings.sh to python for consistency) 
and a script that runs them all. I suppose how the script finds the 
checkers depends on how real of a program we want it to be, whether it 
would need an argument to the directory or could just search via python 
imports.