Re: [perl #60642] [CAGE] add a codingstd test to ensure TODOed tests have an RT ticket number

James Keenan via RT wrote:
> On Tue Nov 18 10:22:25 2008, [email protected] wrote:
> 
> This will probably be quite challenging.  Let's assume that all tests
> are found in files with names ending in '.t'.  Those .t files can be
> written in any language, which probably have different ways of
> classifying a test as TODO.
> 
> My count tonight is that 1384 .t files in the distribution.  Of these
> 524 are *not* found under ./languages/.
> 
> I wonder if we could formulate the specification in this ticket a bit
> more precisely before someone embarks on coding.

Yes, absolutely.  I just added the basic ticket on the spur of the 
moment, to make sure I didn't forget about it.

I've been thinking about this.  A few things come to mind, for instance 
detecting the language based on the hashbang (if any) or subdirectory 
it's in, and invoking a language-specific parser.  And detecting the 
cases we can't handle, and skipping those.

But to me that sounds like way too much work.  It doesn't really matter 
to me whether the ticket number occurs within the TODO output string, a 
nearby comment is good enough for me.  So how about skipping all the 
above nonsense and just ignoring the test language entirely?  How about 
a simple regex-based test that tallies all instances of /TODO/ in the 
set of test files, skipping the lines that start with obvious comment 
characters, and for each instance, looks for a match of /#\d+/?  It can 
even expand the search to also look a couple lines above and below the 
TODO line, for additional flexibility.  I think that should be 
reasonable for most, if not all, possible test languages.

Do you think that would catch all the cases?  It's a heck of a lot more 
feasable, it would work in every example I can think of (except maybe 
Befunge), and it seems flexible enough to deal with languages we haven't 
thought up yet.  So I guess I'm seriously proposing this.

Looking forward to your opinion,

Mark