Re: HOWTO: Writing Perl6 Tests (was: Project Start: Section 1)

Richard Nuttall wrote:

> I agree with that. take the example of reverse (array) in this thread.
> Really, the testing should have a number of other tests to be complete,
> including thorough testing of boundary conditions.
> 
> e.g. - tests of reverse on
> 0. undef
> 1. Empty list
> 2. (0..Inf) - Error ?
> 3. Mixed data types
> 4. Complex data types
> 5. Big (long) lists
> 6. Big (individual items) lists
> 7. Array References
> 8. Things that should raise a compile-time error
> 9. Things that should raise a run-time error
> 
> This gets pretty boring in main documentation.
> Writing a complete test suite really also needs reasonable knowledge
> of how the internals are written in order to understand the kinds of
> tests that are likely to provoke errors. (More thoughts on this if 
> requested).

This get back to defining the focus/level of the testing that we want to 
achieve. Some of these items may make sense for paranoid testing; but 
not as part of a comprehensive test suite.

Consider item 0. Do we need to test C<reverse(undef)>? The answer is 
probably "no": conversion of undef to an empty lsit is a property of the 
  list context in the caller. We tested that as part of our compehensive 
set of arglist evaluations. The reverse function never sees the undef, 
so we don't need to test it.

Item 1 may be worth testing; but then we see that what we realy have is 
a parameterised test: one control path with multiple datasets. One 
parametarised test would code most of the other items.

The most interesting case is Item:2. This is a question that a user 
might want to ask. The question is a language-issue, not implementation. 
  Derivative thoughts ask about lazy lists in general (is the reverse 
still lazy?); and hence onto tied lists. Perhaps there is an interface 
to list-like objects: perhaps we need to document/test that.

In sammary: yes, lists of tests can get boring; and yes, we would want 
to construct documentation that hide most of the tedious details. I 
treat it as a challenge: to create a unified infrastructure for 
documentation and testing, which is neither too tedious for the user, 
nor too vague for the tester; but which has projections (viewpoints) 
that give the desired level of boredom. Perhaps its not possible, but we 
should at least try. Perhaps we can only go as far as creating 
executable examples as tests. But if we can get that far, then most of 
the infrastructure for a more detailed (boring) document will be in place.


Dave.