[TFUI] Re: [XP] Hyperactive (or "Brittle"?) Tests

glew wrote:

> To avoid such brittleness, many of my tests are of
> the form:
> NOT
>      assertEquals("<strong><em>so
> what?</em></strong>", htmlResult);
> BUT
>     
>
assertRegexpMatch(".*<strong>.*</strong>.*",htmlResult);
>     
> assertRegexpMatch(".*<em>.*</em>.*",htmlResult);
>      assertEquals("so what?",stripXml(htmlResult));
> but this doesn't detect unbalanced <strong><em>so
> what?</strong></em>

    Fuzzy Matches
When esthetic graphic elements need tests, these might
require "wiggle room". Some tests for "red" should
pass "fuschia" or "magenta". As you develop, if a
hyperactive assertion fails, pick a fix from this
list:

* Remove the assertion
* Make the assertion match the code
* Make the code match the assertion
* Make the assertion fuzzy.

Don't make the assertion match the code, or vice
versa, more than thrice. The pattern of hyperactivity
will reveal how to provide the correct level of
fuzziness. 

The fuzziest test for Red would ensure the Red
component of an RGB value exceeds the Green and Blue
components. This would pass Brown.

Institute fuzziness when you predict you'll need it,
when hyperactive assertions reveal you need it, or
when a Design Smell in the tests, such as duplication,
reveals the need for it:

    assert( color == red or
            color == fuschia or
            color == magenta )  #  that's a smell

A test that the color occupies the purple, red, orange
or yellow side of the color wheel, not green or blue,
would give you the warm fuzzies.

    Regular Expression Matches
Given a string, you might only care it contains the
words "Frisbee" and "Antonio Carlos Jobim". The text
between could localize, or reflect user input, or even
just betray programmer whims.

assert_match(
 /\bFrisbee\b.*\bAntonio\w+Carlos\w+Jobim\b/, result)

Many test rigs come with an assertion for Regular
Expressions on strings. The above checks:

* "Frisbee" comes between word \boundaries, 
* "Antonio Carlos Jobim" comes after, 
  o he similarly has word \boundaries, and
  o he may have any kind of one or more blank spaces
     (\w+) between the elements of his name.

Web e-search engines provide even fuzzier matches;
they skip punctuation between parts of strings,
permits words out of order, etc. They err on the side
of returning pages. If we find ourselves needing that
much fuzziness, we might instead need a better
strategy.

    Parsed Fuzzy Matches
Markup languages often require matches following the
markup language's grammar, so the match forgives the
same variations as the language forgives. HTML
renderers, for example, dismiss all excess blanks, so
we don't need assertions that fail when those
irrelevant blanks change. We will soon learn
forgiveness is a mixed blessing. 

The examples here use HTML because it's widely
familiar, but they apply to any rendering system,
whether it accepts a string containing a language, or
a stream of commands.

A parsed fuzzy match on HTML containing "Frisbee" and
"Antonio Carlos Jobim" could selectively only match
those words in clear text, not inside HTML <tags>. Or
vice-versa.

These match techniques decouple the important part of
output from the transient part. For example, this
Regular Expression Match might ensure a Web browser
(wrapped by the object @ie) has connected with a
server:

   assert_no_match(
            /\<TITLE>Cannot find server\<\/TITLE>/, 
            @ie.htmlNode.innerHTML
                    )

(Browsers provide better ways to test their current
state. That sample illustrates testing the GUI Layer,
using terms introduced so far.)

Note the special escapes \ before the <TITLE> tag to
prevent the regular expression parser from
interpreting the angle brackets, < and >, as commands.
This is not yet a Parsed Fuzzy Match, because the
matcher does not parse HTML-it parses regular
expressions.

To truly match this situation, our tests need a
library that parses HTML. A library that parses XML
could also work, but only if the HTML were perfectly
well-formed XHTML. XML parsers provide a high-level
query system called XPath, to traverse XML nodes,
match patterns, and return result sets. Given XHTML
containing "<HEAD><TITLE>yack yack
yack</TITLE></HEAD>", the XPath address of that
crucial information, "yack yack yack", is
"/HEAD/TITLE". XML parsers use XPath to fetch elements
from inside XML notation and convert them into
high-level objects. So the text member of one of these
objects would contain "yack yack yack".

Let's try to use XPath to detect if Internet Explorer
found its target server:

       doc = Document.new(@ie.htmlNode.innerHTML)
       e = XPath.first(doc, '/HEAD/TITLE')
       assert_not_equal('Cannot find server', e.text)

That would be ideal. But-as we will often discover
when we probe an environment for testability-it does
not work. Some HTML is also well-formed XHTML, but @ie
controls Internet Explorer, whose built-in "Cannot
find server" page is not well-formed XHTML. The
parsing fails on the Document.new() statement.

Our own HTML should be the highest-quality XHTML. That
expands the number of parsers we can use to test it
with parsed fuzzy matches, and expands the number of
ways to operate those parsers. The act of parsing also
tests.

HTML has a strict mode-XHTML. But, in general, we will
always encounter libraries that depend on forgiveness
in ways that interfere with our testage. Different Web
browsers forgive in different ways. We must remain on
a higher level.

Tests that force code to exist will constrain more
things than such low-level tests retrofitted to
existing code could ever hope to constrain. And all
such tests have a significant risk of hyperactivity.
The Test-Driven Development rules permit an easy Undo
over a hard investigation into most unexpected test
failures.

The end user might not care if a given stretch of HTML
said:

<strong><em>so what?</strong></em>
or:
<strong><em>so what?</em></strong>

Honestly, we don't care either. But our tests will,
because the second is strictly well-formed. Web
browsers forgive to enable Web development by hook or
by crook-or by Notepad. That accelerates some kinds of
development, but not ours. Our tests accelerate us,
often by remaining more strict than humanly possible.


=====
Phlip
  http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces


	
		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail 


------------------------ Yahoo! Groups Sponsor --------------------~--> 
Yahoo! Domains - Claim yours for only $14.70
http://us.click.yahoo.com/Z1wmxD/DREIAA/yQLSAA/nhFolB/TM
--------------------------------------------------------------------~-> 

To unsubscribe, email:
TestFirstUserInterfaces-unsubscribe-hHKSG33TihhbjbujkaE4pw@public.gmane.org
 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
     http://groups.yahoo.com/group/TestFirstUserInterfaces/

<*> To unsubscribe from this group, send an email to:
     TestFirstUserInterfaces-unsubscribe-hHKSG33TihhbjbujkaE4pw@public.gmane.org

<*> Your use of Yahoo! Groups is subject to:
     http://docs.yahoo.com/info/terms/