Re: Testing websites

David Kramer wrote:

> I want to start investigating software to test
> websites.  I found HttpUnit and 
> HtmlUnit.  They seem to work quite differently. Can
> someone with experience 
> with both outline what one is better at than the
> other?

Here's a Web application's architecture and test
options. Our "GUI Layer" splits in two. Most of it
ought to execute on the server's side of the HTTP
link, but some of it (DHTML, JavaScript, CSS, Java
Applets, etc.) executes on the client's side:


Logic--Representation--GUI--Server--HTTP--Browser--DOM

Tests on each layer mock the layer to its right. The
Representation Layer thinks its tests are the
server-side GUI Layer. The 

GUI Layer creates XHTML, thinking a Web server will
transmit it. The Web server broadcasts HTTP to clients
that it thinks are Web browsers. And Web browsers,
hosting the client side of the GUI Layer, create
"Document Object Model" representations with hopes
that they will paint into display hardware and users
will look at them.

At each layer boundary, tests intercept I/O by mocking
its protocol. XHTML Tests sample XHTML directly from
the functions 

that create it. HTTP Tests emulate your browser, and
stream the XHTML out of an HTTP server. DOM Tests
enslave a real Web browser, and command it to navigate
to your sample pages. Then test manipulate the objects
that DHTML and JavaScript use.

To test-first a Web project, spend a little up-front
design effort pushing all hard logic down below the
Representation Layer. Most of that code will run
headless, permitting the GUI Layer modules to shrink
down as thin as possible. If they are 

so thin they only need Authoring, you are almost done.
Write a few XHTML Tests, HTTP Tests, and DOM Tests,
just to prove you can, and to kick-start those
modules. Most Web site development consists of
authoring and reviewing content.

The more complex your appearance and client-side
behavior, the more tests you will need to the right.
To illustrate simple solutions for hard problems, this
post tests some easy things in hard ways. Sometimes we
will use DOM Tests where more XHTML Tests are needed.

  Minimize System Diversity
Web apps require a mix of programming languages,
scripts, systems, and platforms. Make sure a
development workstation can hold an image or miniature
version of the entire project, from database to
browser. This enables programmers to experiment with
any part.

Write most tests in the language most convenient for
the testee. Preferably the same language, but
sometimes that's impossible. When you edit, and hit
the One Test Button, the module-level tests evaluate
in the common language. But, at need, test cases may
shell to command-lines that run more tests in a
different language.

For example, consider XSLT that converts XML into
XHTML. You could write tests in XSLT, but if most of
your logic is in Java then you will have trouble
changing your mindset, and the test fixtures, between
the two languages. So, write tests in Java language,
and use an XSLT interpreter to produce the XHTML. Then
test the XHTML using XPath for Parsed Fuzzy Matches to
demonstrate it contains the correct fields. XPath
queries can find target nodes insensitive to their
surrounding details.

  XHTML Tests
Most Web tests need only assert that lower layers
generate correct HTML. The book /Extreme Programming
for Web Projects/, by Doug Wallace, Isobel Raggett,
and Joel Aufgang, provides very good advice to resolve
duplication between the many modules that form a Web
application. Their Prime Directive: Convert all data
into XML, author all HTML markup inside XSLT, and
transform them together to produce each page.

Not all HTML projects are content-rich data-driven
public Web sites. I can only recommend that all output
be XHTML, regardless of its source. WebXP complies
when the XSLT enforces XHTML output:

    <xsl:output method="xml" media-type="text/html"
       standalone="no" omit-xml-declaration="yes"
       encoding="UTF-8"/>

XHTML Tests show XPath seeking expected values inside
XHTML.

WebXP's prime directive, XML for data and XSLT for
HTML, provides a very high level of testing at very
low cost. The act of parsing also tests. The book also
describes a healthy lifecycle for incrementally adding
and reviewing a site's esthetic content (verbiage and
pictures). "At all times a complete working site can
be browsed in entirety without errors." (Does that
sound familiar?)

  Bootstrapping CGI Tests in Perl
Each TFUI development effort begins in a tricky spot.
Your project must learn to use test-first, and not its
wizard or its Web server, typically without support
from your GUI Toolkit's documentation.

This script squeezes two modules together - a
GuiLayer, and its test, called main: 

  #!/usr/bin/env perl -w
  
  use strict;  #  don't leave home without it
  use CGI;  
  use Test::Unit::TestRunner;
  use XML::XPath;
  
  package GuiLayer;
  
    sub processRequest
    {
      my $self = shift;
      my $cgi = shift;
      
      my $payload = $cgi->param("payload");
    
      return $cgi->html(
             $cgi->body("This page says $payload")
             );
    }

  package main;
    use base qw(Test::Unit::TestCase);
    
    sub test_verify_page_contents {
      my $self = shift;
      
    # make CGI think a Web browser called 
    # it to build a page
      
      $ENV{'REQUEST_METHOD'} = 'GET';
      $ENV{'QUERY_STRING'} = 'payload=Daddy+Warbucks';
      my $cgi = CGI->new();
      
    # build the page
      
      my $response = GuiLayer->processRequest($cgi);
      
    #  parse the page's contents
      
      my $xp    = XML::XPath->new(xml => $response);
      my $payload = $xp->find('/html/body/text()');
      
    #  check our payload influenced the page
      
   $self->assert_matches(qr/Daddy Warbucks/,
$payload);
    }

  my $test = Test::Unit::TestRunner->new;
  $test->start('main');  #  Test Collector pattern

The test makes $cgi and GuiLayer think a Web browser
requested its page, obeying an URL such as:

http://localhost/cgi-bin/whatever.pl?payload=Daddy+Warbucks

CGI passes data between a Web server and a script by
packing it all into environmental variables. Perl's
CGI library permits us to spoof these data using by
passing a string or a map into its constructor:
CGI->new( "payload=Daddy+Warbucks" ), or CGI->new({
payload => 'Daddy Warbucks' }). I set the variables
directly, to illustrate the most portable technique.

If your QUERY_STRING must contain a more complex URL,
remember to use URLEncode() to escape any non-ASCII
parameters. I fudged when I represented a space as a
+.

A normal CGI script would call print
GuiLayer->processRequest($cgi). Our test intercepts
that HTML, and parses it with our favorite utility,
XPath. Then assert_matches() checks that it contains
our sample string. That Regular Expression Match,
could have been arbitrarily complex.

The GuiLayer module depends on no Web server, only
$cgi. If GuiLayer used any lower modules, the ones
containing business logic would naturally avoid $cgi.

(Thanks to Tony Byrne, on the TDD mailing list, for
seeding this topic.)

  Temporary Visual HTML Inspections

Write a test that samples some XHTML from your site,
copies it to a file called "test.html", and commands
your Web browser to display that file:

      open(TEMP, '>c:/temp/temp.html');
      print TEMP $response;
      close(TEMP);
      system('start c:/temp/temp.html');

"start" depends on your MS Windows CMD.EXE, so you may
need some other system to view the page, such as
"konqueror".

Wrapping that into a function is left as an exercise
for the reader.

  HTTP Tests
A package that emulates a Web browser, following the
W3C recommendations, can keep all the real browsers
honest. The Java library HttpUnit, by Russell Gold, at
http://www.httpunit.org/, simulates a browser so well
one can develop server-side features using test-first
on the client side. Don't make that a habit.

  Temporary Visual HttpUnit Inspections
Many engineers' sanity depends on reading the daily
comic strip "Dilbert". It illustrates the lives of
highly intelligent engineers working for managers who
owe their position to qualities other than
intelligence.

When we read Dilbert on its Web site, we should not
endure all the popup adds and jiggling baloney around
the payload. We need to test that Dilbert, in
isolation from its Web page, is funny.

This HttpUnit snippet reads the HTML page - without
reading all its extras. Then it locates the actual
Dilbert cartoon, downloads it to your C:\temp folder,
and optionally presents it in Internet Explorer.

The presentation system can easily reconfigure for
other browsers.

The source appears here:

  http://www.c2.com/cgi/wiki?HttpUnitTutorial

Note the // reveal(page) statement. If you de-comment
it, you will see the Dilbert HTML page, without all
its supporting images.

When HttpUnit helps grow your Web site, you can write
a reveal() fixture that takes a WebResponse argument.
It writes the response's HTML contents to a temporary
file, "test.html", and then raises your Web browser.
Repeatedly displaying this page is much more efficient
than repeatedly restarting your server and then
surfing to the target page. Programmers leverage tests
to force down the cost of the feedback required to
write tests.

If your Web page requires extra files, such as style
sheets or images, your test fixture must ensure they
are available in a folder where your Web browser can
find them with the file: protocol. This implies the
local folder keeps them in the same place, relative to
your test.html's location, as the server keeps them
relative to the target page.

  DOM Tests
The original Web browsers displayed motionless text
and graphics. To provide active client-side special
effects, based on user interactions, HTML embedded a
light scripting layer called JavaScript. (The
language's official name is "ECMAScript", but nobody
calls it that.) HTML pages embed this language inside
<script> tags, and inside certain node attributes. A
Web browser publishes a few identifiers, such as
document, to its JavaScript layer. Each identifier
supports members that navigate a Document Object
Model, representing all the nodes in a document.

Some Web browsers publish this model externally,
through a standard Object Request Broker. Some
browsers publish a back-door into their JavaScript
layer, so a program can command the browser to fetch a
Web page, then evaluate a string containing
JavaScript. Tests may exploit these systems to drive
DOM and inspect its behavior. These tests have the
benefit of running a real Web browser, not an
emulator. They have the drawback of running only one
breed of Web browser. All browsers are different, so
projects that rely on dynamic client-side behaviors
must somehow provide Abstract Tests that run on each
browser. As usual, limiting client-side behaviors will
also narrow the risk of bugs.

This Ruby code drives IE to surf to a Web page on my
local server, write on its form, and click its Save
button:

    ie = WIN32OLE.new('InternetExplorer.Application')
ie.navigate('http://127.0.0.1:8080/WikiTranscludeText')

    while ie.busy
      sleep(0)
    end

    until ie.readyState == READYSTATE_COMPLETE
      sleep(0)
    end

  # ie.Visible = true

    formNode = ie.document.forms("files/sample.txt")
    assert_not_nil(formNode)
    inputNode = formNode.namedItem("contents")
    assert_not_nil(inputNode)

    inputNode.setAttribute('value', 'What\'s up Doc?')

    saveButton = formNode.namedItem("Save")
    assert_not_nil(saveButton)
    saveButton.click()

Note the line # ie.Visible = true. To Regulate the
Event Queue from this test, simply command the
controlled Internet Explorer object to paint itself on
your screen. However, this line does not block. As
usual, a GUI Toolkit provides a convenience that both
assists and interferes with our TFUI Principles. After
displaying your Web page, this test continues to run,
and the subsequent test statements will evaluate. If
they command IE to do other things, you will see them
happen in real-time. 

The strict Temporary Interactive Tests definition
says, "All testing shall block until you close the
window." That permits a window to appear in a
predictable state before you click on it. DOM tests
would need a system to block subsequent test
statements until you closed IE.

The strict definition also says, "After the window
closes, other tests shall run, if any." If you don't
mind losing that Principle, you can fix both these
issues terminally:

  ie.Visible = true
  exit(0)

The controlled Internet Explorer object can outlive
the Ruby process that spawned it. Tests cease to drive
DOM, and your can see your Web page's exact condition
at the place you chose to put ie.Visible = true.

The ability to predict a temporarily tested Web page's
state is more important than Incremental Testing.

    Other Platforms
Because DOM enables scripting, any Web browser could
test through DOM. The trick is binding a script from
your tests into the Web browser's internal objects.
Some browsers do not publish their DOMs on public
ORBs. 

The "KDE Desktop Environment", for example, provides a
fine Web browser called Konqueror, and a JavaScript
layer called KJSEmbed, with a command-line interface
called kjscmd. The link from external JavaScript to
Konqueror uses a Qt system called "KParts", but this
only publishes the browser's external methods, not the
DOM inside it.

One drives that DOM by writing external JavaScript
statements that build strings containing JavaScript,
and submitting them to a Konqueror part through the
external method .executeScript(), after the document
loads. (Notice the external script could use any
language that supports KParts, not just JavaScript.
The goal "Minimize System Diversity" can sometimes be
a little specious.)

Thanks to Koos Vriezen for providing this
bootstrapping example:

  #!/usr/bin/env kjscmd

  function Slots() {
    this.pageLoaded = function() {
       println ("loaded");
     
  part.executeScript(
'document.forms[0].elements("text").value="What\'s up
                           Doc?"');
    }
  }
  var mw = new KMainWindow();
  var box = new QVBox( mw );
  var slots_obj = new Slots();
  mw.setCentralWidget(box);
  var part = Factory.createROPart( "text/html", box, 
                       "html widget" );
  part.openURL(
      
"http://127.0.0.1/wiki.rb?edit=WikiTranscludeText" );

  mw.connect(part, "completed()", slots_obj,
"pageLoaded");
  print(part.signals());
  mw.show();
  application.exec();

A test derived from that sample could return 0 for
success or 1 for failure to its environment, and a
test rig could shell to it and collect that value.

Note the statements mw.show() and application.exec().
Koos spot-checked his script using the Temporary
Interactive Test Principle.

    Test Integration
One button must test all aspects of every module, and
one top-level AllTests command must test all modules.
Our survey of techniques here challenges rambunctious
Web projects to minimize their language diversity.
Ideally, only one language should host all tests. Then
our survey showed how different Web layers can require
tests in various languages.

Tests written in Ruby, for example, may shell to
ksjcmd to run a few DOM tests, and collect their
return value to "bubble up" their status:

  result = system("kjscmd myDomTests.js")
  assert_equal(0, result)

The patterns of failures in these "shelled tests", and
their interactions with your editors, will reveal when
and how to improve their Fault Navigation.

When you change a test, get it to fail, change the
source, and get it to pass, each test run must be as
easy as a few keystrokes - hopefully just one. That's
why you must spend a little extra effort to cobble
together scripts and embedded command lines to put all
these different test rigs, on both the server and
client side, under one of your editor's buttons.

Ideally, a project that limits client-side JavaScript
will require only a few DOM tests.

Before introducing our main project, I'm compelled to
showcase a fine example of a test-first project that
exploits advanced client-side JavaScript.

    Remote User Interface
This light GUI Toolkit-written & tested in Perl, Java
& JavaScript by Ran Eliam and hosted at
http://cortext.co.il/-is a 

magnificent example of these solutions:

* TFUI, with tests on both the server and client
* Duplication between the server and client resolved
* Seamless client-side experience, without fetching
pages for each click
* Client-side controls via Model View Controller.

Please read its entire Web site and source, and insert
all of that content here.
  
A product that stretches languages' abilities so far
(Perl to serve Web pages, JavaScript to paint
controls, etc.) is often too fragile. But tests keep
this product robust.

The only downside is the product requires Internet
Explorer 6. In theory, to port to another browser, one
need only get all the JavaScript tests to pass.

    Mini Ruby Wiki
This project uses XHTML tests, DOM tests, and HTTP
tests. Download it thru here...

   http://www.rubygarden.org/ruby?MiniRubyWiki

...and DON'T RUN THOSE TESTS! They are not yet
productized, and they depend on many dependencies
without listing them anywhere. I have not yet built an
MRW package that's safe to install and test. (MRW
itself is quite safe to install and run, though!)

Oh, also, MRW has an Acceptance Test Framework
built-in. This is also preliminary, but I already put
it to use.

Here's an XHTML test. formatWikiTest() is a fixture
that calls formatWiki(), the function that converts
Wiki notation into a fragment of XHTML.

    def test_transcludeText()
        hello_world = "hello world"
        writeFile("files/sample.txt", hello_world)
        transcluder = "!text!ruby -v!files/sample.txt"

        contents = formatWikiTest(transcluder)        
        doc =
Document.new("<BODY>"+contents+"</BODY>")
        e = XPath.first(doc, '/BODY')

        assert_no_match(/#{transcluder}/, e.text)
        textarea = XPath.first(doc, '//TEXTAREA')
        assert_equal(hello_world, textarea.text)
    end

The test wraps the fragment with
"<BODY>"+contents+"</BODY>", to make it complete. Then
it puts the fragment into an XML parser, and uses
XPath to query out the expected part.

That test introduces this Wiki notation:

    !text!ruby -v!files/sample.txt

Bang ! introduces a command. The command !text! tells
the Wiki to read files/sample.txt, and push its
contents into a <textarea> field, embedded as a form
in the middle of the page. The 'ruby -v' part is a
safe placeholder for a command line that could
execute, with that file on its command line, when you
hit a "Test" button. This is just the first of several
features that collude to make MRW's Acceptance Test
Framework more user-friendly than some.



=====
Phlip
  http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces


	
		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - 100MB free storage!
http://promotions.yahoo.com/new_mail 


------------------------ Yahoo! Groups Sponsor --------------------~--> 
$9.95 domain names from Yahoo!. Register anything.
http://us.click.yahoo.com/J8kdrA/y20IAA/yQLSAA/NhFolB/TM
--------------------------------------------------------------------~-> 

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/testdrivendevelopment/

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/