(usagi-users 03854) Re: Help needed.

On Fri, 18 May 2007 10:20:19 +0530, Channa said:

> Its for "determining API design correctness".
> After the test is complete i would like eto know whether the API
> implementation is correct or not.

That's 2 different requirements - design and implementation ;)

Should an implementation that correctly implements a broken/misdesigned API
pass the test, or be failed? (For examples of what I mean, read section 17
of RFC342, which lists all the bugfixes in the RFC2292 API).  I'm sure that
there's still some silly things in the 3542 version that need fixing...

There's the TAHI test suite - but even *that* needs to be double-checked,
as you need to be sure that the tests are themselves not buggy (a given
test case could be flagged as "fail" because the case is buggy - or even
worse, a case passes because the case fails to actually stress-test the
point in question..)

And in 25 years in this business, I've seen *far* too many products that manage
to pass conformance tests, but totally fail to work in the real world, or
fail the testing but still work just fine in reality....

Tricky business, this testing. ;)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Exmh version 2.5 07/13/2001

iD8DBQFGTbVXcC3lWbTT17ARAlAZAJ9DDmgkwcmEkgAF3NCdhzIKAU+OIACg5CFy
Sv09FSLupSzJt4p8MnIzTFY=
=Ryx0
-----END PGP SIGNATURE-----