RE: ExtendedTextTestRunner

Philippe FREMY <[email protected]> Tue, 28 Jan 2003 14:27:26 +0100
Newsgroups gmane.comp.lib.cppunit.devel
Message-ID <191CBBF91062D411855D00D0B76F1C300117A762@PROXIMA>
> From: Baptiste Lepilleur [mailto:[email protected]]
>
> CppUnit 1.9.10 already provides similar functionalities 
> through the generic TestRunner class.

Hi Baptiste.

If you are coding a new version of CppUnit, I have some suggestion for it.
There are a few things that I dislike in CppUnit, which has lead me to code
my own testrunner and assertions. I attach the files so that you can look at
it.

The differences with CppUnit are explained in the headers, which I copy
here:

/**
 * I wrote this class because I was dissatisfied with the text runner of
 * CppUnit. This works with CppUnit 1.8.0 I don't think it work with any
other version.
 *
 * The goal of PhilTestRunner was for me to make the output of the testing
 * more readable and useful. It has the following characteristics:
 *
 * - it prints the name of the suite being run and separate two suites with
a
 *   blank line, so that you know better in which suite you are.
 *
 * - it prints the name of the test being run, so that when you have some
kind
 *   of debug output, you know to which test it belongs.
 *
 * - the failures are reported in detail while the suite still runs,
directly
 *   in the offending test. This differs from CppUnit which reports them at
 *   the end. This is very useful if you have test suites that take a long
 *   time. By placing the most offending test in the first position of the
 *   suite, you don't need to wait for the whole test to finish before
 *   spotting and understanding the failure.
 *
 * - on windows, the failures are also printed in the Debug window, using
 *   OutputDebugString. This output is in the Visual C++ debug format, so
 *   that pressing F4 (even during test run) sets your cursor directly on
the
 *   right line, in the right file. Very useful!
 *
 * - it prints a short but nice summary at the end, with the number of test
 *   run, failed, and the percentage.
 *   
 * The output of a run looks like this:

--- suite TestFoo
        + testOne
        + testTwo
d:\software\cppunit-1.8.0\testunitexample\testfoo.cpp(40) : Failure!
Expected: 7, but was: 2.

        + testThree
d:\software\cppunit-1.8.0\testunitexample\testfoo.cpp(45) : Failure!
Uh uh, this test failed

--- suite TestMyClass

 *
 * All you need to use in this class is the PhilTestRunner. Ignore the other
 * classes, there are only used within PhilTestRunner. I package
 * everything in one header file and one source file to make distribution
more
 * convenient.
 *
 * To use the PhilTestRunner, just do like with any other runner :

	PhilTestRunner runner;
	runner.addTest(
CppUnit::TestFactoryRegistry::getRegistry().makeTest() );
	return runner.run( "", false, true, true );

 *
 *
 * Send any comment, bug, suggestion, patch to [email protected]
 *
 */


and

/**
 * This is a set of replacements for the default asserts of CppUnit. This
 * works with CppUnit 1.8.0 . Thank to the architecture of CppUnit, it
 * integrates seemlessly.
 *
 * The differences are:
 *
 * - my asserts are shorter and easier to type because I do not prepend them
 *   by CPPUNIT_ and because I use mainly non capitalised letters. This does
 *   not respect the convention that macros should be in capitalised letters
 *   but it saves me a lot of pain, so it is worthwile.
 *
 * - the order of equality assertion is reversed, because it is more
intuitive
 *   for me to have the actual value before the expected one. So
 *   CPPUNIT_ASSERT_EQUALS( 1, a ) becomes checkEquals( a, 1 )
 *
 * - I display the name of the expression being tested inside the failure,
not
 *   just the value that did not match.
 *
 * - for int equality asserts, I display the int value in decimal and 
 *   hexadecimal
 *
 * - I support native string equality tests, which avoids the following
 *   drawbacks of CppUnit:
 *
 *   	1. With Visual C++, CPPUNIT_ASSERT_EQUAL( std::string("a"), "a" )
 *   	won't compile because the it does not know which template to use.
 *
 *   	2. If you do a 
 *
	char * s1 = "abcd";
	char s2[10];
	strcpy( s2, s1);
	CPPUNIT_ASSERT_EQUAL( s1, (char *) s2 );

 * 		CppUnit will report a failure with the unhelpful message : 
 *
 * 		"Expected: abcd, but was: abcd." 
 *
 * 		This is because CppUnit has unintuitively compared the two
pointers
 * 		instead of comparing the strings. My functions know how to
cast a
 * 		(char *) to a string
 *
 *
 * - when displaying an error with a string, my strings are in quote, so
that
 *   you can see precicely where it begins and finish. No more
 *   your-strings-are-not-equal-but-you-do-not-know-why-because-you-have-
 *   not-seen-this-naughty-space-at-the-end-of-one-of-them.
 *
 * - for string equality asserts, I support case sensitive and insenstive
 *   check.
 *
 * - I support also a assertion to test whether a string contains another
one,
 *   with adjustable case sensitivity
 *
 * - if you define QT_DLL, I support tests on QString too
 *
 * 
 * This list of macros available are :
 * check( assertion ) for bool
 * checkEquals( actual, expected ) for int, long, double, std::string,
QString and char *
 * checkNotEquals( actual, notExpected ) for the same types as checkEquals
 * checkDeltaEquals( doubleValue1, doubleValue2 ) for double
 * checkIEquals( s1, s2 )  for std::string, QString, (char *)
 * checkContains( s1, s2 ) for std::string, QString, (char *)
 * checkIContains( s1, s2 )  for std::string, QString, (char *)
 *
 * Send any bug, comment, suggestion, patch to [email protected]
 *
 */ 

It would probably make sense to add this to CppUnit. I am probably not the
only one who have felt these problems. I know my colleagues prefer to use my
addition than the standard CppUnit way.

I attach the files since they are quite short.

Could all people here provide feedback ?

	regards,

	Philippe
PhilTestRunner.h (application/octet-stream, 4.4 KB)
#ifndef PHIL_TEST_OUTPUTTER_H
#define PHIL_TEST_OUTPUTTER_H

#include <cppunit/ui/text/TestRunner.h>
#include <cppunit/TestListener.h>
#include <cppunit/TextTestProgressListener.h>
#include <cppunit/TestResultCollector.h>
#include <cppunit/Test.h>
#include <cppunit/Outputter.h>


/**
 * I wrote this class because I was dissatisfied with the text runner of
 * CppUnit. In fact, except for the concept of running test suites, I am
 * dissatisfied with CppUnit. I find the overall architecture very complex,
 * and still not as helpful as I wish it were. This works with CppUnit 1.8.0
 * I don't think it work with any other version.
 *
 * The goal of PhilTestRunner was for me to make the output of the testing
 * more readable and useful. It has the following characteristics:
 *
 * - it prints the name of the suite being run and separate two suites with a
 *   blank line, so that you know better in which suite you are.
 *
 * - it prints the name of the test being run, so that when you have some kind
 *   of debug output, you know to which test it belongs.
 *
 * - the failures are reported in detail while the suite still runs, directly
 *   in the offending test. This differs from CppUnit which reports them at
 *   the end. This is very useful if you have test suites that take a long
 *   time. By placing the most offending test in the first position of the
 *   suite, you don't need to wait for the whole test to finish before
 *   spotting and understanding the failure.
 *
 * - on windows, the failures are also printed in the Debug window, using
 *   OutputDebugString. This output is in the Visual C++ debug format, so
 *   that pressing F4 (even during test run) sets your cursor directly on the
 *   right line, in the right file. Very useful!
 *
 * - it prints a short but nice summary at the end, with the number of test
 *   run, failed, and the percentage.
 *   
 * The output of a run looks like this:

--- suite TestFoo
        + testOne
        + testTwo
d:\software\cppunit-1.8.0\testunitexample\testfoo.cpp(40) : Failure!
Expected: 7, but was: 2.

        + testThree
d:\software\cppunit-1.8.0\testunitexample\testfoo.cpp(45) : Failure!
Uh uh, this test failed

--- suite TestMyClass

 *
 * All you need to use in this class is the PhilTestRunner. Ignore the other
 * classes, there are only used within PhilTestRunner. I package
 * everything in one header file and one source file to make distribution more
 * convenient.
 *
 * To use the PhilTestRunner, just do like with any other runner :

	PhilTestRunner runner;
	runner.addTest( CppUnit::TestFactoryRegistry::getRegistry().makeTest() );
	return runner.run( "", false, true, true );

 *
 *
 * Send any comment, bug, suggestion, patch to [email protected]
 *
 */


/**
 * This class belongs to CppUnit. I hope they will include it!
 */
namespace CppUnit {

	class ExtendedTestRunner : public TextUi::TestRunner {

	public:
	ExtendedTestRunner( Outputter *outputter =NULL, TestListener * progressListener =NULL) 
		: TextUi::TestRunner( outputter), m_progressListener( progressListener)  {
		if (m_progressListener == NULL) {
			m_progressListener = new TextTestProgressListener();
		}
	}

  virtual bool runTest( Test *test,
	  bool doPrintProgress );
  
  virtual void setTestProgressListener( TestListener * listener) {
	  delete m_progressListener;
	  m_progressListener = listener;
  }

  virtual void setTestResultCollector( TestResultCollector * result) {
	  delete m_result;
	  m_result = result;
  }
  
	
protected:
	TestListener * m_progressListener;
};

}; // CppUnit


class PhilTestRunner : public CppUnit::ExtendedTestRunner {

public:
	/** I use all the Phil customisers in this runner */
	PhilTestRunner();
	
};



// -------------------------------------------------------------------
// You don't need the class after this line, they are used inside the
// PhilTestRunner.
//


class PhilProgressListener : public CppUnit::TestListener {

public:
	virtual ~PhilProgressListener () {}

	virtual void startTest (CppUnit::Test *test);
	virtual void addFailure (const CppUnit::TestFailure &failure);
	virtual void endTest (CppUnit::Test *test);

protected:
	std::string _currentSuite;
};



class PhilOutputter : public CppUnit::Outputter {
public:
	PhilOutputter( CppUnit::TestResultCollector * resultCollector )
		: m_resultCollector( resultCollector ) {}

	virtual ~PhilOutputter() {}
	void write();

protected:
	CppUnit::TestResultCollector * m_resultCollector;
};





#endif // PHIL_TEST_OUTPUTTER_H
PhilAsserts.h (application/octet-stream, 6.9 KB)
#ifndef PHIL_ASSERT_H
#define PHIL_ASSERT_H

#include <cppunit/Portability.h>
#include <cppunit/Exception.h>

#include <cppunit/TestAssert.h>
#include <string>

/**
 * This is a set of replacements for the default asserts of CppUnit. This
 * works with CppUnit 1.8.0 . Thank to the architecture of CppUnit, it
 * integrates seemlessly.
 *
 * The differences are:
 *
 * - my asserts are shorter and easier to type because I do not prepend them
 *   by CPPUNIT_ and because I use mainly non capitalised letters. This does
 *   not respect the convention that macros should be in capitalised letters
 *   but it saves me a lot of pain, so it is worthwile.
 *
 * - the order of equality assertion is reversed, because it is more intuitive
 *   for me to have the actual value before the expected one. So
 *   CPPUNIT_ASSERT_EQUALS( 1, a ) becomes checkEquals( a, 1 )
 *
 * - I display the name of the expression being tested inside the failure, not
 *   just the value that did not match.
 *
 * - for int equality asserts, I display the int value in decimal and 
 *   hexadecimal
 *
 * - I support native string equality tests, which avoids the following
 *   drawbacks of CppUnit:
 *
 *   	1. With Visual C++, CPPUNIT_ASSERT_EQUAL( std::string("a"), "a" )
 *   	won't compile because the it does not know which template to use.
 *
 *   	2. If you do a 
 *
	char * s1 = "abcd";
	char s2[10];
	strcpy( s2, s1);
	CPPUNIT_ASSERT_EQUAL( s1, (char *) s2 );

 * 		CppUnit will report a failure with the unhelpful message : 
 *
 * 		"Expected: abcd, but was: abcd." 
 *
 * 		This is because CppUnit has unintuitively compared the two pointers
 * 		instead of comparing the strings. My functions know how to cast a
 * 		(char *) to a string
 *
 *
 * - when displaying an error with a string, my strings are in quote, so that
 *   you can see precicely where it begins and finish. No more
 *   your-strings-are-not-equal-but-you-do-not-know-why-because-you-have-
 *   not-seen-this-naughty-space-at-the-end-of-one-of-them.
 *
 * - for string equality asserts, I support case sensitive and insenstive
 *   check.
 *
 * - I support also a assertion to test whether a string contains another one,
 *   with adjustable case sensitivity
 *
 * - if you define QT_DLL, I support tests on QString too
 *
 * 
 * This list of macros available are :
 * check( assertion ) for bool
 * checkEquals( actual, expected ) for int, long, double, std::string, QString and char *
 * checkNotEquals( actual, notExpected ) for the same types as checkEquals
 * checkDeltaEquals( doubleValue1, doubleValue2 ) for double
 * checkIEquals( s1, s2 )  for std::string, QString, (char *)
 * checkContains( s1, s2 ) for std::string, QString, (char *)
 * checkIContains( s1, s2 )  for std::string, QString, (char *)
 *
 * Send any bug, comment, suggestion, patch to [email protected]
 *
 */ 

void philAssert( std::string actualExpr, bool assertion, 
				long lineNumber, std::string fileName );


// numbers
void philAssertEquals( std::string actualExpr, long actual, long expected,
						long lineNumber, std::string fileName );
void philAssertDeltaEquals( std::string actualExpr, long actual, long expected, long delta,
						long lineNumber, std::string fileName );
void philAssertNotEquals( std::string actualExpr, long actual, long notExpected,
						long lineNumber, std::string fileName );

// std::string

void philAssertEquals( std::string actualExpr, std::string actual, std::string expected,
						long lineNumber, std::string fileName );
// same as previous, but case insensitive
void philAssertIEquals( std::string actualExpr, std::string actual, std::string expected,
						long lineNumber, std::string fileName );
void philAssertNotEquals( std::string actualExpr, std::string actual, std::string expected,
						long lineNumber, std::string fileName );
void philAssertContains( std::string expr, std::string s, std::string sub,
						long lineNumber, std::string fileName );
void philAssertIContains( std::string expr, std::string s, std::string sub,
						long lineNumber, std::string fileName );


#ifdef QT_DLL
#include <qstring.h>

/* It is necessary to provide QString functions, because VC++ does not cast
 * QString to char * or std::string
 */

inline void philAssertEquals( std::string actualExpr, QString actual, QString expected,
					  long lineNumber, std::string fileName );
inline void philAssertIEquals( std::string actualExpr, QString actual, QString expected,
						long lineNumber, std::string fileName );

inline void philAssertNotEquals( std::string actualExpr, QString actual, QString expected,
						long lineNumber, std::string fileName );

inline void philAssertContains( std::string actualExpr, QString string, QString substring,
						long lineNumber, std::string fileName );

inline void philAssertIContains( std::string actualExpr, QString string, QString substring, 
						 long lineNumber, std::string fileName );

inline void philAssertEquals( std::string actualExpr, QString actual, QString expected,
					  long lineNumber, std::string fileName ) {
	philAssertEquals( actualExpr, std::string( actual.latin1() ), 
		std::string( expected.latin1() ), lineNumber, fileName );
}

inline void philAssertIEquals( std::string actualExpr, QString actual, QString expected,
						long lineNumber, std::string fileName ) {
	philAssertIEquals( actualExpr, std::string(actual.latin1()), std::string( expected.latin1()), lineNumber, fileName );
}

inline void philAssertNotEquals( std::string actualExpr, QString actual, QString expected,
						long lineNumber, std::string fileName ) {
	philAssertNotEquals( actualExpr, std::string( actual.latin1()), std::string(expected.latin1()), lineNumber, fileName );
}

inline void philAssertContains( std::string actualExpr, QString string, QString substring,
						long lineNumber, std::string fileName ) {
	philAssertContains( actualExpr, std::string(string.latin1()), std::string(substring.latin1()), lineNumber, fileName );
}

inline void philAssertIContains( std::string actualExpr, QString string, QString substring, long lineNumber, std::string fileName ) {
	philAssertIContains( actualExpr, std::string(string.latin1()), std::string(substring.latin1()), lineNumber, fileName );
}

#endif // Qt


#define check( actual )	(philAssert( (#actual), (actual),__LINE__,__FILE__))

#define checkEquals( actual, expected ) \
  (philAssertEquals ( (#actual), (actual),\
    (expected),__LINE__,__FILE__))

#define checkIEquals( actual, expected ) \
  (philAssertIEquals ( (#actual), (actual),\
    (expected),__LINE__,__FILE__))

#define checkNotEquals( actual, notExpected ) \
  (philAssertNotEquals ( (#actual), (actual),\
    (notExpected),__LINE__,__FILE__))

#define checkContains( s, sub ) \
  (philAssertContains ( (#s), (s),\
    (sub),__LINE__,__FILE__))

#define checkIContains( s, sub ) \
  (philAssertIContains ( (#s), (s),\
    (sub),__LINE__,__FILE__))

#define checkDeltaEquals( actual, expected, delta ) \
  (philAssertDeltaEquals ( (#actual), (actual),\
    (expected), (delta),__LINE__,__FILE__))
	


#endif  // PHIL_ASSERT_H
PhilTestRunner.cpp (application/octet-stream, 2.6 KB) - not displayed
PhilAsserts.cpp (application/octet-stream, 3.7 KB) - not displayed