Re: [PEAR-QA] How does jenkins determine which dir / file to run tests from?

Hi Markus:

> looking at these three packages, there are three variants of how the
> tests are executed:
> 
> Cache_Lite: phpunit tests/
> File_Bittorrent: phpunit Tests/AllTests.php
> File_Bittorrent2: phpunit tests/AllTests.php

Let me answer a question that yours touches on, but isn't exactly your
question.

There are several ways to call PHPUnit style tests.  The choice used
depends on how the test files are set up.

1)  The preferred way to do things is have all test files with names
ending in "Test.php" and calling PHPUnit like this:
	phpunit tests

If the test files are not named *Test.php, an AllTests.php script is
needed.  The way to execute the script depends on how the AllTests.php
file is configured.

2)  If the file has the standard "define('PHPUnit_MAIN_METHOD')", eg
http://svn.php.net/viewvc/pear/packages/File_Archive/trunk/tests/AllTests.php?revision=320595&view=markup
it is necessary to invoke "php" to run it because using "phpunit" will
cause the suite to run twice, which can lead to problems (such as
errors from constants being redefined).  So call them like this:
    php tests/AllTests.php

3)  If it has the enhanced "$call_main / define('PHPUnit_MAIN_METHOD')", eg
http://svn.php.net/viewvc/pear/packages/Structures_Graph/trunk/tests/AllTests.php?revision=321538&view=markup
it is possible to call it via "php" or "phpunit".
    phpunit tests/AllTests.php
    OR
    php tests/AllTests.php


Also, some packages have only .phpt tests, but they also provide an
AllTests.php script providing the developers the option to run the tests
using PHPUnit.  Please don't.  This can mean the tests get run twice in
some situations.  Just invoke the tests like this:
    pear run-tests -r tests


Now that's clarified, let me touch on error reporting levels.  In 5.3,
E_DEPRECATED is included in E_ALL but E_STRICT is not.  But in 5.4,
E_ALL includes both.  So if your the package in question is not E_STRICT
and E_DEPRECATED compliant, you need to add the following argument to
the command:  "-d error_reporting=22527".  So, for example:
    php -d error_reporting=22527 tests/AllTests.php

Here's a rundown on the reporting level choices that work in both 5.3
and 5.4:

Show       Show
Deprecated Strict  Value
---------- ------  -----
N          N       22527
N          Y       24575
Y          N       30719
Y          Y       32767


Thanks,

--Dan

-- 
 T H E   A N A L Y S I S   A N D   S O L U T I O N S   C O M P A N Y
            data intensive web and database programming
                http://www.AnalysisAndSolutions.com/
 4015 7th Ave #4, Brooklyn NY 11232  v: 718-854-0335 f: 718-854-0409