is_a() and the need for improved testing

Hi Folks:

As you may have noticed or heard about, is_a() was modified in 5.3.7.
One issue was adding a warning if the first argument wasn't an object,
though that was removed before 5.3.7 went out the door.  Another issue
is autoload being called if the first argument is a string:
	https://bugs.php.net/bug.php?id=55475
    http://marc.info/?t=131399222400001&r=1&w=2
    http://marc.info/?l=php-internals&m=131415106807521&w=2

Un-reverting the fix for PEAR bug 18656 (done in r313340) (or something
along those lines) PLUS fixing the other uses of is_a() througout pear-core
and getting a new release out the door seems prudent.

Then Alan provided a list of all uses of is_a() in all PEAR packages:
    http://marc.info/?l=php-internals&m=131432741919689&w=2
I've take care of the is_a() usage in DB and XML_RPC and will get releases
out soon.

That aside, recent discussions on internals are touching on the need to
improve testing of PHP as it is being developed and released.  To that
end, I figured I'd take a look at PEAR's automated testing to ensure
PEAR::DB's tests were working properly.  They were not, so I adjusted
them.

Several issues came up during that exploration which I hope can be
addressed.  Improving PEAR's CI infrastructure will benefit PHP in general
by reducing the all too frequent regressions and unexpected changes.

1) The test server should be running the latest PHP code available.  Ideally
the PHP executable would be built from a recent SVN checkout, even better,
all branches that are producing or heading toward releases.

Rather than having to build a whole environemnt to make that happen, perhaps
we can integrate our tests into gcov.php.net, or at least piggy back on that
server.  It also benefits PHP's testing by expanding code coverage and
getting (closer to) real-world use cases.  If y'all find this interesting,
please get in touch with the PHP QA team to see if they're intrigued as
well.  If so, I hope y'all will be able to inspire some people to implement
it.

2) The output needs to be tracked.  New failures/etc need to generate a
notification to the QA team or something.

3) The Cruise Control server is currently down (2011-08-26 15:00 UTC).
It was working on Wednesday night.
http://test.pear.php.net:8080/cruisecontrol/ (linked to by
https://wiki.php.net/pear/qa/ci) produces "Connection refused".

4) The output files generated by pear/ci/phpuc/trunk/tests.php (the
"phpunit results" and "phpt results" found on
http://test.pear.php.net/unit-test-results/) are appended to incrementally
as tests.php is executed.  While viewing the file as it's being
generated can be confusing, more importantly, there is no way to tell
definitively when the test run has been completed.  Can I please have karma
to pear/ci to clean up the process?

5) Nearly all of the PHPUnit tests and some of the phpt tests are choking
on the inability to find required files.  This is due to the files having
been checked out, not installed.  For example, the PEAR_Command_Packaging
class is in PEAR_Command_Packaging/Packaging.php, so when
packages-all/PEAR_Command_Packaging/tests/setup.php.inc calls
"require PEAR/Command/Packaging.php", the needed file is not in place.
This also likely means that some of the tests are really being run against
packages in the server's PEAR installation, not the SVN checkout, due to the
magic of include_path.  A workaround would be to have tests.php to create
symlynks as needed.  I can implement this, if desired.

6) The test suite (http://test.pear.php.net/unit-test-results/deprecated.txt)
indicates several deprecated bits of code still exist in various packages.
In addition, the test suites for PEAR_Command_Packaging and Image_Transform
disable deprecated warnings.  I can remove those settings, if requested.

7) On the deprecation front, the places where
set_magic_quotes_runtime(true) is used, it seems we should check
get_magic_quotes_runtime() and then call stripslashes() on the returned
value if need be.  I can make those and the ereg changes.  It would be nice
if people on the QA team can package QA releases with the mods.

8) There have been some instances, like in 5.3.7, where PHP modified
when autoload is called.  But the test suite doesn't currently catch this
scenario.  The suite needs to include a simple autoload function that prints
out "AUTOLOAD HAS BEEN CALLED TO FIND '$class'".  I can work this into my
tests.php adjustments.

9) While mocking database interactivity is often doable, having actual
database access makes writing tests easier and more accurate.  When it
comes to packages like DB and MDB2, database access is really needed.  If we
are fortunate enough to get on the gcov site, PEAR tests can take advantage
of the database connection there using the MYSQL_TEST_* environemnt
variables.  If not, tests.pear.net needs a MySQL instance.

10) The QA team needs to go through all of the data on the QA wiki to make
sure it's up to date.  For example, I just changed some of the repository
info there from CVS to SVN.

Thanks for your attention,

--Dan

-- 
 T H E   A N A L Y S I S   A N D   S O L U T I O N S   C O M P A N Y
            data intensive web and database programming
                http://www.AnalysisAndSolutions.com/
 4015 7th Ave #4, Brooklyn NY 11232  v: 718-854-0335 f: 718-854-0409