Test results in CI builds

Hi,

I'm not sure if most people in this group subscribe to Phoenix GitHub 
notifications.  Maybe they could be automatically forwarded to this Google 
Group with a Mail hook: 
https://help.github.com/articles/managing-notifications-for-pushes-to-a-repository/ 
?

But for those who don't get them, I just submitted a (Work In Progress) 
pull request which I'd be interested in some feedback on: 
https://github.com/wxWidgets/Phoenix/pull/248

Basically, I would like to see Phoenix tests running in CI (continuous 
integration) services like AppVeyor and Travis CI.  I couldn't get the 
tests from the official branch (@ 4fc5f9ed1047c243a7fe0c2a6489064c7dc78a4b 
<https://github.com/wxWidgets/Phoenix/commit/4fc5f9ed1047c243a7fe0c2a6489064c7dc78a4b>) 
to run smoothly, which I think is due to race conditions in the tearDown of 
one test sometimes preventing subsequent tests from running.

After making a few changes in my fork of Phoenix (
https://github.com/wettenhj/Phoenix/tree/appveyor), I got the tests running 
on the AppVeyor (Windows) CI service (just using Python 2.7.13 for now), as 
shown here: https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7

However some of the changes I made to get the tests running probably went a 
bit too far, e.g. preventing certain timers from running if there is no 
MainLoop.  I still believe that it might be necessary to avoid some timers 
when running tests, but testing for wx.GetMainLoop() is not the right way 
to do it, because a timer could be legitimately started in an application's 
OnInit method before the MainLoop has started.  A suggestion for an 
alternative approach is here: 
https://github.com/wxWidgets/Phoenix/pull/248#issuecomment-287644362

I think the priority for improving tests should be to avoid exceptions 
which can cause the interpreter to crash or hang partway through the tests. 
 (Feel free to correct me if that has never been an issue for other people 
running "python build.py test".)

But it's also nice to avoid ugly tracebacks when enabling stderr 
(--extra_pytest="--capture=no" or --extra_pytest="-s").  I've had multiple 
attempts at fixing one such issue (tearDown of the UltimateListCtrl tests 
somehow terminating the App too early, triggering wx.PyNoAppErrors or "No 
wx.App created yet") which I still haven't fully resolved, as you can see 
in the following lines of the test output: 
https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5691

wx\lib\agw\ultimatelistctrl.py:10876: in __init__ wx.CallAfter(self.Layout)

...

def CallAfter(callableObj, *args, **kw):
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5692> ...
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5705> assert 
callable(callableObj), "callableObj is not callable"
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5706> app = 
wx.GetApp()
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5707>> 
assert app is not None, 'No wx.App created yet'
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5708>E 
AssertionError: No wx.App created yet

<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5711>
I've enabled coverage (some output here: 
https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5346), but 
keep in mind that this is only one platform.  Codecov.io is good at merging 
test coverage from multiple platforms.

Cheers,
James

-- 
You received this message because you are subscribed to the Google Groups "wxPython-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.