Re: Test results in CI builds

James,

No time to review any of this :-(

But great work! Having a proper CI would be really nice.

-CHB

On Mar 20, 2017, at 12:03 AM, James Wettenhall <[email protected]>
wrote:

Hi,

I'm not sure if most people in this group subscribe to Phoenix GitHub
notifications.  Maybe they could be automatically forwarded to this Google
Group with a Mail hook:
https://help.github.com/articles/managing-notifications-for-pushes-to-a-repository/
?

But for those who don't get them, I just submitted a (Work In Progress)
pull request which I'd be interested in some feedback on:
https://github.com/wxWidgets/Phoenix/pull/248

Basically, I would like to see Phoenix tests running in CI (continuous
integration) services like AppVeyor and Travis CI.  I couldn't get the
tests from the official branch (@ 4fc5f9ed1047c243a7fe0c2a6489064c7dc78a4b
<https://github.com/wxWidgets/Phoenix/commit/4fc5f9ed1047c243a7fe0c2a6489064c7dc78a4b>)
to run smoothly, which I think is due to race conditions in the tearDown of
one test sometimes preventing subsequent tests from running.

After making a few changes in my fork of Phoenix (
https://github.com/wettenhj/Phoenix/tree/appveyor), I got the tests running
on the AppVeyor (Windows) CI service (just using Python 2.7.13 for now), as
shown here: https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7

However some of the changes I made to get the tests running probably went a
bit too far, e.g. preventing certain timers from running if there is no
MainLoop.  I still believe that it might be necessary to avoid some timers
when running tests, but testing for wx.GetMainLoop() is not the right way
to do it, because a timer could be legitimately started in an application's
OnInit method before the MainLoop has started.  A suggestion for an
alternative approach is here:
https://github.com/wxWidgets/Phoenix/pull/248#issuecomment-287644362

I think the priority for improving tests should be to avoid exceptions
which can cause the interpreter to crash or hang partway through the tests.
 (Feel free to correct me if that has never been an issue for other people
running "python build.py test".)

But it's also nice to avoid ugly tracebacks when enabling stderr
(--extra_pytest="--capture=no" or --extra_pytest="-s").  I've had multiple
attempts at fixing one such issue (tearDown of the UltimateListCtrl tests
somehow terminating the App too early, triggering wx.PyNoAppErrors or "No
wx.App created yet") which I still haven't fully resolved, as you can see
in the following lines of the test output:
https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5691

wx\lib\agw\ultimatelistctrl.py:10876: in __init__ wx.CallAfter(self.Layout)

...

def CallAfter(callableObj, *args, **kw):
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5692> ...
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5705> assert
callable(callableObj), "callableObj is not callable"
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5706> app =
wx.GetApp()
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5707>>
assert app is not None, 'No wx.App created yet'
<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5708>E
AssertionError: No wx.App created yet

<https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5711>
I've enabled coverage (some output here:
https://ci.appveyor.com/project/wettenhj/phoenix/build/1.0.7#L5346), but
keep in mind that this is only one platform.  Codecov.io is good at merging
test coverage from multiple platforms.

Cheers,
James

-- 
You received this message because you are subscribed to the Google Groups
"wxPython-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to [email protected].
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups "wxPython-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
For more options, visit https://groups.google.com/d/optout.