Re: mr.roboto and Jenkins false positives

Am 02.06.2014 21:15, schrieb Sean Upton:
> It looks like there was some kind of temporary condition on whatever
> build slave is used for Plone 4.3 / Python 2.6 that caused these
> issues:
> 
>   http://jenkins.plone.org/job/plone-4.3-python-2.6/2109/testReport/
> 
> These were unrelated to changes preceding the build, and were also
> evident in the previouslf
> http://jenkins.plone.org/job/plone-4.3-python-2.6/2108/
> 
> These were also cleared in the next build --
> http://jenkins.plone.org/job/plone-4.3-python-2.6/2110/
>
> This brings two questions:
> 
> 1. What should be the protocol for handling the false-positive
> automated comments mister-roboto places on github commits (e.g. [1])?
> Should they merely be responded to, or deleted?

The person who broke the build is responsible to fix it, no matter if
the test failures are related to the commit itself. Usually it is enough
to just re-run the build. Unfortunately we still do not have our LDAP
connected to Jenkins, so the only way to trigger a build is to do a commit.

Our policy regarding unstable tests is that we try to make them work
reliably and if that does not work out just delete them. An unstable
test is worse than no test at all.

> 2. Is there any way mr.roboto can be modified to omit blame for a
> broken build if the previous build (prior to a commit) was broken for
> the same reasons?

Yes. Though, committing to a broken build is a violation of our CI best
practices and should be reported in any case:

https://buildoutcoredev.readthedocs.org/en/latest/continous-integration.html?highlight=jenkins

A broken build should be fixed as soon as possible and committing on a
broken build makes things more complicated to the person who is trying
to fix the build. We could certainly make the mr.roboto messages more
sophisticated. Though, Ramon (who mostly wrote mr.roboto) is currently
pretty busy with other things, so I don't think this will happen soon.

We are planing a CI sprint this year to work on our CI infrastructure
and maybe also improve mr.roboto. Everybody who is willing to help is
more than welcome to join!

Cheers,
Timo

------------------------------------------------------------------------------
Learn Graph Databases - Download FREE O'Reilly Book
"Graph Databases" is the definitive new guide to graph databases and their 
applications. Written by three acclaimed leaders in the field, 
this first edition is now available. Download your free book today!
http://p.sf.net/sfu/NeoTech