Re: Regressions getting more common

On Wednesday, October 14 2020, Simon Marchi wrote:

> Then, we can look at having try jobs work again.  I never really liked
> submitting try jobs through the buildbot command line tool, I always
> found there was to little feedback (did my upload work? what jobs did I
> initiate?).  I would receive test results by email and would have a ahrd
> time figuring out which result was for which patch.
>
> One idea would be to re-use the now abandonned Gerrit instance for this.
> Those who have use Gerrit will probably agree that it integrates quite
> well with CI.  After pushing your patch for review, you can ask the CI
> to test a given patch (often by setting a label on it).  The CI posts a
> comment on the patch with a link to the build, so you can follow it if
> you want.  Once it has ran, the CI posts comments on the patch to say
> that version N of the patch has this result, again with a link to the
> build.

There is another idea, which is simpler to implement, which is to make
the Buildbot monitor user branches with a special naming convention,
like user/sergiodj/try-foobar or some such.  When a new branch showed
up, the Buildbot would then build it, test things, and then send the
results via email to the author of the commit (bonus points if it can
also send an email to the committer).  One of the problems to be solved
(not particular to this solution, but to the Buildbot in general) is to
decide which upstream commit the Buildbot will compare the results
against.  If you're considering running one/two builds per day, this
problem becomes easier to solve.  I had a hard time solving it because I
was running builds for every commit, as you know.

This was next on my TODO list for the Buildbot (along with stop building
separate commits, and start grouping them), but I could not find the
time to implement it.

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
https://sergiodj.net/