Re: Adding binutils to the GNU Toolchain buildbot on sourceware

Hi Mark,

I see Frank has put together a worker, but I just wanted to reply to 
some points.

On 4/25/22 19:20, Mark Wielaard wrote:
> Hi Luis,
> 
> On Mon, Apr 25, 2022 at 01:16:44PM +0100, Luis Machado wrote:
>> On 4/25/22 11:43, Frank Ch. Eigler wrote:
>>>> As a general comment, I think we should have a single buildbot entry for the
>>>> whole of binutils-gdb.
>>>
>>> Perhaps!
>>>
>>>> Given changes to bfd and opcodes can affect gdb, why not build gdb alongside
>>>> the other tools? You don't need to run the gdb testsuite, which should make
>>>> things much more deterministic.
>>>
>>> But this is not an argument for building gdb as a part of testing binutils.
>>> This is an argument for building & testing gdb if binutils changes.
>>> That is: an additional buildbot job
>>
>> An additional buildbot job for gdb/gdbserver would be fine. It's just that a
>> single build for everything is simpler, in my opinion.
>>
>> Is the idea to revive the old buildbot we had for GDB, but for binutils?
> 
> We could provide some of the builds that the old buildbot did for
> GDB. But the new builder is not GDB specific and I hope we can learn
> from the old gdb buildbot.
> 
> The problem with the old gdb buildbot is that it did too much and had
> flaky test results. This caused people to not care, think the reports
> were annoying, broken builds sometimes only got reported after hours or
> even days.
> 
> I think we should try to keep builds/checks under 10 minutes, that the
> checks should be for things that the maintainers think should always
> be green. So that you get a report about something important breaking
> within 10 minutes while you still know what you did. Another reason to
> keep build/check times short is so you can test multiple commits per
> hour.

I agree with the quick build idea. That's why I suggested only making 
sure gdb/gdbserver/sim builds properly. That's reasonably reliable.

Unfortunately gdb's testsuite is not too reliable. It's been improved 
over the years, but still gives quite a bit of non-deterministic results 
based on distro version/compiler version etc. So I'd leave those out in 
favor of just making sure things build properly.

If this effort can accomodate the gdb/gdbserver/sim builds, that would 
be great. I don't mean to broaden the scope of the initial proposed 
effort, but I think it is beneficial to make sure everything is building OK.

> 
> If there are extra resources then we can also add builders that take
> longer and/or run testsuites on arches/distros that are known
> broken. But those would then probably not sent out reports but you
> would use them to see the current status of some setup that might not
> be 100% green. And that would probably mean adding more
> hardware/workers.

That makes sense, but gdb-wise I think that can wait.

> 
> I assume that knowing that gdb and gdbserver still build OK, without
> running any tests might be important to the gdb maintainers. And that
> just a build of gdb and gdbserver will take < 10 minutes on most
> setups.
> 
> If so lets just pick one or more of the workers which seem like stable
> distros/arches that should always build gdb:
> https://builder.sourceware.org/buildbot/#/workers
> (don't pick sourceware, which is special, debian-arm64, debian-armhf,
> debian-i386 or fedora-ppc64 which are too slow)
> 
> Provide a file list (directories) of files in the binutils-gdb.git
> repo that should trigger a build.
> 
> A configure and make line that does a quick build for just
> gdb/gdbserver which should always build.

I think the following will do:

./configure --enable-targets=all; make all-gdb all-gdbserver

or

./configure --enable-targets=all --disable-sim; make all-gdb all-gdbserver

gdbserver, unlike gdb, will only build natively, hence why it would 
benefit from being built everywhere.

> 
> If the is a make check-something that can be executed quickly, < 5
> minutes runtime, and that should always be green please include
> it. But please exclude anything that takes too long, isn't known
> all-green or contains flaky tests.

Some tests should always work, no matter what. We could hand-pick some 
tests from the GDB testsuite that we consider critical. I don't think we 
should focus on that at this point though.

> 
> And the mailinglist to which to report any failing commits.
> gdb-patches I assume?

gdb-testers should be fine.

> 
> Cheers,
> 
> Mark
>