Re: revisiting CI setup

--===============7451300024578658720==
Content-Type: multipart/alternative; boundary="000000000000630baf0599d8146d"

--000000000000630baf0599d8146d
Content-Type: text/plain; charset="UTF-8"

I'd say since appveyor windows builds have sped up so much we can decrease
the split per py type from /4 to maybe /2 or maybe /1
I think the max runtime per build is 1hr?

On Mon, Dec 16, 2019 at 2:26 PM Mats Wichmann <[email protected]> wrote:

>
> time passes... so time to ask some questions again.
>
>
> The travis build does two code-coverage runs (2.7 and 3.6), then seven
> regular runs (2.7, 3.5, 3.6, 3.7, 3.8-dev, pypy2 and pypy3.
>
> The appveyor build, despite a recent caching hack that sped it up a lot,
> is still much slower, and thus is asked to do less work, but has a more
> complex scenario: it needs to test itself on three different appveyor
> images, each provisioned with a different Visual Studio version, so what
> it runs is: 2.7 on VS2017 image, 3.5 on VS2015 image, 3.6 on VS2017
> image (this is also a coverage run), 3.7 on VS2019 image.
>
> We're having some test problems due to evolution of Windows images - in
> particular (there are github issues on this), clang tests do not run
> properly, because before it always found the mingw clang, but in the
> VS2019 image that is no longer installed by default, and the setup
> doesn't work with a "native" clang - basically this exposed an untested
> issue, rather than breaking something that was working.
>
> 3.8 is fully released now.  only the Linux CI (travis) is running the
> tests under 3.8, and it's using "3.8-dev" as opposed to an officially
> released version of 3.8. Although it's early days yet, there ought to be
> a 3.9-dev (last I checked, appveyor hadn't enabled that choice, but that
> was a while back)
>
> what sets of these are actually important for checking that a PR does
> not introduce testing regressions?
>
> I'm unconvinced the pypy tests add any value in this context. Maybe we
> could move them to the buildbot setup instead? In particular, pypy3 has
> never gained the performance of pypy2, and doesn't seem to be in active
> development, and the performance is a killer for our use of it.
>
> We should be able to stop testing 2.7 once develoment flips to "4.0",
> since it won't carry the compatibility promise any longer.
>
> Is there any other way to rearrange/improve this, including the
> mentioned buildbot - things which aren't deemed critical to run on every
> PR but which still would be useful to see the progress of could move to
> buildbot....
>
> _______________________________________________
> Scons-dev mailing list
> [email protected]
> https://pairlist2.pair.net/mailman/listinfo/scons-dev
>

--000000000000630baf0599d8146d
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>I&#39;d say since appveyor windows builds have sped u=
p so much we can decrease the split per py type from /4 to maybe /2 or mayb=
e /1 <br></div><div>I think the max runtime per build is 1hr?<br></div></di=
v><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On M=
on, Dec 16, 2019 at 2:26 PM Mats Wichmann &lt;<a href=3D"mailto:mats@wichma=
nn.us">[email protected]</a>&gt; wrote:<br></div><blockquote class=3D"gmail_=
quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,=
204);padding-left:1ex"><br>
time passes... so time to ask some questions again.<br>
<br>
<br>
The travis build does two code-coverage runs (2.7 and 3.6), then seven <br>
regular runs (2.7, 3.5, 3.6, 3.7, 3.8-dev, pypy2 and pypy3.<br>
<br>
The appveyor build, despite a recent caching hack that sped it up a lot, <b=
r>
is still much slower, and thus is asked to do less work, but has a more <br=
>
complex scenario: it needs to test itself on three different appveyor <br>
images, each provisioned with a different Visual Studio version, so what <b=
r>
it runs is: 2.7 on VS2017 image, 3.5 on VS2015 image, 3.6 on VS2017 <br>
image (this is also a coverage run), 3.7 on VS2019 image.<br>
<br>
We&#39;re having some test problems due to evolution of Windows images - in=
 <br>
particular (there are github issues on this), clang tests do not run <br>
properly, because before it always found the mingw clang, but in the <br>
VS2019 image that is no longer installed by default, and the setup <br>
doesn&#39;t work with a &quot;native&quot; clang - basically this exposed a=
n untested <br>
issue, rather than breaking something that was working.<br>
<br>
3.8 is fully released now.=C2=A0 only the Linux CI (travis) is running the =
<br>
tests under 3.8, and it&#39;s using &quot;3.8-dev&quot; as opposed to an of=
ficially <br>
released version of 3.8. Although it&#39;s early days yet, there ought to b=
e <br>
a 3.9-dev (last I checked, appveyor hadn&#39;t enabled that choice, but tha=
t <br>
was a while back)<br>
<br>
what sets of these are actually important for checking that a PR does <br>
not introduce testing regressions?<br>
<br>
I&#39;m unconvinced the pypy tests add any value in this context. Maybe we =
<br>
could move them to the buildbot setup instead? In particular, pypy3 has <br=
>
never gained the performance of pypy2, and doesn&#39;t seem to be in active=
 <br>
development, and the performance is a killer for our use of it.<br>
<br>
We should be able to stop testing 2.7 once develoment flips to &quot;4.0&qu=
ot;, <br>
since it won&#39;t carry the compatibility promise any longer.<br>
<br>
Is there any other way to rearrange/improve this, including the <br>
mentioned buildbot - things which aren&#39;t deemed critical to run on ever=
y <br>
PR but which still would be useful to see the progress of could move to <br=
>
buildbot....<br>
<br>
_______________________________________________<br>
Scons-dev mailing list<br>
<a href=3D"mailto:[email protected]" target=3D"_blank">[email protected]=
g</a><br>
<a href=3D"https://pairlist2.pair.net/mailman/listinfo/scons-dev" rel=3D"no=
referrer" target=3D"_blank">https://pairlist2.pair.net/mailman/listinfo/sco=
ns-dev</a><br>
</blockquote></div>

--000000000000630baf0599d8146d--

--===============7451300024578658720==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Scons-dev mailing list
[email protected]
https://pairlist2.pair.net/mailman/listinfo/scons-dev

--===============7451300024578658720==--