Re: CGOS source on github

--===============8565224217122728083==
Content-Type: multipart/alternative; boundary="000000000000fa73ab05b97d93d6"

--000000000000fa73ab05b97d93d6
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Jan 22, 2021 at 8:08 AM R=C3=A9mi Coulom <[email protected]> wr=
ote:

> You are right that non-determinism and bot blind spots are a source of
> problems with Elo ratings. I add randomness to the openings, but it is
> still difficult to avoid repeating some patterns. I have just noticed tha=
t
> the two wins of CrazyStone-81-15po against LZ_286_e6e2_p400 were caused b=
y
> very similar ladders in the opening:
> http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733333.sgf
> http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733301.sgf
> Such a huge blind spot in such a strong engine is likely to cause rating
> compression.
> R=C3=A9mi
>

I agree, ladders are definitely the other most noticeable way that Elo
model assumptions may be broken, since pure-zero bots have a hard time with
them, and can easily cause difference(A,B) + difference(B,C) to be very
inconsistent with difference(A,C). If some of A,B,C always handle ladders
very well and some are blind to them, then you are right that probably no
amount of opening randomization can smooth it out.

--000000000000fa73ab05b97d93d6
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Fri, Jan 22, 2021 at 8:08 AM R=C3=A9mi=
 Coulom &lt;<a href=3D"mailto:[email protected]">[email protected]<=
/a>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote class=3D"gma=
il_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,2=
04,204);padding-left:1ex"><div dir=3D"ltr"><div>You are right that non-dete=
rminism and bot blind spots are a source of problems with Elo ratings. I ad=
d randomness to the openings, but it is still difficult to avoid repeating =
some patterns. I have just noticed that the two wins of CrazyStone-81-15po =
against=20
LZ_286_e6e2_p400 were caused by very similar ladders in the opening:</div><=
div><a href=3D"http://www.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/=
733333.sgf" target=3D"_blank">http://www.yss-aya.com/cgos/viewer.cgi?19x19/=
SGF/2021/01/21/733333.sgf</a></div><div><a href=3D"http://www.yss-aya.com/c=
gos/viewer.cgi?19x19/SGF/2021/01/21/733301.sgf" target=3D"_blank">http://ww=
w.yss-aya.com/cgos/viewer.cgi?19x19/SGF/2021/01/21/733301.sgf</a></div><div=
>Such a huge blind spot in such a strong engine is likely to cause rating c=
ompression.</div><div>R=C3=A9mi</div></div></blockquote><div><br></div><div=
>I agree, ladders are definitely the other most noticeable way that Elo mod=
el assumptions may be broken, since pure-zero bots have a hard time with th=
em, and can easily cause difference(A,B)=C2=A0+ difference(B,C) to be very =
inconsistent with difference(A,C). If some of A,B,C always handle ladders v=
ery well and some are blind to them, then you are=C2=A0right that probably =
no amount of opening randomization can smooth it out.</div><div><br></div><=
/div></div>

--000000000000fa73ab05b97d93d6--

--===============8565224217122728083==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Computer-go mailing list
[email protected]
http://computer-go.org/mailman/listinfo/computer-go

--===============8565224217122728083==--