Re: Training an AlphaGo Zero-like algorithm with limited hardware on 7x7 boards
Igor Polyakov <[email protected]> Mon, 27 Jan 2020 00:22:32 +0800
| Newsgroups | gmane.games.devel.go |
|---|---|
| Message-ID | <CAPMj7maMGUJtmPO-iK4OERJsS_N4MEMZfDgkXmCaZxT2ZKdMMw@mail.gmail.com> |
--===============0115847383301116244== Content-Type: multipart/alternative; boundary="000000000000cd5f5a059d0d6663" --000000000000cd5f5a059d0d6663 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I trained using David Wu's code for a few months on 9x9 only and it's been superhuman after a few months. I'm not sure if anyone's interested, but I can release my network to the world. It's around the strength of KataGo, but only on 9x9. I could do a final test before releasing it into the wild On Mon, Jan 27, 2020, 00:17 R=C3=A9mi Coulom <[email protected]> wrote: > Yes, using komi would help a lot. Still, I feel that something else must > be wrong, because winning 100% of the games as Black without komi should = be > very easy on 7x7. > > I have not written anything about what I did with Crazy Stone. But my > experiments and ideas were really very similar to what David Wu did: > https://blog.janestreet.com/accelerating-self-play-learning-in-go/ > > To clarify what I wrote in my previous message: "strong from scratch in a > single day" was for 7x7. I like testing new ideas with small networks on > small boards, because training is very fast, and what works on small boar= ds > with small networks usually also works on large boards with big networks. > > R=C3=A9mi > > On Sun, Jan 26, 2020 at 12:30 AM cody2007 <[email protected]> wrote= : > >> Hi R=C3=A9mi, >> >> Thanks for your comments! I am not using any komi and had not given much >> thought to it. Although, I suppose by having black win most games, I'm >> depriving the network of its only learning signal. I will have to try wi= th >> an appropriately set komi next... >> >> >When I started to develop the Zero version of Crazy Stone, I spend a lo= t >> of time optimizing my method on a single (V100) GPU >> Any chance you've written about it somewhere? I'd be interested to learn >> more but wasn't able to find anything on the Crazy Stone website. >> >> Thanks, >> Cody >> >> =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 Original= Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80=90 >> On Saturday, January 25, 2020 5:49 PM, R=C3=A9mi Coulom <remi.coulom@gma= il.com> >> wrote: >> >> Hi, >> >> Thanks for sharing your experiments. >> >> Your match results are strange. Did you use a komi? You should use a kom= i >> of 9: >> https://senseis.xmp.net/?7x7 >> >> The final strength of your network looks surprisingly weak. When I >> started to develop the Zero version of Crazy Stone, I spend a lot of tim= e >> optimizing my method on a single (V100) GPU. I could train a strong netw= ork >> from scratch in a single day. Using a wrong komi might have hurt you. Al= so, >> on such a small board, it is not so easy to make sure that the self-play >> games have enough variety. You'd have to find many balanced random initi= al >> positions in order to avoid replicating the same game again and again. >> >> R=C3=A9mi >> >> >> _______________________________________________ > Computer-go mailing list > [email protected] > http://computer-go.org/mailman/listinfo/computer-go > --000000000000cd5f5a059d0d6663 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">I trained using David Wu's code for a few months on 9= x9 only and it's been superhuman after a few months.<div dir=3D"auto"><= br></div><div dir=3D"auto">I'm not sure if anyone's interested, but= I can release my network to the world. It's around the strength of Kat= aGo, but only on 9x9. I could do a final test before releasing it into the = wild</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gm= ail_attr">On Mon, Jan 27, 2020, 00:17 R=C3=A9mi Coulom <<a href=3D"mailt= o:[email protected]">[email protected]</a>> wrote:<br></div><blo= ckquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #c= cc solid;padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><div>Yes, usin= g komi would help a lot. Still, I feel that something else must be wrong, b= ecause winning 100% of the games as Black without komi should be very easy = on 7x7.</div><div><br></div><div>I have not written anything about what I d= id with Crazy Stone. But my experiments and ideas were really very similar = to what David Wu did:</div><div><a href=3D"https://blog.janestreet.com/acce= lerating-self-play-learning-in-go/" target=3D"_blank" rel=3D"noreferrer">ht= tps://blog.janestreet.com/accelerating-self-play-learning-in-go/</a></div><= div><br></div><div>To clarify what I wrote in my previous message: "st= rong from scratch in a single day" was for 7x7. I like testing new ide= as with small networks on small boards, because training is very fast, and = what works on small boards with small networks usually also works on large = boards with big networks.</div><div><br></div><div>R=C3=A9mi<br></div></div= ><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Su= n, Jan 26, 2020 at 12:30 AM cody2007 <<a href=3D"mailto:cody2007@protonm= ail.com" target=3D"_blank" rel=3D"noreferrer">[email protected]</a>&g= t; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0p= x 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>H= i R=C3=A9mi,<br></div><div><br></div><div>Thanks for your comments! I am no= t using any komi and had not given much thought to it. Although, I suppose = by having black win most games, I'm depriving the network of its only l= earning signal. I will have to try with an appropriately set komi next...<b= r></div><div><br></div><div>>When I started to develop the Zero version = of Crazy Stone, I spend a lot of time optimizing my method on a single (V10= 0) GPU<br></div><div>Any chance you've written about it somewhere? I= 9;d be interested to learn more but wasn't able to find anything on the= Crazy Stone website.<br></div><div><br></div><div>Thanks,<br></div><div>Co= dy<br></div><div><br></div><div>=E2=80=90=E2=80=90=E2=80=90=E2=80=90=E2=80= =90=E2=80=90=E2=80=90 Original Message =E2=80=90=E2=80=90=E2=80=90=E2=80=90= =E2=80=90=E2=80=90=E2=80=90<br></div><div> On Saturday, January 25, 2020 5:= 49 PM, R=C3=A9mi Coulom <<a href=3D"mailto:[email protected]" target= =3D"_blank" rel=3D"noreferrer">[email protected]</a>> wrote:<br></di= v><div> <br></div><blockquote type=3D"cite"><div dir=3D"ltr"><div>Hi,<br></= div><div><br></div><div>Thanks for sharing your experiments.<br></div><div>= <br></div><div>Your match results are strange. Did you use a komi? You shou= ld use a komi of 9:<br></div><div><a href=3D"https://senseis.xmp.net/?7x7" = target=3D"_blank" rel=3D"noreferrer">https://senseis.xmp.net/?7x7</a><br></= div><div><br></div><div>The final strength of your network looks surprising= ly weak. When I started to develop the Zero version of Crazy Stone, I spend= a lot of time optimizing my method on a single (V100) GPU. I could train a= strong network from scratch in a single day. Using a wrong komi might have= hurt you. Also, on such a small board, it is not so easy to make sure that= the self-play games have enough variety. You'd have to find many balan= ced random initial positions in order to avoid replicating the same game ag= ain and again.<br></div><div><br></div><div>R=C3=A9mi<br></div></div></bloc= kquote><div><br></div></blockquote></div></div> _______________________________________________<br> Computer-go mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank" rel=3D"nor= eferrer">[email protected]</a><br> <a href=3D"http://computer-go.org/mailman/listinfo/computer-go" rel=3D"nore= ferrer noreferrer" target=3D"_blank">http://computer-go.org/mailman/listinf= o/computer-go</a><br> </blockquote></div> --000000000000cd5f5a059d0d6663-- --===============0115847383301116244== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Computer-go mailing list [email protected] http://computer-go.org/mailman/listinfo/computer-go --===============0115847383301116244==--