Re: Personal proxy not honoring buffer size settings
Jonathan Dixon <[email protected]> Tue, 18 Feb 2014 19:07:04 -0500
| Newsgroups | gmane.comp.distributed-net.rc5 |
|---|---|
| Message-ID | <CAFYs+cqF9x4SqTPxZ571MxG6KvefKycS0g5Qh60=7RZtgA6Wpg@mail.gmail.com> |
--===============3902388722756057171== Content-Type: multipart/alternative; boundary=001a11c25dea30d46504f2b72b34 --001a11c25dea30d46504f2b72b34 Content-Type: text/plain; charset=ISO-8859-1 It helps with flaky internet connections too. I started using a proxy server because my cable internet kept dropping excessive numbers of TCP packets(which has since been resolved). Basically, the problem was when my client would try to update, it would download some packets, but the connection would be dropped before it flushed leaving me with an ever growing out-buffer. Every time it updated, it would download a few more and be interrupted before it could flush. It wasn't a big deal with CPU clients because they only processed/updated a few packets at a time. If they were interrupted, they would just flush them the next time around. It was when I started using the CUDA client that pushed me to needing the proxy. The proxy let me queue up large numbers of packets for a proper sized fetch and would keep retrying until it flushed all the completed packets. Jonathan Dixon > > Message: 5 > Date: Tue, 18 Feb 2014 21:00:30 +0200 > From: Toomas Aas <[email protected]> > To: rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected] > Subject: [RC5] Personal proxy not honoring buffer size settings > Message-ID: <20140218210030.66512sh1afbwjxs0-2RFepEojUI22287D0lG6h9k44Pud+NeK@public.gmane.org> > Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; > format="flowed" > > Hello! > > I've been running a personal proxy for almost as long as I've been > involved in distributed.net, which is since 1997. The current version > that I have is build 347 running on FreeBSD 7.4. > > For last several years I've been concentrating mostly on OGR. My > clients are configured to work on OGR whenever possible and only fall > back to RC5 in case of network problems, so I have rarely done any RC5 > blocks. Accordingly, I had limits in proxyper.ini set like this: > > [rc5-72] > minkeysready=5 > maxkeysready=400 > maxkeysdone=5 > > [ogrng] > minkeysready=34 > maxkeysready=2400 > maxkeysdone=34 > > Given my client settings, this resulted in proxy connecting to > fullserver whenever it had 34 OGR blocks in its out-buffer, which was > typically once in an hour or so. > > Now that we are out of OGR work, all my clients started working on > RC5, so I adjusted the buffer size settings accordingly: > > [rc5-72] > minkeysready=100 > maxkeysready=7200 > maxkeysdone=100 > > However, this does not have the desired effect. Instead of waiting > until 100 blocks have been gathered in the rc5 out-buffer, the proxy > connects to upstream server every couple of minutes when it has only > few RC5 blocks to return: > > $ grep "Last uplink" console20140218.log > > 2014-02-18 00:09:40,Status: rc5-72: Last uplink sent 2 and received 0 > 2014-02-18 00:09:40,Status: ogrng: Last uplink sent 0 and received 0 > 2014-02-18 00:12:59,Status: rc5-72: Last uplink sent 5 and received 0 > 2014-02-18 00:12:59,Status: ogrng: Last uplink sent 0 and received 0 > 2014-02-18 00:15:40,Status: rc5-72: Last uplink sent 2 and received 0 > 2014-02-18 00:15:40,Status: ogrng: Last uplink sent 0 and received 0 > 2014-02-18 00:18:42,Status: rc5-72: Last uplink sent 6 and received 0 > 2014-02-18 00:18:42,Status: ogrng: Last uplink sent 0 and received 0 > ... etc > > I modified the configuration of my personal proxy this morning and > thought that maybe it needs some time to "settle in", but 12 hours > later the situation seems unchanged. I don't want to DoS the upstream > keyservers with my constant connection attempts, but I don't know how > to stop this. I tried adding 'expertmode=1' to the [rc5-72] section, > but this doesn't help. > > -- > Toomas Aas > participant #32,484 > > > ------------------------------ > > Message: 6 > Date: Tue, 18 Feb 2014 22:16:19 +0200 > From: Toomas Aas <[email protected]> > To: rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected] > Subject: Re: [RC5] Personal proxy not honoring buffer size settings > Message-ID: <20140218221619.331959e5e6ii7bgo-2RFepEojUI22287D0lG6h9k44Pud+NeK@public.gmane.org> > Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; > format="flowed" > > *facepalm* > > I adjusted the limits in [rc5-72} section, but didn't touch the > [ogrng] section: > > > > > [ogrng] > > minkeysready=34 > > maxkeysready=2400 > > maxkeysdone=34 > > As it still had minkeysready=34, the proxy desperately tried to get > more OGR blocks. I changed minkeysready for ogrng to 0, looks like now > it's working as expected. > > -- > Toomas Aas > participant #32,484 > > > ------------------------------ > > Message: 7 > Date: Tue, 18 Feb 2014 15:25:28 -0800 > From: "Kevin G. McCoy" <[email protected]> > To: "D.net Discussion" <rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]> > Subject: Re: [RC5] Personal proxy not honoring buffer size settings > Message-ID: <[email protected]> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 2/18/2014 12:16 PM, Toomas Aas wrote: > > *facepalm* > > > > I adjusted the limits in [rc5-72} section, but didn't touch the > > [ogrng] section: > > > >> > >> [ogrng] > >> minkeysready=34 > >> maxkeysready=2400 > >> maxkeysdone=34 > > > > As it still had minkeysready=34, the proxy desperately tried to get > > more OGR blocks. I changed minkeysready for ogrng to 0, looks like now > > it's working as expected. > > > > I have never understood the need for the dnet proxy server. Is it for > people with dial-up or some sort of pay-by-the-minute Internet access? > It seems like more trouble than it is worth - plus it messes up logging > so that you can't check individual workstation stats. > > I have over 50 computers working on OGR and never saw the need for a > proxy. Is there something I am missing? > > -- > > Best regards, > > Kevin G. McCoy > > > > ------------------------------ > > _______________________________________________ > rc5 mailing list > rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected] > http://lists.distributed.net/mailman/listinfo/rc5 > > > End of rc5 Digest, Vol 103, Issue 2 > *********************************** > --001a11c25dea30d46504f2b72b34 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote"><div= >It helps with flaky internet connections too.=A0 I started using a proxy s= erver because my cable internet kept dropping excessive numbers of TCP pack= ets(which has since been resolved).=A0 </div> <div>=A0</div><div>Basically, the problem was when my client would try to u= pdate, it would download some packets, but the connection would be dropped = before it flushed leaving me with an ever growing out-buffer.=A0 Every time= it updated, it would download a few more and be interrupted before it coul= d flush.=A0 It wasn't a big deal with CPU clients because they only pro= cessed/updated a few packets at a time.=A0 If=A0they were interrupted,=A0th= ey would just flush them the next time around.=A0 It was when I started usi= ng the CUDA client that pushed me to needing the proxy.=A0 The proxy let me= queue up large numbers of packets for a proper sized fetch and would keep = retrying until it flushed all the completed packets.=A0 </div> <div>=A0</div><div>Jonathan Dixon=A0</div><blockquote style=3D"margin:0px 0= px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-lef= t-width:1px;border-left-style:solid" class=3D"gmail_quote"> <br> Message: 5<br> Date: Tue, 18 Feb 2014 21:00:30 +0200<br> From: Toomas Aas <<a href=3D"mailto:[email protected]">toomas.aas= @raad.tartu.ee</a>><br> To: <a href=3D"mailto:rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]">rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]<= /a><br> Subject: [RC5] Personal proxy not honoring buffer size settings<br> Message-ID: <<a href=3D"mailto:[email protected]= aad.tartu.ee">20140218210030.66512sh1afbwjxs0-2RFepEojUI22287D0lG6h9k44Pud+NeK@public.gmane.org</a>>= <br> Content-Type: text/plain; charset=3DISO-8859-1; DelSp=3D"Yes";<br= > =A0 =A0 =A0 =A0 format=3D"flowed"<br> <br> Hello!<br> <br> I've been running a personal proxy for almost as long as I've been<= br> involved in <a href=3D"http://distributed.net" target=3D"_blank">distribute= d.net</a>, which is since 1997. The current version<br> that I have is build 347 running on FreeBSD 7.4.<br> <br> For last several years I've been concentrating mostly on OGR. My<br> clients are configured to work on OGR whenever possible and only fall<br> back to RC5 in case of network problems, so I have rarely done any RC5<br> blocks. Accordingly, I had limits in proxyper.ini set like this:<br> <br> [rc5-72]<br> minkeysready=3D5<br> maxkeysready=3D400<br> maxkeysdone=3D5<br> <br> [ogrng]<br> minkeysready=3D34<br> maxkeysready=3D2400<br> maxkeysdone=3D34<br> <br> Given my client settings, this resulted in proxy connecting to<br> fullserver whenever it had 34 OGR blocks in its out-buffer, which was<br> typically once in an hour or so.<br> <br> Now that we are out of OGR work, all my clients started working on<br> RC5, so I adjusted the buffer size settings accordingly:<br> <br> [rc5-72]<br> minkeysready=3D100<br> maxkeysready=3D7200<br> maxkeysdone=3D100<br> <br> However, this does not have the desired effect. Instead of waiting<br> until 100 blocks have been gathered in the rc5 out-buffer, the proxy<br> connects to upstream server every couple of minutes when it has only<br> few RC5 blocks to return:<br> <br> $ grep "Last uplink" console20140218.log<br> <br> 2014-02-18 00:09:40,Status: rc5-72: Last uplink sent 2 and received 0<br> 2014-02-18 00:09:40,Status: ogrng: Last uplink sent 0 and received 0<br> 2014-02-18 00:12:59,Status: rc5-72: Last uplink sent 5 and received 0<br> 2014-02-18 00:12:59,Status: ogrng: Last uplink sent 0 and received 0<br> 2014-02-18 00:15:40,Status: rc5-72: Last uplink sent 2 and received 0<br> 2014-02-18 00:15:40,Status: ogrng: Last uplink sent 0 and received 0<br> 2014-02-18 00:18:42,Status: rc5-72: Last uplink sent 6 and received 0<br> 2014-02-18 00:18:42,Status: ogrng: Last uplink sent 0 and received 0<br> ... etc<br> <br> I modified the configuration of my personal proxy this morning and<br> thought that maybe it needs some time to "settle in", but 12 hour= s<br> later the situation seems unchanged. I don't want to DoS the upstream<b= r> keyservers with my constant connection attempts, but I don't know how<b= r> to stop this. I tried adding 'expertmode=3D1' to the [rc5-72] secti= on,<br> but this doesn't help.<br> <br> --<br> Toomas Aas<br> participant #32,484<br> <br> <br> ------------------------------<br> <br> Message: 6<br> Date: Tue, 18 Feb 2014 22:16:19 +0200<br> From: Toomas Aas <<a href=3D"mailto:[email protected]">toomas.aas= @raad.tartu.ee</a>><br> To: <a href=3D"mailto:rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]">rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]<= /a><br> Subject: Re: [RC5] Personal proxy not honoring buffer size settings<br> Message-ID: <<a href=3D"mailto:[email protected]= aad.tartu.ee">20140218221619.331959e5e6ii7bgo-2RFepEojUI22287D0lG6h9k44Pud+NeK@public.gmane.org</a>>= <br> Content-Type: text/plain; charset=3DISO-8859-1; DelSp=3D"Yes";<br= > =A0 =A0 =A0 =A0 format=3D"flowed"<br> <br> *facepalm*<br> <br> I adjusted the limits in [rc5-72} section, but didn't touch the<br> [ogrng] section:<br> <br> ><br> > [ogrng]<br> > minkeysready=3D34<br> > maxkeysready=3D2400<br> > maxkeysdone=3D34<br> <br> As it still had minkeysready=3D34, the proxy desperately tried to get<br> more OGR blocks. I changed minkeysready for ogrng to 0, looks like now<br> it's working as expected.<br> <br> --<br> Toomas Aas<br> participant #32,484<br> <br> <br> ------------------------------<br> <br> Message: 7<br> Date: Tue, 18 Feb 2014 15:25:28 -0800<br> From: "Kevin G. McCoy" <<a href=3D"mailto:[email protected]"= >[email protected]</a>><br> To: "D.net Discussion" <<a href=3D"mailto:rc5-Ra3b/[email protected]= d.net">rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]</a>><br> Subject: Re: [RC5] Personal proxy not honoring buffer size settings<br> Message-ID: <<a href=3D"mailto:[email protected]">5303EBE8.20= [email protected]</a>><br> Content-Type: text/plain; charset=3DISO-8859-1; format=3Dflowed<br> <br> On 2/18/2014 12:16 PM, Toomas Aas wrote:<br> > *facepalm*<br> ><br> > I adjusted the limits in [rc5-72} section, but didn't touch the<br= > > [ogrng] section:<br> ><br> >><br> >> [ogrng]<br> >> minkeysready=3D34<br> >> maxkeysready=3D2400<br> >> maxkeysdone=3D34<br> ><br> > As it still had minkeysready=3D34, the proxy desperately tried to get<= br> > more OGR blocks. I changed minkeysready for ogrng to 0, looks like now= <br> > it's working as expected.<br> ><br> <br> I have never understood the need for the dnet proxy server. =A0Is it for<br= > people with dial-up or some sort of pay-by-the-minute Internet access?<br> It seems like more trouble than it is worth - plus it messes up logging<br> so that you can't check individual workstation stats.<br> <br> I have over 50 computers working on OGR and never saw the need for a<br> proxy. =A0Is there something I am missing?<br> <br> --<br> <br> Best regards,<br> <br> Kevin G. McCoy<br> <br> <br> <br> ------------------------------<br> <br> _______________________________________________<br> rc5 mailing list<br> <a href=3D"mailto:rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]">rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected]</a><= br> <a href=3D"http://lists.distributed.net/mailman/listinfo/rc5" target=3D"_bl= ank">http://lists.distributed.net/mailman/listinfo/rc5</a><br> <br> <br> End of rc5 Digest, Vol 103, Issue 2<br> ***********************************<br> </blockquote></div><br></div></div> --001a11c25dea30d46504f2b72b34-- --===============3902388722756057171== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ rc5 mailing list rc5-Ra3b/QYEcJ3d140v2zMXi0fjHoOT/h/[email protected] http://lists.distributed.net/mailman/listinfo/rc5 --===============3902388722756057171==--