Re: DISTCC_POTENTIAL_HOSTS, localhost, and slots

--===============3090691019698025682==
Content-Type: multipart/alternative; boundary=001a11c1f3ec8afd4804f95b94dc

--001a11c1f3ec8afd4804f95b94dc
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Wed, May 14, 2014 at 6:16 AM, Dave Abrahams <[email protected]> wrote:

> DISTCC_POTENTIAL_HOSTS seems like a great idea; as I scale up the build
> farm I=E2=80=99d like to be able to deal with unavailability, but I=E2=80=
=99m having
> trouble getting it to work right.
>
> First, after much fooling around I surmise two undocumented facts(?)
>
> 1. it adds ,lzo,cpp to all the hosts implicitly
>

Yes, that's done in ExportDISTCC_HOSTS function in the pump script (source
file pump.in).


> 2. it only accepts bare host names, not the usual
> @?hostname/<slots>(,<extension>)* syntax
>
Is that correct?
>

Pretty much, yes.  It just passes the variable's value to lsdistcc, and
lsdistcc accepts either bare host names, or a printf format string
containing %d, e.g. "mydistcc%d" or "distcc%d.my.own.org.com", not *host
specifications* as described the distcc man page.

If so, it would be good if these things were documented explicitly
> somewhere.
>

Agreed.  I would be happy to accept a patch to add such documentation.

Another possibility would be to change lsdistcc to handle host
specifications.


> Also, if I want localhost to be one of the hosts, it looks like I need to
> run distccd there.  That doesn=E2=80=99t seem like the most efficient thi=
ng to do,
> since normal local compilation will presumably have fewer layers to work
> through.  Am I mistaken?
>

Well, you only need to run distccd for lsdistcc's initial query (lsdistcc
doesn't have any special casing for localhost).
But any actual compilations done will not go via the distccd on localhost;
distcc itself does have special casing for localhost.

I'd be happy to accept a patch to src/lsdistcc.c to make it special-case
"localhost".


>  Lastly, I=E2=80=99ve only been able to get it to run the default 6 (?) s=
lots per
> host using DISTCC_POTENTIAL_HOSTS.  My .distcc/hosts file looks like this=
,
> and has been working pretty well:
>
> --randomize
> localhost/10
> bruford/10,lzo,cpp
> fripp/10,lzo,cpp
>
>
> Is there any way to get an effect like that using DISTCC_POTENTIAL_HOSTS,
> i.e. 10 slots per host, no local distccd?
>

Not currently.  I imagine that it would require only very minor changes to
the ExportDISTCC_HOSTS function in the pump script (source file pump.in),
though.

Cheers,
   Fergus.

--=20
Fergus Henderson <[email protected]>

"Defend the user, exclude no one, and create magic." -- Eric Schmidt.

--001a11c1f3ec8afd4804f95b94dc
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On W=
ed, May 14, 2014 at 6:16 AM, Dave Abrahams <span dir=3D"ltr">&lt;<a href=3D=
"mailto:[email protected]" target=3D"_blank">[email protected]</a>&gt;</spa=
n> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div dir=3D"ltr">DISTCC_POTENTIAL_HOSTS seems like a great=
 idea; as I scale up the build farm I=E2=80=99d like to be able to deal wit=
h unavailability, but I=E2=80=99m having trouble getting it to work right.<=
div>

<br></div><div>First, after much fooling around I surmise two undocumented =
facts(?)</div>

<div><br></div><div>1. it adds ,lzo,cpp to all the hosts implicitly<br clea=
r=3D"all"></div></div></blockquote><div><br></div><div>Yes, that&#39;s done=
 in=C2=A0ExportDISTCC_HOSTS function in the pump script (source file <a hre=
f=3D"http://pump.in">pump.in</a>).</div>

<div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px =
0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-l=
eft-style:solid;padding-left:1ex"><div dir=3D"ltr"><div><div>2. it only acc=
epts bare host names, not the usual @?hostname/&lt;slots&gt;(,&lt;extension=
&gt;)* syntax=C2=A0</div>

</div></div></blockquote><div><blockquote class=3D"gmail_quote" style=3D"ma=
rgin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,=
204);border-left-style:solid;padding-left:1ex"><div dir=3D"ltr">Is that cor=
rect?</div>

</blockquote><div>=C2=A0</div></div><div>Pretty much, yes. =C2=A0It just pa=
sses the variable&#39;s value to lsdistcc, and lsdistcc accepts either bare=
 host names, or a printf format string containing %d, e.g. &quot;mydistcc%d=
&quot; or &quot;distcc%<a href=3D"http://d.my.own.org.com">d.my.own.org.com=
</a>&quot;, not <i>host specifications</i> as described the distcc man page=
.</div>

<div><br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p=
x 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-lef=
t-style:solid;padding-left:1ex"><div dir=3D"ltr"><div><div>If so, it would =
be good if these things were documented explicitly somewhere.<br>

</div></div></div></blockquote><div><br></div><div>Agreed. =C2=A0I would be=
 happy to accept a patch to add such documentation.</div><div><br></div><di=
v>Another possibility would be to change lsdistcc to handle host specificat=
ions.</div>

<div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px =
0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-l=
eft-style:solid;padding-left:1ex"><div dir=3D"ltr"><div><div></div>

<div>Also, if I want localhost to be one of the hosts, it looks like I need=
 to run distccd there. =C2=A0That doesn=E2=80=99t seem like the most effici=
ent thing to do, since normal local compilation will presumably have fewer =
layers to work through. =C2=A0Am I mistaken?<br>

</div></div></div></blockquote><div><br></div><div>Well, you only need to r=
un distccd for lsdistcc&#39;s initial query (lsdistcc doesn&#39;t have any =
special casing for localhost).</div><div>But any actual compilations done w=
ill not go via the distccd on localhost; distcc itself does have special ca=
sing for localhost.=C2=A0</div>

<div><br></div><div>I&#39;d be happy to accept a patch to src/lsdistcc.c to=
 make it special-case &quot;localhost&quot;.</div><div>=C2=A0<br></div><blo=
ckquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left=
-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;paddi=
ng-left:1ex">

<div dir=3D"ltr"><div>

<div>Lastly, I=E2=80=99ve only been able to get it to run the default 6 (?)=
 slots per host using DISTCC_POTENTIAL_HOSTS. =C2=A0My .distcc/hosts file l=
ooks like this, and has been working pretty well:<br></div><div><br></div>

</div><blockquote style=3D"margin:0px 0px 0px 40px;border:none;padding:0px"=
><div><div>--randomize</div></div><div><div>localhost/10</div></div><div><d=
iv>bruford/10,lzo,cpp</div></div><div><div>fripp/10,lzo,cpp</div></div></bl=
ockquote>



<div><div>=C2=A0</div><div>Is there any way to get an effect like that usin=
g DISTCC_POTENTIAL_HOSTS, i.e. 10 slots per host, no local distccd? =C2=A0<=
/div></div></div></blockquote><div><br></div><div>Not currently. =C2=A0I im=
agine that it would require only very minor changes to the ExportDISTCC_HOS=
TS function in the pump script (source file <a href=3D"http://pump.in">pump=
.in</a>), though.</div>

<div><br></div></div>Cheers,</div><div class=3D"gmail_extra">=C2=A0 =C2=A0F=
ergus.<br clear=3D"all"><div><br></div>-- <br><div>Fergus Henderson &lt;<a =
href=3D"mailto:[email protected]" target=3D"_blank">[email protected]</a>&g=
t;</div><div>
<br>
</div><div><div><span style=3D"font-family:arial,sans-serif;font-size:13px;=
border-collapse:collapse">&quot;Defend the user, exclude no one, and create=
 magic.&quot; -- Eric Schmidt.</span></div></div><div><span style=3D"font-f=
amily:arial,sans-serif;font-size:13px;border-collapse:collapse"><br>

</span></div>
</div></div>

--001a11c1f3ec8afd4804f95b94dc--

--===============3090691019698025682==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

__
distcc mailing list            http://distcc.samba.org/
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/distcc
--===============3090691019698025682==--