Re: error: Open connections exceed FD_SETSIZE limit (1024 >= 1024)

------=_Part_70882_1707837006.1763023353237
Content-Type: multipart/alternative; 
	boundary="----=_Part_70883_673836932.1763023353237"

------=_Part_70883_673836932.1763023353237
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Nick,

Thanks for the suggestion to change from $ to @ when referencing the list=
=20
of clients. This helped in the way that rest of the promises get to run=20
even if FD_SETSIZE is being hit by the cf-agent process.

I forgot to mention it but I did try altering the=20
control_agent_maxconnections variable via augments but I did not see any=20
difference. Thanks for updating the documentation though!

Is FD_SETSIZE a limitation in the cf-agent program itself perhaps? While=20
looking at the core source (with my VERY limited C knowledge) it seems that=
=20
the select function is used in the TryConnect function [1] which to my=20
understanding has a limitation of 1024 file descriptors. I noticed the 1953=
=20
pull request [2] which suggested to change select() to poll() instead but=
=20
was ultimately closed due to Windows not supporting poll().

[1] https://github.com/cfengine/core/blob/e639010a9d2453ed7d8559fb9f15d78b6=
5ebfb52/libcfnet/net.c#L615
[2] https://github.com/cfengine/core/pull/1953
On Wednesday, November 12, 2025 at 6:40:42=E2=80=AFPM UTC+1 nick.a...@north=
ern.tech=20
wrote:

> Hey Markus,=20
>
> The first thing that jumped out at me from above was how you call your=20
> bundle to copy the file:=20
>
> methods:
>   "any" usebundle =3D> filecollector("$(cf_clients)"),
>      action =3D> silent_sometimes;
>
> You reference the list of clients with $. So, the bundle will be actuated=
=20
> once for each client. Have you instead tried passing the list so that the=
=20
> bundle is called just once and the iteration on the client happens inside=
=20
> the bundle?=20
>
> Like:=20
>
> methods:
>   "any" usebundle =3D> filecollector("@(cf_clients)"),
>      action =3D> silent_sometimes;
>
> Regarding the maximum number of connections, I see that you mention=20
> setting control_server_maxconnections, note that variable is used by body=
=20
> server control (cf-serverd). For your policy, where you have one agent th=
at=20
> is creating a bunch of connections from cf-agent you might consider setti=
ng=20
> default:def.control_agent_maxconnections. It's set to 30 in the MPF by=20
> default.=20
>
> I would link you to the docs that mention it, but it doesn't seem to be=
=20
> explicitly called out yet so I just filed an issue about it here=20
> https://northerntech.atlassian.net/browse/CFE-4602=20
>

--=20
You received this message because you are subscribed to the Google Groups "=
help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to [email protected].
To view this discussion visit https://groups.google.com/d/msgid/help-cfengi=
ne/a86d9bd8-1b7e-4b0e-8051-ecc0b015d00bn%40googlegroups.com.

------=_Part_70883_673836932.1763023353237
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Nick,<br /><br />Thanks for the suggestion to change from $ to @ when re=
ferencing the list of clients. This helped in the way that rest of the prom=
ises get to run even if FD_SETSIZE is being hit by the cf-agent process.<br=
 /><br />I forgot to mention it but I did try altering the control_agent_ma=
xconnections variable via augments but I did not see any difference. Thanks=
 for updating the documentation though!<br /><br />Is=C2=A0FD_SETSIZE a lim=
itation in the cf-agent program itself perhaps? While looking at the core s=
ource (with my VERY limited C knowledge) it seems that the select function =
is used in the TryConnect function [1] which to my understanding has a limi=
tation of 1024 file descriptors. I noticed the 1953 pull request [2] which =
suggested to change select() to poll() instead but was ultimately closed du=
e to Windows not supporting poll().<br /><br />[1]=C2=A0https://github.com/=
cfengine/core/blob/e639010a9d2453ed7d8559fb9f15d78b65ebfb52/libcfnet/net.c#=
L615<br />[2]=C2=A0https://github.com/cfengine/core/pull/1953<div class=3D"=
gmail_quote"><div dir=3D"auto" class=3D"gmail_attr">On Wednesday, November =
12, 2025 at 6:40:42=E2=80=AFPM UTC+1 [email protected] wrote:<br/></d=
iv><blockquote class=3D"gmail_quote" style=3D"margin: 0 0 0 0.8ex; border-l=
eft: 1px solid rgb(204, 204, 204); padding-left: 1ex;"><p>
Hey Markus,
</p>

<p>
The first thing that jumped out at me from above was how you call your bund=
le to copy the file:
</p>

<div>
<pre><span style=3D"color:#a89984;font-weight:bold">methods</span>:
  <span style=3D"color:#689d6a">&quot;any&quot;</span> usebundle =3D&gt; fi=
lecollector<span style=3D"color:#689d6a">(</span><span style=3D"color:#689d=
6a">&quot;$(cf_clients)&quot;</span><span style=3D"color:#689d6a">)</span>,
     action =3D&gt; silent_sometimes;
</pre>
</div>
<p>
You reference the list of clients with <code>$</code>. So, the bundle will =
be actuated once for each client. Have you instead tried passing the list s=
o that the bundle is called just once and the iteration on the client happe=
ns inside the bundle?
</p>

<p>
Like:
</p>

<div>
<pre><span style=3D"color:#a89984;font-weight:bold">methods</span>:
  <span style=3D"color:#689d6a">&quot;any&quot;</span> usebundle =3D&gt; fi=
lecollector<span style=3D"color:#689d6a">(</span><span style=3D"color:#689d=
6a">&quot;@(cf_clients)&quot;</span><span style=3D"color:#689d6a">)</span>,
     action =3D&gt; silent_sometimes;
</pre>
</div>

<p>
Regarding the maximum number of connections, I see that you mention setting=
 <code>control_server_maxconnections</code>, note that variable is used by =
body server control (cf-serverd). For your policy, where you have one agent=
 that is creating a bunch of connections from cf-agent you might consider s=
etting <code>default:def.control_agent_maxconnections</code>. It&#39;s set =
to 30 in the MPF by default.
</p>

<p>
I would link you to the docs that mention it, but it doesn&#39;t seem to be=
 explicitly called out yet so I just filed an issue about it here <a href=
=3D"https://northerntech.atlassian.net/browse/CFE-4602" target=3D"_blank" r=
el=3D"nofollow" data-saferedirecturl=3D"https://www.google.com/url?hl=3Den&=
amp;q=3Dhttps://northerntech.atlassian.net/browse/CFE-4602&amp;source=3Dgma=
il&amp;ust=3D1763104665552000&amp;usg=3DAOvVaw3qbU06ZYnp0HdQLbVAFrmy">https=
://northerntech.atlassian.net/browse/CFE-4602</a>
</p>
</blockquote></div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;help-cfengine&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]">help-=
[email protected]</a>.<br />
To view this discussion visit <a href=3D"https://groups.google.com/d/msgid/=
help-cfengine/a86d9bd8-1b7e-4b0e-8051-ecc0b015d00bn%40googlegroups.com?utm_=
medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msgid/help-=
cfengine/a86d9bd8-1b7e-4b0e-8051-ecc0b015d00bn%40googlegroups.com</a>.<br /=
>

------=_Part_70883_673836932.1763023353237--

------=_Part_70882_1707837006.1763023353237--