error: Open connections exceed FD_SETSIZE limit (1024 >= 1024)

------=_Part_148762_865402300.1762932313386
Content-Type: multipart/alternative; 
	boundary="----=_Part_148763_1531683611.1762932313386"

------=_Part_148763_1531683611.1762932313386
Content-Type: text/plain; charset="UTF-8"

I have a bundle that runs on the hub which collects files from all clients 
that was last seen 2 hours ago. This bundle create a lot of open 
connections towards the clients.

Here is a snippet of the bundle (in the ful bundle 22 files in total are 
declared to be collected):

bundle agent collect_files
{
  vars:
    # get a list with all clients which has connected to policy master 
within the last two hours
    cf_clients" slist => hostsseen("2","lastseen","name") ;

  files:
    "/tmp/cmdb_collect"
      create => "true",
      perms => m("600");

  methods:
    "any" usebundle => filecollector("$(cf_clients)"),
       action => silent_sometimes;
}

body action silent_sometimes {
  background => "true";
  ifelapsed => "30";
  expireafter => "30";
}

bundle agent filecollector(cf_client)
{
  files:  
      "/tmp/cmdb_collect/$(cf_client)"
      perms => m("600"),
      copy_from => 
no_backup_short_timeout_rcp("${sys.workdir}/collect/report.csv", 
"$(cf_client)"),
      action => immediate,
      comment => "Get cmd data from all clients",
      move_obstructions => "true";
}

body copy_from no_backup_short_timeout_rcp(from,server)
{
      servers     => { "$(server)" };
      source      => "$(from)";
      compare     => "mtime";
      copy_backup => "false";
      timeout     => "5";
}

While running this bundle on the hub and monitoring the file descriptor 
count for the `cf-agent` process it starts to error with the following 
error message as soon as the file descriptor count reaches ~1024:

error: Open connections exceed FD_SETSIZE limit (1024 >= 1024)

I monitor the fd count using the following command:

# ls -1 /proc/$(pidof cf-agent)/fd | wc -l

I have tried increasing the fd limit:

# grep "Max open files" /proc/$(pidof cf-agent)/limits
Max open files 24000 28000 files

But I believe FD_SETSIZE is not something I can alter for the kernel.

I have maxconnections set to 13000 in cf_serverd.cf:

# grep control_server_maxconnections /var/cfengine/inputs/def.json
    "control_server_maxconnections": "13000",

I recently upgraded my environment from 3.21.2 to 3.21.7 and the hub server 
from Ubuntu 22.04 to 24.04 and I _believe_ that this was not an issue 
before.

Am I missing something or is the method I use to collect files from clients 
not proper?

--
Markus


-- 
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To view this discussion visit https://groups.google.com/d/msgid/help-cfengine/aa080387-d68e-4c42-9dac-d86406ba9a35n%40googlegroups.com.

------=_Part_148763_1531683611.1762932313386
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

I have a bundle that runs on the hub which collects files from all clients =
that was last seen 2 hours ago. This bundle create a lot of open connection=
s towards the clients.<br /><br />Here is a snippet of the bundle (in the f=
ul bundle 22 files in total are declared to be collected):<br /><br />bundl=
e agent collect_files<br />{<br />=C2=A0 vars:<br />=C2=A0 =C2=A0 # get a l=
ist with all clients which has connected to policy master within the last t=
wo hours<br />=C2=A0 =C2=A0 cf_clients" slist =3D&gt; hostsseen("2","lastse=
en","name") ;<br /><br />=C2=A0 files:<br />=C2=A0 =C2=A0 "/tmp/cmdb_collec=
t"<br />=C2=A0 =C2=A0 =C2=A0 create =3D&gt; "true",<br />=C2=A0 =C2=A0 =C2=
=A0 perms =3D&gt; m("600");<br /><br />=C2=A0 methods:<br />=C2=A0 =C2=A0 "=
any" usebundle =3D&gt; filecollector("$(cf_clients)"),<br />=C2=A0 =C2=A0 =
=C2=A0 =C2=A0action =3D&gt; silent_sometimes;<br />}<br /><br />body action=
 silent_sometimes {<br />=C2=A0 background =3D&gt; "true";<br />=C2=A0 ifel=
apsed =3D&gt; "30";<br />=C2=A0 expireafter =3D&gt; "30";<br />}<br /><br /=
>bundle agent filecollector(cf_client)<br />{<br />=C2=A0 files: =C2=A0<br =
/>=C2=A0 =C2=A0 =C2=A0 "/tmp/cmdb_collect/$(cf_client)"<br />=C2=A0 =C2=A0 =
=C2=A0 perms =3D&gt; m("600"),<br />=C2=A0 =C2=A0 =C2=A0 copy_from =3D&gt; =
no_backup_short_timeout_rcp("${sys.workdir}/collect/report.csv", "$(cf_clie=
nt)"),<br />=C2=A0 =C2=A0 =C2=A0 action =3D&gt; immediate,<br />=C2=A0 =C2=
=A0 =C2=A0 comment =3D&gt; "Get cmd data from all clients",<br />=C2=A0 =C2=
=A0 =C2=A0 move_obstructions =3D&gt; "true";<br />}<br /><br />body copy_fr=
om no_backup_short_timeout_rcp(from,server)<br />{<br />=C2=A0 =C2=A0 =C2=
=A0 servers =C2=A0 =C2=A0 =3D&gt; { "$(server)" };<br />=C2=A0 =C2=A0 =C2=
=A0 source =C2=A0 =C2=A0 =C2=A0=3D&gt; "$(from)";<br />=C2=A0 =C2=A0 =C2=A0=
 compare =C2=A0 =C2=A0 =3D&gt; "mtime";<br />=C2=A0 =C2=A0 =C2=A0 copy_back=
up =3D&gt; "false";<br />=C2=A0 =C2=A0 =C2=A0 timeout =C2=A0 =C2=A0 =3D&gt;=
 "5";<br />}<br /><br />While running this bundle on the hub and monitoring=
 the file descriptor count for the `cf-agent` process it starts to error wi=
th the following error message as soon as the file descriptor count reaches=
 ~1024:<br /><br />error: Open connections exceed FD_SETSIZE limit (1024 &g=
t;=3D 1024)<br /><br />I monitor the fd count using the following command:<=
br /><br /># ls -1 /proc/$(pidof cf-agent)/fd | wc -l<br /><br />I have tri=
ed increasing the fd limit:<br /><br /># grep "Max open files" /proc/$(pido=
f cf-agent)/limits<br />Max open files            24000                2800=
0                files<br /><br />But I believe=C2=A0FD_SETSIZE is not some=
thing I can alter for the kernel.<br /><br />I have maxconnections set to 1=
3000 in cf_serverd.cf:<br /><br /># grep control_server_maxconnections /var=
/cfengine/inputs/def.json<br />=C2=A0 =C2=A0 "control_server_maxconnections=
": "13000",<br /><br />I recently upgraded my environment from 3.21.2 to 3.=
21.7 and the hub server from Ubuntu 22.04 to 24.04 and I _believe_ that thi=
s was not an issue before.<br /><br />Am I missing something or is the meth=
od I use to collect files from clients not proper?<br /><br />--<br />Marku=
s<br /><br /><br />

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;help-cfengine&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:[email protected]">help-=
[email protected]</a>.<br />
To view this discussion visit <a href=3D"https://groups.google.com/d/msgid/=
help-cfengine/aa080387-d68e-4c42-9dac-d86406ba9a35n%40googlegroups.com?utm_=
medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msgid/help-=
cfengine/aa080387-d68e-4c42-9dac-d86406ba9a35n%40googlegroups.com</a>.<br /=
>

------=_Part_148763_1531683611.1762932313386--

------=_Part_148762_865402300.1762932313386--