log delays and logger CPUs

Chris Herdt <[email protected]> Wed, 16 Oct 2019 11:20:40 -0500
Newsgroups gmane.comp.security.detection.bro
Message-ID <CAO8nG4fwQsBGHqfTSoKAtFrZvN5u1jKGY0_M8e5CroWTxf10Cg@mail.gmail.com>
I have a cluster running Bro 2.6.4. One host runs a manager and logger, 8
other hosts run proxy and worker nodes.

Lately the logger node has not been able to keep up with the logs, and I've
noticed that the most recent entries in the current/conn.log are
significantly delayed (I've seen delays as high as 90 minutes).

The logger process has maxed out CPU usage on core 1. The node.cfg file
specifies 8 CPU cores (all on the same NUMA node as the NVMe drive where
the logs are written):

[logger]
type=logger
host=bromanager-01.umn.edu
pin_cpus=1,3,5,7,9,11,13,15

`broctl nodes` shows that only 1 CPU core is pinned:

/usr/local/bro/bin/broctl nodes
         logger - addr=10.x.x.x aux_scripts= brobase= count=1 env_vars=
ether= host=bromanager-01.umn.edu interface= lb_interfaces= lb_method=
lb_procs= name=logger pin_cpus=1 test_mykey= type=logger zone_id=
...

Can pin_cpus be used with a logger node? Any other suggestions for
improving logger performance?


-- 
Chris Herdt
UIS Systems Administrator
[email protected]

_______________________________________________
Zeek mailing list
[email protected]
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek