Re: Number of CPU cores for 100Gbps

These rules aren't current anymore and frankly, have never been accurate.

Your Zeek speed depends on the traffic you have, if you have some elephant
flows (and how you deal with them), scripts you run, etc. I remember
pushing between 5-10Gbit/sec through a server with 24 cores (not threads),
with room to spare.

You will also need memory, and depending on scripts you intend to write,
that might be quite a lot. We run with 192GB / server.

Do you have 100Gbit of traffic or 100Gbit interfaces?

Either way, you're gonna build yourself a cluster with a packet broker in
front of it. Arista works well, other people use different brands,
depending on your needs and your budget.

Give those tuning guides I wrote with Suricata developers a read, while on
it, they apply to Zeek as well. Of course Suricata can process way more
traffic per core, than Zeek, because the processing it does is way simpler.

https://github.com/pevma/SEPTun
https://github.com/pevma/SEPTun-Mark-II


On Sun, Apr 28, 2019 at 11:35 AM Woot4moo <[email protected]> wrote:

> My understanding is that 4,000+ CPU cores would be necessary to support
> this throughput. In the recent meeting from CERN I recall seeing someone
> describe 200Gbps, which would imply 8,000+ CPU cores. Is this accurate, or
> am I doing a conversion incorrectly?
>
> I am basing this purely on this quote, from
>
> https://docs.zeek.org/en/stable/cluster/
>
> “The rule of thumb we have followed recently is to allocate approximately
> 1 core for every 250Mbps of traffic that is being analyzed. However, this
> estimate could be extremely traffic mix-specific. It has generally worked
> for mixed traffic with many users and servers. For example, if your traffic
> peaks around 2Gbps (combined) and you want to handle traffic at peak load,
> you may want to have 8 cores available (2048 / 250 == 8.2). ”
> _______________________________________________
> Zeek mailing list
> [email protected]
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek

_______________________________________________
Zeek mailing list
[email protected]
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek