Re: Zeek with ELK

I read OP question as "I have Zeek running on FreeBSD, what is a
sensible option for shipping logs from the sensor to an Elastic
Stack?" Apologies if it's the wrong read.

In that case I wouldn't want to install either the whole stack nor
even Logstash on the sensor as it alone tends to consume an excessive
amount of memory, not what you want on a sensor. Filebeat (a small
footprint data collector/shipper) is the way to go if you're shipping
remotely.

If Filebeat isn't an option on the platform, maybe explore Fluent Bit:

https://github.com/fluent/fluent-bit
https://fluentbit.io/

Fluent Bit can output directly to Elasticsearch:
https://fluentbit.io/documentation/0.14/output/elasticsearch.html

Even a Fluentd can run with typically lower memory consumption than
Logstash, so perhaps worth exploring both/either:

https://github.com/fluent/fluentd
https://www.fluentd.org/

Fluentd can also output to Elasticsearch:
https://docs.fluentd.org/output/elasticsearch

There are other options for shippers too, such as Syslog-ng:
https://www.syslog-ng.com/community/b/blog/posts/logging-to-elasticsearch-made-simple-with-syslog-ng

- Darren

On Sun, Jan 5, 2020 at 9:11 AM Michael Shirk <[email protected]> wrote:
>
> You should be able to fire up Elastic, Logstash and Kibana on FreeBSD, using recommend Logstash configs to read in the log files from the file system. I can check about the Filebeat port to see if that can be updated or fixed.
>
> I myself just use the CLI tools but have been working on something "Not Java" to ingest log files into other than Splunk.
>
>
> --
> Michael Shirk
> Daemon Security, Inc.
> https://www.daemon-security.com
>
> On Sun, Jan 5, 2020, 10:35 sec-x sec-x <[email protected]> wrote:
>>
>> Hi,
>>
>> I recently used zeek IDS on FreeBSD 12.1 - Default Policy (GetTraffic
>> from TAP on the network) and i want to send all the logs to ELK in
>> realtime.
>>
>> I saw Filebeat ports on BSD is old and has problems.
>>
>> How can i send the logs from the BSD to the Elastic (what is the
>> correct/best way)?
>>
>>
>> Thanks,
>>
>> CM.
>> _______________________________________________
>> Zeek mailing list
>> [email protected]
>> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek
>
> _______________________________________________
> Zeek mailing list
> [email protected]
> http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek



-- 
Darren Spruell
[email protected]
_______________________________________________
Zeek mailing list
[email protected]
http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/zeek