Re: Odd bug when using alert_json plugin

Thanks Noah.  We will look into it.  This is behavior ported from Snort 2.9.

On 6/16/18 5:37 AM, Noah Dietrich wrote:
> Ok, i think I've figured out what's going on with this issue, but I'm 
> not sure it's a good idea the way things are working.
>
> What seems to happen is when you start Snort3, it begins filling the 
> alert_json.txt file.  Once that file hits the limit (say 100 MB), 
> Snort re-names this file to alert_json.txt.nnnnnnnnnn, where the n's 
> represent the unixtime of when the file was re-named.  Snort then 
> creates a new alert_json.txt file (without the unixtime), and 
> continues to write new events to that file.
>
> The issues i see with this method of rollover are that log-monitoring 
> programs won't be expecting file names to change.  So i'd be forced to 
> only consume only log files with the unixtime portion, meaning i 
> wouldn't get to consume new events until the log rolled over. If i 
> consumed events from the original file, there is a good chance that 
> i'd also re-consume (duplicate) those events again when the filename 
> was renamed.
>
> My request is that when snort starts, it just outputs it's events to a 
> new file, with the unixtime appended to the 
> filename(alert_json.txt.nnnnnnnnnn) , and not output to alert_json.txt 
> at any time, and especially not rename files. This will ensure that 
> there are no issues ingesting this data into SIEMs like Splunk or the 
> ELK stack.
>
> Thank you,
> Noah
>
>
>
>
>
> On Wed, Jun 13, 2018 at 9:20 AM, Noah Dietrich 
> <[email protected] <mailto:[email protected]>> wrote:
>
>     I'm playing around with the alert_json plugin, and i found a bit
>     of odd behavior.
>
>     When running snort in alert mode, it correctly created the
>     /var/log/snort/alert_json.txt file and filled it with events
>     (about 17 MB of data in that file). I then restarted snort and
>     pushed a huge amount of data at it (sudo ping -c 10000 -i .001 -s
>     4046 10.0.0.106), and snort created a new file with the timestamp
>     (alert_json.txt.1528868120), even though the first file hadn't
>     filled up and filled it with 100 MB of data.
>
>     I then re-ran snort, with a smaller dataset from local pcaps, and
>     it appended the events to the original alert_json.txt file.
>
>     I'd call this an odd bug because most applications that consume
>     the json files will ignore the first file once the second file is
>     created. I'd say there are two issues here:
>     1. Snort rolled over to the second json file early, before the
>     original one (without the timestamp) was full. I'm not sure if
>     this was because of the large ammount of data i threw at it with
>     the ping command above.
>     2. snort appended alert data to the original alert_json file
>     because it wasn't full, even though there was a newer (by UTC
>     timestamp) json file in the directory.
>
>     i'd recommend that snort not create an initial alert_json file
>     without a timestamp if no alert_json files exist, just so there
>     are no differences between the output files.
>
>     I am running snort with community rules installed (unmodified),
>     two of my own local rules (one that alerts on icmp traffic), and
>     builtin rules enabled.
>
>     First run:
>     sudo snort -c /usr/local/etc/snort/snort.lua --pcap-filter \*.pcap
>     --pcap-dir ~/snort_src/Pcaps -R
>     /usr/local/etc/snort/rules/snort3-community.rules -A alert_json -s
>     65535 -k none -l /var/log/snort
>
>     Second run:
>     sudo snort -c /usr/local/etc/snort/snort.lua -i ens160 -l
>     /var/log/snort -A alert_json -R
>     /usr/local/etc/snort/rules/snort3-community.rules --warn-all
>
>     third run:
>     sudo snort -c /usr/local/etc/snort/snort.lua -r
>     ~/maccdc2012_00000.pcap -R
>     /usr/local/etc/snort/rules/snort3-community.rules -A alert_json -s
>     65535 -k none -l /var/log/snort
>     (note: pcap from
>     https://download.netresec.com/pcap/maccdc-2012/maccdc2012_00000.pcap.gz
>     <https://download.netresec.com/pcap/maccdc-2012/maccdc2012_00000.pcap.gz>)
>
>
>
>
> _______________________________________________
> Snort-devel mailing list
> [email protected]
> https://lists.snort.org/mailman/listinfo/snort-devel
>
> Please visit http://blog.snort.org for the latest news about Snort!

_______________________________________________
Snort-devel mailing list
[email protected]
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!