Patch to correct the way Snort names output files

Snort team:

Attached is a patch that fixes the issue that I reported regarding the way
that Snort was naming and re-naming the output alert files. The issue was
that snort created the initial file without the unixtime in the name, and
then re-named the file by appending the unixtime when the file size limit
was reached. This causes issues with Splunk and the ELK stack, because they
have to wait for the file to be re-named before the file can be indexed
(otherwise you risk duplicating or missing events, or waiting until the log
file rolls over, which could be a long time).

This patch fixes the issue by modifying the get_instance_file() in
main/thread.cc to append the unixtime to all filenames by default (the
unixtime will indicate when the file was created).  A side-effect of this
change is that i have removed the RollAlertFile() function in log/log.cc,
as it is no longer needed.

If you re-start Snort, events will not be written to a half-full alert
file, instead alerts will be written to a new file (I supposed you could
modify the code to continue filling the most recent alert file, but I don't
think that's necessary, and i can't think of a reason you'd need that
functionality).

This is my first time submitting a patch to a project, so please let me
know if there is anything I should be doing differently. I'm also not a
professional C coder, so it's very possible that my code will need to be
implemented differently to handle issues I am not aware of.

I have tested this patch successfully with the following loggers:
alert_csv
alert_fast
alert_full
log_codecs
log_hext

 and I tested the output with the following options (to make sure that this
patch doesn't screw up more complex output options):
  --run-prefix
  --id-zero
  --id-subdir

Note: this will also append the unixtime to the appid_stats.log file

this patch can be installed by copying it to the snort3 folder, navigating
to that folder, and running:
patch -p1 < unixtime-filenames.diff

The real benefit of this patch is that your file-based output will be
created in a way that Splunk or the ELK stack (or other log-collecting
software) can easily, quickly, and correctly ingest Snort alerts and other
outputted information.  I have written a Splunk plugin that takes advantage
of the functionality this patch enables, and will make ingesting Snort log
data much easier. Basically this new method of file naming works the way
most log-collecting software expects, which should make it easier to load
Snort alerts into those tools.

Thanks, and happy new year.
Noah

_______________________________________________
Snort-devel mailing list
[email protected]
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!

diff -Naur snort3/src/log/log.cc snort3-modified/src/log/log.cc
--- snort3/src/log/log.cc	2018-12-31 12:02:47.893390201 -0500
+++ snort3-modified/src/log/log.cc	2018-12-31 12:04:57.830835363 -0500
@@ -88,40 +88,6 @@
     return file;
 }
 
-/****************************************************************************
- *
- * Function: RollAlertFile(char *)
- *
- * Purpose: rename existing alert file with by appending time to name
- *
- * Arguments: filearg => the filename to rename (same as for OpenAlertFile())
- *
- * Returns: 0=success, else errno
- *
- ***************************************************************************/
-int RollAlertFile(const char* filearg)
-{
-    char newname[STD_BUF+1];
-    time_t now = time(nullptr);
-
-    if ( !filearg )
-        filearg = "alert.txt";
-
-    std::string name;
-    get_instance_file(name, filearg);
-    const char* oldname = name.c_str();
-
-    SnortSnprintf(newname, sizeof(newname)-1, "%s.%lu", oldname, (unsigned long)now);
-
-
-    if ( rename(oldname, newname) )
-    {
-        FatalError("RollAlertFile() => rename(%s, %s) = %s\n",
-            oldname, newname, get_error(errno));
-    }
-    return errno;
-}
-
 //--------------------------------------------------------------------
 // default logger stuff
 //--------------------------------------------------------------------
diff -Naur snort3/src/log/log.h snort3-modified/src/log/log.h
--- snort3/src/log/log.h	2018-12-31 12:02:47.893390201 -0500
+++ snort3-modified/src/log/log.h	2018-12-31 12:05:12.434549500 -0500
@@ -34,7 +34,6 @@
 }
 
 FILE* OpenAlertFile(const char*);
-int RollAlertFile(const char*);
 
 void OpenLogger();
 void CloseLogger();
diff -Naur snort3/src/log/text_log.cc snort3-modified/src/log/text_log.cc
--- snort3/src/log/text_log.cc	2018-12-31 12:02:47.893390201 -0500
+++ snort3-modified/src/log/text_log.cc	2018-12-31 12:05:39.774014961 -0500
@@ -166,7 +166,6 @@
         return;
 
     TextLog_Close(txt->file);
-    RollAlertFile(txt->name);
     txt->file = TextLog_Open(txt->name);
 
     txt->last = time(nullptr);
diff -Naur snort3/src/main/thread.cc snort3-modified/src/main/thread.cc
--- snort3/src/main/thread.cc	2018-12-31 12:02:47.901390043 -0500
+++ snort3-modified/src/main/thread.cc	2018-12-31 12:04:19.671583461 -0500
@@ -106,6 +106,11 @@
 
     file += name;
 
+    // modification by Noah Dietrich to append unixtime to all files
+    file += '.';
+    time_t now = time(nullptr);
+    file += std::to_string(now);
+
     return file.c_str();
 }
 }