Re: ingesting mod-security audit log to elasticsearch (or other system)

Thanks for the great info/ideas, I really appreciate this.

Sorry, I'd missed the JSON log format contribution, that should indeed parsing/mapping the data much easier.

I'm fortunately not affected by PCI, but GDPR is a potential issue. As I understand it sanitisation only applies to the error log, and not the audit report, which for me, (and I presume others) creates some challenges. One thought I'd had was to pipe the serialised audit log through elastics beats (or similar) via logstash to elastic search. I think this would give me the opportunity to sanitise or encrypt any sensitive fields before they were stored/touched the disk.

Best Regards,
Paul

________________________________
From: Manuel Spartan <[email protected]>
Sent: Wednesday, October 9, 2019 2:51:36 PM
To: [email protected] <[email protected]>
Subject: Re: [mod-security-users] ingesting mod-security audit log to elasticsearch (or other system)

Hi Paul, other than the full payloads the info is already in the error log which is a many times smaller and easier to process, depending on the size of your implementation and the kind of transactions you handle processing the audit log even in json format, it is at least a factor of magnitude heavier than the error log.
I prefer to generate links to the auditlog based on the transaction id in the error log on my monitoring console as I only had to check the audit log for forensics or when payloads are too big.
There is one little detail that may bite you hard PII and financial data on audit logs, be careful in some cases it may be better to just drop the request body or response body parts from the audit log. GDPR and PCI won’t like the audit log.
Regards,
Manuel

Sent from my iPhone

On Oct 9, 2019, at 5:39 AM, Robert Paprocki <[email protected]<mailto:[email protected]>> wrote:

We had this exact problem, which is why we wrote JSON audit logging for 2.x releases. It should still be available in the 2.9.x series of releases.

https://www.dreamhost.com/blog/making-sense-of-modsecurity-json-audit-logs/

https://www.feistyduck.com/library/modsecurity-handbook-2ed-free/online/ch04-logging.html

https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-%28v2.x%29#SecAuditLogFormat

On Oct 9, 2019, at 00:27, Paul Beckett <[email protected]<mailto:[email protected]>> wrote:


I'm interested in ingesting the mod-security audit log (generated by modsecurity 2.9.x) into elasticsearch or other system. However, parsing the audit log format looks like it will require a substantial amount of work. I was hoping that someone might have solved this problem, and that I could reuse an existing solution.

I've spent a while googling, trying to find what existing solutions exist.
The only thing I've managed to find are a couple of blog posts with github links from several years ago:
https://github.com/bitsofinfo/logstash-modsecurity<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbitsofinfo%2Flogstash-modsecurity&data=02%7C01%7CP.Beckett%40uea.ac.uk%7C2373f6ec44b248380f8508d74c84b3c7%7Cc65f8795ba3d43518a070865e5d8f090%7C0%7C0%7C637062005146617288&sdata=ECTTZfGFdDHiwMzps20ss9i%2FEX1m7DD3LF5is0LI4%2BM%3D&reserved=0>
https://github.com/bitsofinfo/fluentd-modsecurity

Before I dive to far down this rabbit hole, I was wondering if anyone else out there in the community had a solution for this, and if so whether they would be willing to share their high level approach, and/or any implementation details.

Thanks,
Paul
_______________________________________________
mod-security-users mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/
_______________________________________________
mod-security-users mailing list
[email protected]<mailto:[email protected]>
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/

_______________________________________________
mod-security-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mod-security-users
Commercial ModSecurity Rules and Support from Trustwave's SpiderLabs:
http://www.modsecurity.org/projects/commercial/rules/
http://www.modsecurity.org/projects/commercial/support/