Re: Help with Rules file to support Vyatta Firewall logs

On Wed, Aug 18, 2010 at 10:03:47AM -0500, Ron Rosson wrote:
> I am looking at moving to prelude from a different opensource SIEM and
> the only thing that is holding me back is the parsing of Vyatta's
> firewall logs does not look to be supported. I have provided a log
> sample below of what the firewall log entries look like. From my
> investigation it looks like I just need to find the correct regular
> expression for the action (Accept, Drop,Reject) the firewall takes.

Hi,

First of all a quick note: the example lines *should* match standard
netfilter rules, since they are exactly the same (except the prefix).
If they don't match, then your problem is somewhere else.

I'll reply in two parts, first for just matching the rules, then trying
to extract the decision (Accept, Drop or Reject).

> 
> #Accept
> #Dec 11 16:50:31 vyatta kernel: [ 936.677646] [wan-local-120-A] IN=eth1 OUT= MAC=00:30:48:de:ce:01:00:25:3c:8d:a6:59:08:00 SRC=75.9.51.236 DST=99.49.XX.XX LEN=136 TOS=0x00 PREC=0x00 TTL=55 ID=8141 PROTO=ESP SPI=0x9f9f37f4

This one exactly matches the rule 1303 (netfilter.rules line 147).
I replaced the XX.XX by 0.0 to test the regex (otherwise the regex won't
match, it searches for numbers).
Here's how I tested:

$ pcretest
PCRE version 8.02 2010-03-19

  re> /IN=(\w*) OUT=(\w*)( MAC=)?([\w:]+)?\s+SRC=([\d\.]+) DST=([\d\.]+)
LEN=(\d+) TOS=(\w+) PREC=(\w+) TTL=(\d+) ID=(\d+) (CE )?(DF )?(MF
)?(FRAG:\d+ )?(OPT \(\w+\) )?PROTO=(AH|ESP) (INCOMPLETE \[\d+ bytes\]
)?SPI=(\w+)/
data> Dec 11 16:50:31 vyatta kernel: [ 936.677646] [wan-local-120-A]
IN=eth1 OUT= MAC=00:30:48:de:ce:01:00:25:3c:8d:a6:59:08:00
SRC=75.9.51.236 DST=99.49.0.0 LEN=136 TOS=0x00 PREC=0x00 TTL=55 ID=8141
PROTO=ESP SPI=0x9f9f37f4
Matched, but too many substrings
 0: IN=eth1 OUT= MAC=00:30:48:de:ce:01:00:25:3c:8d:a6:59:08:00
SRC=75.9.51.236 DST=99.49.0.0 LEN=136 TOS=0x00 PREC=0x00 TTL=55 ID=8141
PROTO=ESP SPI=0x9f9f37f4
 1: eth1
 2: 
 3:  MAC=
 4: 00:30:48:de:ce:01:00:25:3c:8d:a6:59:08:00
 5: 75.9.51.236
 6: 99.49.0.0
 7: 136
 8: 0x00
 9: 0x00
10: 55
11: 8141
12: <unset>
13: <unset>
14: <unset>


> #Drop
> #Dec 11 16:47:18 vyatta kernel: [ 743.708685] [wan-in-9998-D] IN=eth1 OUT=eth1 SRC=174.36.30.9 DST=192.168.XXX.XXX LEN=250 TOS=0x00 PREC=0x00 TTL=49 ID=26630 DF PROTO=TCP SPT=80 DPT=56677 WINDOW=54 RES=0x00 ACK PSH URGP=0 

Same here, with rule id 1300

> #Reject
> #Dec 11 16:47:18 vyatta kernel: [ 743.708685] [lan-untrusted-10-R] IN=eth1.100 OUT=eth2 SRC= 192.168.XXX.XXX DST=174.36.30.9 LEN=250 TOS=0x00 PREC=0x00 TTL=49 ID=26630 DF PROTO=TCP SPT=80 DPT=56677 WINDOW=54 RES=0x00 ACK PSH URGP=0 
> 

This one didn't match, because of a small bug in the regex: the input
interface name is compared to IN=(\w*) which won't match because of the
dot in the interface name eth1.100
I'll fix this one in the repository.


Now, how to extract the decision part of the rule. I'll take as an
example the second (drop) line.
Basically, you just have to insert a small regex before the existing to
match the decision. Since you modify a common rules file, I'll suggest
to copy & rename the file, and change rules id to be sure you won't
loose your changes.

Complete regex definition for rule 1300 is:
regex=IN=(\w*) OUT=(\w*)( MAC=)?([\w:]+)?\s+SRC=([\d\.]+) DST=([\d\.]+)
LEN=(\d+) TOS=(\w+) PREC=(\w+) TTL=(\d+) ID=(\d+) (CE )?(DF )?(MF
)?(FRAG:\d+ )?(OPT \(\w+\) )?PROTO=TCP (INCOMPLETE \[\d+ bytes\]
)?SPT=(\d+) DPT=(\d+) (SEQ=\d+ ACK=\d+ )?WINDOW=(\d+) (RES=(\w+) )?(CWR
)?(ECE )?(URG )?(ACK )?(PSH )?(RST )?(SYN )?(FIN )?URGP=(\d+); \

You have to insert something that matches [wan-in-9998-D], and put in
parents the decision. Something like the following should do the trick:
\[[\w-]+-([ADR])\]

Quick test:
$ pcretest
PCRE version 8.02 2010-03-19

  re> /\[[\w-]+-D\]/
data> [wan-in-9998-D]
 0: [wan-in-9998-D]

See, the 'D' has been put in group 1
So, the complete regex for rule 1300 becomes:
regex=\[[\w-]+-D\] IN=(\w*) OUT=(\w*)( MAC=)?([\w:]+)?\s+SRC=([\d\.]+) DST=([\d\.]+)
LEN=(\d+) TOS=(\w+) PREC=(\w+) TTL=(\d+) ID=(\d+) (CE )?(DF )?(MF
)?(FRAG:\d+ )?(OPT \(\w+\) )?PROTO=TCP (INCOMPLETE \[\d+ bytes\]
)?SPT=(\d+) DPT=(\d+) (SEQ=\d+ ACK=\d+ )?WINDOW=(\d+) (RES=(\w+) )?(CWR
)?(ECE )?(URG )?(ACK )?(PSH )?(RST )?(SYN )?(FIN )?URGP=(\d+); \

I just inserted the previous part in front of the existing.

Testing with pcretest (not displayed here ..) shows that it works.
This gives one rule matching the drop.

If you want to keep the verdict and set it in the IDMEF alert.
You have to create (copy) one rule for each different decision, and can
add in the rule contents something like
 assessment.impact.completion=failed; \

Since the packet was dropped.

Do the same with accept (completion = succeeded) and reject, and you're done.


There are of course many other possibilities, for ex. you can log the
decision in the assessment.impact.description entry, etc. Just be
careful that if you add parents around the decision to keep the result
in the regex, then the existing groups ($1 $2 etc.) will have to be
shifted (adding 1).

HTH,
Pierre



_______________________________________________
Prelude-user site list
[email protected]
http://lists.prelude-technologies.com/mailman/listinfo/prelude-user