Re: pcr and
Frank <[email protected]> Fri, 20 Apr 2018 17:19:59 +0200
| Newsgroups | gmane.network.argus |
|---|---|
| Message-ID | <[email protected]> |
Hi Carter, thx for your quick answer. 1. That did the trick, thx! 2. Yes, i've set ARGUS_FLOW_KEY="CLASSIC_5_TUPLE" The reasons for my big ARGUS_FLOW_STATUS_INTERVAL value: - i don't need instant live updates about active flows - i don't want those updates filling up my log files ( we have many long connections going on, and when there is a status update entry for each of them every second, it gets unnecessarily big) - As far as i understand, flow information are written as soon as the flow is "done" (timeout reached, connection closed,...), so this value shouldn't affect those, right? Ok, what i want in the end is a list of saddr daddr pairs that have long connections with each other, based on the mean field. I have for example those records: rafilteraddr -r argus.out -f /tmp/filter -p0 -s -state -s -flgs -s -dir -s +mean +seq StartTime Proto SrcAddr Sport DstAddr Dport TotPkts TotBytes Mean Seq 21:37:19 tcp 10.10.10.10.42972 10.20.20.20.https 32536 26592499 1790 30487007 22:07:19 tcp 10.10.10.10.42972 10.20.20.20.https 4961 3942584 1795 30487007 22:37:21 tcp 10.10.10.10.42972 10.20.20.20.https 1125 272903 1791 30487007 23:07:21 tcp 10.10.10.10.42972 10.20.20.20.https 494 138085 780 30487007 [...] 06:19:14 tcp 10.10.10.10.23391 10.20.20.20.https 730 72320 1793 31292145 06:49:16 tcp 10.10.10.10.23391 10.20.20.20.https 647 57559 1796 31292145 07:19:22 tcp 10.10.10.10.23391 10.20.20.20.https 701 70449 1793 31292145 07:49:25 tcp 10.10.10.10.23391 10.20.20.20.https 685 60024 1794 31292145 08:19:26 tcp 10.10.10.10.23391 10.20.20.20.https 727 72165 1799 31292145 08:49:35 tcp 10.10.10.10.23391 10.20.20.20.https 672 59209 1793 31292145 There are two connections (the first lines are from the day before ) As i can't aggregate on the sequence number, i use the following command as my first aggregation step: rafilteraddr -r argus.out -f /tmp/filter -w - | racluster -m proto saddr sport daddr dport -M norep -p0 -s -state -s -flgs -s -dir -s +mean +seq +trans +sum StartTime Proto SrcAddr Sport DstAddr Dport TotPkts TotBytes Mean Seq Trans Sum 06:19:14 tcp 10.10.10.10.23391 10.20.20.20.https 8231 776687 0 31292145 0 0 21:37:19 tcp 10.10.10.10.42972 10.20.20.20.https 39116 30946071 0 30487007 0 0 I use the '-M norep' option, as without it, i get this, which gives me the mean duration time over all records, but not for connections: StartTime Proto SrcAddr Sport DstAddr Dport TotPkts TotBytes Mean Seq Trans Sum 06:19:14 tcp 10.10.10.10.23391 10.20.20.20.https 8231 776687 1789 31292145 12 21468 21:37:19 tcp 10.10.10.10.42972 10.20.20.20.https 39116 30946071 1539 30487007 4 6156 And in my last step, i aggregate with saddr daddr: rafilteraddr -r argus.out -f /tmp/filter -w - | racluster -m proto saddr sport daddr dport -M norep -w - | racluster -m saddr daddr -p0 -s -state -s -flgs -s -dir -s +mean +seq +trans +sum StartTime Proto SrcAddr Sport DstAddr Dport TotPkts TotBytes Mean Seq Trans Sum 21:37:19 tcp 10.10.10.10 10.20.20.20.https 47347 31722758 13877 31292145 2 27753 For this scenario, it works as expected: there were 2 connections between those two ip addresses, with a total duration of 27753 and mean 13877. But as i said, the remaining problem are connections that use the same 5-tuple for multiple connections/flows as with ntp. Did i understand/do somehting wrong here? 3. Ah, okay, understood. Thx and Regards, Frank On 04/20/2018 03:38 PM, Carter Bullard wrote: > Hey Frank, > The PCR requires that you have ARGUS_GENERATE_APPBYTE_METRIC set in argus. The PCR is based on the values of sappbytes and dappbytes, so as a sanity check, print these fields using ra to see if the metric is being generated. > > So not sure what you’re wanting to achieve in 2. The 5-tuple aggregation is the default, as that is the “basic” way of forming flows. Its always better if you include the command line that you are using, so we can see whats up. > > But, why use such a large ARGUS_FLOW_STATUS_INTERVAL, I use either 5 or 1, and never go much higher. This is the definition of your “quantum" of observation. With a 3 hour interval, man, do you have to wait a long time before anything comes out if your monitoring live traffic. Biggest problem is that your going to see a lot of port reuse in 3 hours, so flows will collide, and you’ll get large flow duration times. If you use default aggregation, then the mean, stddev, max and min are calculated using the ‘due’ field of each record. So it looks like it is working. > > Use a small ARGUS_FLOW_STATUS_INTERVAL, and then use aggregation to build up the long lived flows. > > So, #3 is a ‘gotcha’ of sorts. “tcp” and “udp” are really shortcut aliases, tcp means “ip proto tcp”, so you're generating filter errors by accident ??? Because tcp and udp and icmp are used so often, they are aliased, seems that all the other protocols (i.e udplite) need the ip proto. > > So this works… > > ra -b - not udp > ra -b - not \(ip proto 17\) > > So quoting them makes them a string, rather than the keyword, and we then lookup them up, rather than assign the value. Sorry for that inconvenience. > > Carter > >> On Apr 20, 2018, at 6:39 AM, Frank <[email protected]> wrote: >> >> Hi, >> >> i'm new to the list and to argus, so i hope i don't repeat already asked >> questions. >> First of all: i really like the workflow of argus and its functionality. >> But i encountered some obstacles that however might also be caused by my >> misunderstanding. >> >> Argus server and client version: 3.0.8.2 >> >> >> 1. Is it possible that the PCR functionality is broken? I always get a >> PCR of -0 >> >> ra -r argus.out -s sbytes dbytes pcr | head >> SrcBytes DstBytes PCRatio >> 83 126 -0.000000 >> 1644 4213 -0.000000 >> 83 126 -0.000000 >> 194 0 -0.000000 >> 194 0 -0.000000 >> 73 89 -0.000000 >> 73 101 -0.000000 >> 74 60 -0.000000 >> 3351 4025 -0.000000 >> >> >> 2. Did i see right that it is not possible to aggregate on the argus >> sequence number, or did i overlook something? The main reason i'd >> want/need this is for an accurate mean duration time. The best i was >> able to accomplish was an aggregation (via racluster) over proto src/dst >> addr/port, but this does e.g. not work with ntp queries. >> I've set ARGUS_FLOW_STATUS_INTERVAL=1800 , and when i look for long >> connections and aggregate with '-m proto saddr sport daddr dport' , i >> get a maximum mean time of 1800, as each status message seems to be >> treated as a separate flow (at least it looks that way to me). When i >> also use -M norep, it works for all connections where the selected >> features are unique for the connection, but, as mentioned before, does >> not work with ntp connections, or for chatty hosts that use the same >> source port for a destination host twice or more. >> >> >> 3. Another issue that i encounter is a weird filtering behaviour. >> For example: >> >> ra -r argus.out -- 'not (ip proto udp)' >> ra[26153]: 1524219408.776398 filter syntax error: 'not (ip proto udp)' >> >> ra -r argus.out -- 'not (ip proto tcp)' >> ra[26219]: 12:19:04.157946 filter syntax error: 'not (ip proto tcp)' >> >> ra -r argus.out -- 'not (ip proto udp or tcp)' >> ra[26233]: 12:19:31.355346 filter syntax error: 'not (ip proto udp or ...' >> >> But this works: >> ra -r argus.out -- 'not (ip proto "udp")' >> ra -r argus.out -- 'not (ip proto 17 or tcp)' >> ra -r argus.out -- 'not (ip proto "udp" or tcp)' >> ra -r argus.out -- 'not udp and not tcp' >> >> >> Thanks in advance and regards, >> Frank >> >> >>