Re: Netflow (NSEL) updates from Cisco ASA
Marat Khalili <[email protected]>
| Newsgroups | gmane.linux.ntop.general |
|---|---|
| Organization | Russian Quantum Center |
| Message-ID | <[email protected]> |
> ASA (and other devices like PaloAlto) generate flows based on firewall > events, like flow create, deletion etc. and do not send interim > updates as flow devices do. ASA for instance sends only total octets. According to documentation <https://www.cisco.com/c/en/us/td/docs/security/asa/special/netflow/guide/asa_netflow.html#pgfId-1324322> there are Flow-Update events sent every minute by default. > Shall you have further comments on this matter, please file ticket on > github so we can keep a log of the discussion Ok, I'll re-test it my environment and open the ticket with what I'm seeing. -- With Best Regards, Marat Khalili On 13/07/17 01:26, Luca Deri wrote: > Hi all, > ASA (and other devices like PaloAlto) generate flows based on firewall > events, like flow create, deletion etc. and do not send interim > updates as flow devices do. ASA for instance sends only total octets. > > Due to this if we take into account flow updates we need to make sure > that the frequency of these updates is greater than the flow > inactivity timeout in ntopng as otherwise we would generate incorrect > values (i.e. we will account more traffic than the one that has really > passed on the network). The flow delete instead is not affected by > this problem as it is the very last message emitted for a given flow. > > This said, reading this thread it seems that you all have different > option and this I have modified nProbe to implement the behaviour you > expect. Shall you have further comments on this matter, please file > ticket on github so we can keep a log of the discussion > > Regards Luca > >> On 6 Jul 2017, at 14:13, Marat Khalili <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hello Simone, >> >> I'm also very interested in this issue, observing same behaviour as >> Pelham. Do you mean we need a custom fields configuration file to >> make this work? How will it be different from your attempts where >> "events other than flow-delete often contain to few attributes to >> properly update the flow"? >> >> -- >> >> With Best Regards, >> Marat Khalili >> >> On 06/07/17 14:54, Simone Mainardi wrote: >>> Dear Pelham, >>> >>> Thanks for sharing the pcap. Currently, in nProbe we deliberately >>> ignore ASA firewall events different from flow-delete. We have made >>> this choice as we have seen that events other than flow-delete often >>> contain to few attributes to properly update the flow. This is the >>> reason why other templates are silently ignored by nprobe. >>> >>> If you want your nProbe to proxy all the templates received as-is, >>> you may want to have a look at: >>> http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/ >>> >>> >>> Simone >>> >>> Regards, >>> Simone >>> >>> >>> >>> On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore >>> <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hey Simone, >>> >>> Thanks for the reply. I have configured the timeout values you >>> mentioned in nprobe however it seemed to have no effect. >>> I have generated a .pcap file that includes templates, flow >>> create, update, and teardown events. >>> >>> .pcap file can be downloaded from here: >>> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download >>> <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download> >>> >>> One thing I did notcie from the packet capture is that flow >>> update events are being listed as "Firewall Event: Unknown (5)" . >>> I'm not sure if that is to be expected. >>> >>> >>> Regards, >>> Pelham Whitmore >>> >>> >>> -----Original Message----- >>> From: [email protected] >>> <mailto:[email protected]> >>> [mailto:[email protected] >>> <mailto:[email protected]>] On Behalf Of >>> [email protected] >>> <mailto:[email protected]> >>> Sent: 05 July 2017 8:00 PM >>> To: [email protected] <mailto:[email protected]> >>> Subject: Ntop Digest, Vol 158, Issue 1 >>> >>> Send Ntop mailing list submissions to >>> [email protected] <mailto:[email protected]> >>> >>> To subscribe or unsubscribe via the World Wide Web, visit >>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>> or, via email, send a message with subject or body 'help' to >>> [email protected] >>> <mailto:[email protected]> >>> >>> You can reach the person managing the list at >>> [email protected] >>> <mailto:[email protected]> >>> >>> When replying, please edit your Subject line so it is more >>> specific than "Re: Contents of Ntop digest..." >>> >>> >>> Today's Topics: >>> >>> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi) >>> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi) >>> >>> >>> ---------------------------------------------------------------------- >>> >>> Message: 1 >>> Date: Tue, 4 Jul 2017 14:25:19 +0200 >>> From: Simone Mainardi <[email protected] <mailto:[email protected]>> >>> To: [email protected] <mailto:[email protected]> >>> Cc: "[email protected] >>> <mailto:[email protected]>" <[email protected] >>> <mailto:[email protected]>> >>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA >>> Message-ID: >>> >>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com >>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>> >>> Content-Type: text/plain; charset="utf-8" >>> >>> Pelham, >>> >>> ASA flow-updates are received and processed by nprobe. However, >>> I am not sure they contain all the necessary information >>> required to properly update flow statistics. Can you please >>> generate and send a .pcap capture file of your ASA netflow (make >>> sure it contains both templates and data records for >>> flow-updates and flow-teardown) for our inspection? >>> >>> By the way, nprobe gives you a couple of configurable timeout >>> that you can use to periodically export long-lived flows: >>> >>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum >>> (seconds) >>> flow >>> | lifetime [default=120] >>> [--idle-timeout|-d] <timeout> | It specifies the maximum >>> (seconds) >>> flow >>> | idle lifetime [default=30] >>> >>> Regards, >>> Simone >>> >>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < >>> [email protected] >>> <mailto:[email protected]>> wrote: >>> >>> > Hello, >>> > >>> > >>> > >>> > I have a Cisco ASA configured to send Netflow to an instance of >>> > nprobe, and then on to ntopng. >>> > >>> > The configuration is working, however I have noticed that >>> nprobe is >>> > only emitting flows when it receives a flow-teardown event >>> from the >>> > ASA. This is causing inaccurate bandwidth reporting for long-lived >>> > flows as the total byte count is being recorded as a single >>> spike once the flow is torn down. >>> > >>> > >>> > >>> > My understanding is that Cisco ASA netflow is very >>> non-standard and >>> > that this behaviour used to be expected on older version of ASA. >>> > However, newer versions of ASA are capable of sending flow-update >>> > events using a refresh-interval for active flows. When I run >>> tcpdump >>> > on my nprobe server I can see the flow-create and flow-update >>> events >>> > being sent from the ASA, however nprobe does not seem to use these >>> > events, or act on them in any way. I have enabled verbose >>> logging, but >>> > can only see logs being generated for flow-teardown events, >>> not flow-create or flow-update. >>> > >>> > >>> > >>> > My question is, should I expect nprobe to use the flow-updates >>> from >>> > Cisco ASA for long-lived active flows, or is it normal for it >>> to only >>> > process flow-teardown events? >>> > >>> > >>> > >>> > Nprobe (dev build v.8.1.170626) is running in collector mode >>> with the >>> > following settings: >>> > >>> > >>> > >>> > --zmq="tcp://*:5559" >>> > >>> > --collector-port=2055 >>> > >>> > -i=none >>> > >>> > -n=none >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regards, >>> > >>> > Pelham >>> > >>> > >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > This electronic mail is solely for the use of the addressee >>> and may >>> > contain information which is confidential or privileged. If you >>> > receive this electronic mail in error, please delete it from your >>> > system immediately and notify the sender by electronic mail. Any >>> > opinion expressed in this email is not represented as the >>> opinion of >>> > Australian Communication Limited unless that is stated or >>> apparent from its terms. >>> > ------------------------------ >>> > >>> > >>> > _______________________________________________ >>> > Ntop mailing list >>> > [email protected] <mailto:[email protected]> >>> > http://listgateway.unipi.it/mailman/listinfo/ntop >>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>> > >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: >>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm >>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>> >>> >>> ------------------------------ >>> >>> Message: 2 >>> Date: Tue, 4 Jul 2017 14:25:19 +0200 >>> From: Simone Mainardi <[email protected] <mailto:[email protected]>> >>> To: [email protected] <mailto:[email protected]> >>> Cc: "[email protected] >>> <mailto:[email protected]>" <[email protected] >>> <mailto:[email protected]>> >>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA >>> Message-ID: >>> >>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com >>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>> >>> Content-Type: text/plain; charset="utf-8" >>> >>> Pelham, >>> >>> ASA flow-updates are received and processed by nprobe. However, >>> I am not sure they contain all the necessary information >>> required to properly update flow statistics. Can you please >>> generate and send a .pcap capture file of your ASA netflow (make >>> sure it contains both templates and data records for >>> flow-updates and flow-teardown) for our inspection? >>> >>> By the way, nprobe gives you a couple of configurable timeout >>> that you can use to periodically export long-lived flows: >>> >>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum >>> (seconds) >>> flow >>> | lifetime [default=120] >>> [--idle-timeout|-d] <timeout> | It specifies the maximum >>> (seconds) >>> flow >>> | idle lifetime [default=30] >>> >>> Regards, >>> Simone >>> >>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < >>> [email protected] >>> <mailto:[email protected]>> wrote: >>> >>> > Hello, >>> > >>> > >>> > >>> > I have a Cisco ASA configured to send Netflow to an instance of >>> > nprobe, and then on to ntopng. >>> > >>> > The configuration is working, however I have noticed that >>> nprobe is >>> > only emitting flows when it receives a flow-teardown event >>> from the >>> > ASA. This is causing inaccurate bandwidth reporting for long-lived >>> > flows as the total byte count is being recorded as a single >>> spike once the flow is torn down. >>> > >>> > >>> > >>> > My understanding is that Cisco ASA netflow is very >>> non-standard and >>> > that this behaviour used to be expected on older version of ASA. >>> > However, newer versions of ASA are capable of sending flow-update >>> > events using a refresh-interval for active flows. When I run >>> tcpdump >>> > on my nprobe server I can see the flow-create and flow-update >>> events >>> > being sent from the ASA, however nprobe does not seem to use these >>> > events, or act on them in any way. I have enabled verbose >>> logging, but >>> > can only see logs being generated for flow-teardown events, >>> not flow-create or flow-update. >>> > >>> > >>> > >>> > My question is, should I expect nprobe to use the flow-updates >>> from >>> > Cisco ASA for long-lived active flows, or is it normal for it >>> to only >>> > process flow-teardown events? >>> > >>> > >>> > >>> > Nprobe (dev build v.8.1.170626) is running in collector mode >>> with the >>> > following settings: >>> > >>> > >>> > >>> > --zmq="tcp://*:5559" >>> > >>> > --collector-port=2055 >>> > >>> > -i=none >>> > >>> > -n=none >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > Regards, >>> > >>> > Pelham >>> > >>> > >>> > >>> > >>> > >>> > ------------------------------ >>> > >>> > This electronic mail is solely for the use of the addressee >>> and may >>> > contain information which is confidential or privileged. If you >>> > receive this electronic mail in error, please delete it from your >>> > system immediately and notify the sender by electronic mail. Any >>> > opinion expressed in this email is not represented as the >>> opinion of >>> > Australian Communication Limited unless that is stated or >>> apparent from its terms. >>> > ------------------------------ >>> > >>> > >>> > _______________________________________________ >>> > Ntop mailing list >>> > [email protected] <mailto:[email protected]> >>> > http://listgateway.unipi.it/mailman/listinfo/ntop >>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>> > >>> -------------- next part -------------- >>> An HTML attachment was scrubbed... >>> URL: >>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm >>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>> >>> >>> ------------------------------ >>> >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] <mailto:[email protected]> >>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>> >>> End of Ntop Digest, Vol 158, Issue 1 >>> ************************************ >>> This electronic mail is solely for the use of the addressee and >>> may contain information which is confidential or privileged. >>> If you receive this electronic mail in error, please delete it >>> from your system immediately and notify the sender by electronic >>> mail. >>> Any opinion expressed in this email is not represented as the >>> opinion of Australian Communication Exchange Limited unless that >>> is stated or >>> apparent from its terms. >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] <mailto:[email protected]> >>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>> >>> >>> >>> >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop >> >> _______________________________________________ >> Ntop mailing list >> [email protected] <mailto:[email protected]> >> http://listgateway.unipi.it/mailman/listinfo/ntop > > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop