Re: Netflow (NSEL) updates from Cisco ASA
Marat Khalili <[email protected]>
| Newsgroups | gmane.linux.ntop.general |
|---|---|
| Organization | Russian Quantum Center |
| Message-ID | <[email protected]> |
Hello Luca, I've opened an issue on github as you requested: https://github.com/ntop/ntopng/issues/1359 . Please contact me off-list if you need full .pcap or other data. -- With Best Regards, Marat Khalili On 13/07/17 10:27, Marat Khalili wrote: > >> ASA (and other devices like PaloAlto) generate flows based on >> firewall events, like flow create, deletion etc. and do not send >> interim updates as flow devices do. ASA for instance sends only total >> octets. > According to documentation > <https://www.cisco.com/c/en/us/td/docs/security/asa/special/netflow/guide/asa_netflow.html#pgfId-1324322> > there are Flow-Update events sent every minute by default. >> Shall you have further comments on this matter, please file ticket on >> github so we can keep a log of the discussion > Ok, I'll re-test it my environment and open the ticket with what I'm > seeing. > -- > > With Best Regards, > Marat Khalili > > On 13/07/17 01:26, Luca Deri wrote: >> Hi all, >> ASA (and other devices like PaloAlto) generate flows based on >> firewall events, like flow create, deletion etc. and do not send >> interim updates as flow devices do. ASA for instance sends only total >> octets. >> >> Due to this if we take into account flow updates we need to make sure >> that the frequency of these updates is greater than the flow >> inactivity timeout in ntopng as otherwise we would generate incorrect >> values (i.e. we will account more traffic than the one that has >> really passed on the network). The flow delete instead is not >> affected by this problem as it is the very last message emitted for a >> given flow. >> >> This said, reading this thread it seems that you all have different >> option and this I have modified nProbe to implement the behaviour you >> expect. Shall you have further comments on this matter, please file >> ticket on github so we can keep a log of the discussion >> >> Regards Luca >> >>> On 6 Jul 2017, at 14:13, Marat Khalili <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Hello Simone, >>> >>> I'm also very interested in this issue, observing same behaviour as >>> Pelham. Do you mean we need a custom fields configuration file to >>> make this work? How will it be different from your attempts where >>> "events other than flow-delete often contain to few attributes to >>> properly update the flow"? >>> >>> -- >>> >>> With Best Regards, >>> Marat Khalili >>> >>> On 06/07/17 14:54, Simone Mainardi wrote: >>>> Dear Pelham, >>>> >>>> Thanks for sharing the pcap. Currently, in nProbe we deliberately >>>> ignore ASA firewall events different from flow-delete. We have made >>>> this choice as we have seen that events other than flow-delete >>>> often contain to few attributes to properly update the flow. This >>>> is the reason why other templates are silently ignored by nprobe. >>>> >>>> If you want your nProbe to proxy all the templates received as-is, >>>> you may want to have a look at: >>>> http://www.ntop.org/nprobe/collecting-proprietary-flows-with-nprobe/ >>>> >>>> >>>> Simone >>>> >>>> Regards, >>>> Simone >>>> >>>> >>>> >>>> On Thu, Jul 6, 2017 at 3:55 AM, Pelham Whitmore >>>> <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hey Simone, >>>> >>>> Thanks for the reply. I have configured the timeout values you >>>> mentioned in nprobe however it seemed to have no effect. >>>> I have generated a .pcap file that includes templates, flow >>>> create, update, and teardown events. >>>> >>>> .pcap file can be downloaded from here: >>>> https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download >>>> <https://files.aceinfo.net.au/index.php/s/bj0aFU3lpyUjqM4/download> >>>> >>>> One thing I did notcie from the packet capture is that flow >>>> update events are being listed as "Firewall Event: Unknown (5)" . >>>> I'm not sure if that is to be expected. >>>> >>>> >>>> Regards, >>>> Pelham Whitmore >>>> >>>> >>>> -----Original Message----- >>>> From: [email protected] >>>> <mailto:[email protected]> >>>> [mailto:[email protected] >>>> <mailto:[email protected]>] On Behalf Of >>>> [email protected] >>>> <mailto:[email protected]> >>>> Sent: 05 July 2017 8:00 PM >>>> To: [email protected] <mailto:[email protected]> >>>> Subject: Ntop Digest, Vol 158, Issue 1 >>>> >>>> Send Ntop mailing list submissions to >>>> [email protected] <mailto:[email protected]> >>>> >>>> To subscribe or unsubscribe via the World Wide Web, visit >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>> or, via email, send a message with subject or body 'help' to >>>> [email protected] >>>> <mailto:[email protected]> >>>> >>>> You can reach the person managing the list at >>>> [email protected] >>>> <mailto:[email protected]> >>>> >>>> When replying, please edit your Subject line so it is more >>>> specific than "Re: Contents of Ntop digest..." >>>> >>>> >>>> Today's Topics: >>>> >>>> 1. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi) >>>> 2. Re: Netflow (NSEL) updates from Cisco ASA (Simone Mainardi) >>>> >>>> >>>> ---------------------------------------------------------------------- >>>> >>>> Message: 1 >>>> Date: Tue, 4 Jul 2017 14:25:19 +0200 >>>> From: Simone Mainardi <[email protected] >>>> <mailto:[email protected]>> >>>> To: [email protected] <mailto:[email protected]> >>>> Cc: "[email protected] >>>> <mailto:[email protected]>" <[email protected] >>>> <mailto:[email protected]>> >>>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA >>>> Message-ID: >>>> >>>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com >>>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>> >>>> Content-Type: text/plain; charset="utf-8" >>>> >>>> Pelham, >>>> >>>> ASA flow-updates are received and processed by nprobe. However, >>>> I am not sure they contain all the necessary information >>>> required to properly update flow statistics. Can you please >>>> generate and send a .pcap capture file of your ASA netflow >>>> (make sure it contains both templates and data records for >>>> flow-updates and flow-teardown) for our inspection? >>>> >>>> By the way, nprobe gives you a couple of configurable timeout >>>> that you can use to periodically export long-lived flows: >>>> >>>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum >>>> (seconds) >>>> flow >>>> | lifetime [default=120] >>>> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds) >>>> flow >>>> | idle lifetime [default=30] >>>> >>>> Regards, >>>> Simone >>>> >>>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < >>>> [email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> > Hello, >>>> > >>>> > >>>> > >>>> > I have a Cisco ASA configured to send Netflow to an instance of >>>> > nprobe, and then on to ntopng. >>>> > >>>> > The configuration is working, however I have noticed that >>>> nprobe is >>>> > only emitting flows when it receives a flow-teardown event >>>> from the >>>> > ASA. This is causing inaccurate bandwidth reporting for >>>> long-lived >>>> > flows as the total byte count is being recorded as a single >>>> spike once the flow is torn down. >>>> > >>>> > >>>> > >>>> > My understanding is that Cisco ASA netflow is very >>>> non-standard and >>>> > that this behaviour used to be expected on older version of ASA. >>>> > However, newer versions of ASA are capable of sending flow-update >>>> > events using a refresh-interval for active flows. When I run >>>> tcpdump >>>> > on my nprobe server I can see the flow-create and flow-update >>>> events >>>> > being sent from the ASA, however nprobe does not seem to use >>>> these >>>> > events, or act on them in any way. I have enabled verbose >>>> logging, but >>>> > can only see logs being generated for flow-teardown events, >>>> not flow-create or flow-update. >>>> > >>>> > >>>> > >>>> > My question is, should I expect nprobe to use the >>>> flow-updates from >>>> > Cisco ASA for long-lived active flows, or is it normal for it >>>> to only >>>> > process flow-teardown events? >>>> > >>>> > >>>> > >>>> > Nprobe (dev build v.8.1.170626) is running in collector mode >>>> with the >>>> > following settings: >>>> > >>>> > >>>> > >>>> > --zmq="tcp://*:5559" >>>> > >>>> > --collector-port=2055 >>>> > >>>> > -i=none >>>> > >>>> > -n=none >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > Regards, >>>> > >>>> > Pelham >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > ------------------------------ >>>> > >>>> > This electronic mail is solely for the use of the addressee >>>> and may >>>> > contain information which is confidential or privileged. If you >>>> > receive this electronic mail in error, please delete it from your >>>> > system immediately and notify the sender by electronic mail. Any >>>> > opinion expressed in this email is not represented as the >>>> opinion of >>>> > Australian Communication Limited unless that is stated or >>>> apparent from its terms. >>>> > ------------------------------ >>>> > >>>> > >>>> > _______________________________________________ >>>> > Ntop mailing list >>>> > [email protected] <mailto:[email protected]> >>>> > http://listgateway.unipi.it/mailman/listinfo/ntop >>>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>> > >>>> -------------- next part -------------- >>>> An HTML attachment was scrubbed... >>>> URL: >>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm >>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0002.htm>> >>>> >>>> ------------------------------ >>>> >>>> Message: 2 >>>> Date: Tue, 4 Jul 2017 14:25:19 +0200 >>>> From: Simone Mainardi <[email protected] >>>> <mailto:[email protected]>> >>>> To: [email protected] <mailto:[email protected]> >>>> Cc: "[email protected] >>>> <mailto:[email protected]>" <[email protected] >>>> <mailto:[email protected]>> >>>> Subject: Re: [Ntop] Netflow (NSEL) updates from Cisco ASA >>>> Message-ID: >>>> >>>> <CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com >>>> <mailto:CAJcXkCDHiqCnpXrwNX3sOS6mCSzRKMnANacBUXCjz96iYNzdFg@mail.gmail.com>> >>>> Content-Type: text/plain; charset="utf-8" >>>> >>>> Pelham, >>>> >>>> ASA flow-updates are received and processed by nprobe. However, >>>> I am not sure they contain all the necessary information >>>> required to properly update flow statistics. Can you please >>>> generate and send a .pcap capture file of your ASA netflow >>>> (make sure it contains both templates and data records for >>>> flow-updates and flow-teardown) for our inspection? >>>> >>>> By the way, nprobe gives you a couple of configurable timeout >>>> that you can use to periodically export long-lived flows: >>>> >>>> [--lifetime-timeout|-t] <timeout> | It specifies the maximum >>>> (seconds) >>>> flow >>>> | lifetime [default=120] >>>> [--idle-timeout|-d] <timeout> | It specifies the maximum (seconds) >>>> flow >>>> | idle lifetime [default=30] >>>> >>>> Regards, >>>> Simone >>>> >>>> On Wed, Jun 28, 2017 at 2:38 AM, Pelham Whitmore < >>>> [email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> > Hello, >>>> > >>>> > >>>> > >>>> > I have a Cisco ASA configured to send Netflow to an instance of >>>> > nprobe, and then on to ntopng. >>>> > >>>> > The configuration is working, however I have noticed that >>>> nprobe is >>>> > only emitting flows when it receives a flow-teardown event >>>> from the >>>> > ASA. This is causing inaccurate bandwidth reporting for >>>> long-lived >>>> > flows as the total byte count is being recorded as a single >>>> spike once the flow is torn down. >>>> > >>>> > >>>> > >>>> > My understanding is that Cisco ASA netflow is very >>>> non-standard and >>>> > that this behaviour used to be expected on older version of ASA. >>>> > However, newer versions of ASA are capable of sending flow-update >>>> > events using a refresh-interval for active flows. When I run >>>> tcpdump >>>> > on my nprobe server I can see the flow-create and flow-update >>>> events >>>> > being sent from the ASA, however nprobe does not seem to use >>>> these >>>> > events, or act on them in any way. I have enabled verbose >>>> logging, but >>>> > can only see logs being generated for flow-teardown events, >>>> not flow-create or flow-update. >>>> > >>>> > >>>> > >>>> > My question is, should I expect nprobe to use the >>>> flow-updates from >>>> > Cisco ASA for long-lived active flows, or is it normal for it >>>> to only >>>> > process flow-teardown events? >>>> > >>>> > >>>> > >>>> > Nprobe (dev build v.8.1.170626) is running in collector mode >>>> with the >>>> > following settings: >>>> > >>>> > >>>> > >>>> > --zmq="tcp://*:5559" >>>> > >>>> > --collector-port=2055 >>>> > >>>> > -i=none >>>> > >>>> > -n=none >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > Regards, >>>> > >>>> > Pelham >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > ------------------------------ >>>> > >>>> > This electronic mail is solely for the use of the addressee >>>> and may >>>> > contain information which is confidential or privileged. If you >>>> > receive this electronic mail in error, please delete it from your >>>> > system immediately and notify the sender by electronic mail. Any >>>> > opinion expressed in this email is not represented as the >>>> opinion of >>>> > Australian Communication Limited unless that is stated or >>>> apparent from its terms. >>>> > ------------------------------ >>>> > >>>> > >>>> > _______________________________________________ >>>> > Ntop mailing list >>>> > [email protected] <mailto:[email protected]> >>>> > http://listgateway.unipi.it/mailman/listinfo/ntop >>>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>> > >>>> -------------- next part -------------- >>>> An HTML attachment was scrubbed... >>>> URL: >>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm >>>> <http://listgateway.unipi.it/pipermail/ntop/attachments/20170704/76008a4b/attachment-0003.htm>> >>>> >>>> ------------------------------ >>>> >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] <mailto:[email protected]> >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>> >>>> End of Ntop Digest, Vol 158, Issue 1 >>>> ************************************ >>>> This electronic mail is solely for the use of the addressee and >>>> may contain information which is confidential or privileged. >>>> If you receive this electronic mail in error, please delete it >>>> from your system immediately and notify the sender by >>>> electronic mail. >>>> Any opinion expressed in this email is not represented as the >>>> opinion of Australian Communication Exchange Limited unless >>>> that is stated or >>>> apparent from its terms. >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] <mailto:[email protected]> >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>>> <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] <mailto:[email protected]> >>> http://listgateway.unipi.it/mailman/listinfo/ntop >> >> >> >> _______________________________________________ >> Ntop mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop > > > > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop