Re: Traffic sent and traffic received in historical view in ntopng

Andrew,

Please see below.


On 05/17/2017 07:17 PM, Andrew Hilborne wrote:
> On 16 May 2017 at 19:29, Emanuele Faranda faranda-at-ntop.org 
> <http://faranda-at-ntop.org> |ntop-flugle| <[email protected] 
> <mailto:[email protected]>>wrote:
>
>     Hi Peter and Andrew,
>
>     Please see below.
>
>         On 05/16/2017 01:33 PM, Peter Shute wrote:
>         I'm also interested in this. I can get from my ISP daily
>         totals for our internet usage. I would like ntopng to be able
>         to replicate those daily totals (to give confidence our data
>         is correct), and then analyse the totals to see which devices
>         contributed. Eg. If we have an above average daily total, I
>         want to know why.
>         ​​
>
>     Ntopng can actually produce a traffic report where it shows the
>     top local/remote talkers for a specified time frame (e.g. a day),
>     but this is a pro only feature.
>
>
> ​I don't object to paying for the license, but this still doesn't get 
> me what I want, I think. I want to look back over a historical graph 
> (or jump to a time and day, if I believe there was a problem at that 
> time) and drill down to see the protocols invloved, and the hosts. Top 
> 10 talkers may not include the information I want.​
The most accurate information you can get is via MySQL data (-F option).

I take into account your use use case: you view a local network traffic 
graph and see a peak at 5 am. of the last day and want to know which 
hosts are involved. You double click on the graph to restrict the time 
frame so that a 10 minutes range is selected and 5 am is centered on the 
graph.

Now, if hover the mouse on the graph you will see the top talkers at 5 
am. From the top talkers panel, you can click the historical icon 
(http://fontawesome.io/icon/history/) to access the MySQL data specific 
to that host, and drill down its flows and protocols for that particular 
time frame.

You can also click on the graph historical icon to get an overview of 
all the flows, but you cannot aggregate per host in this way.

What I feel is missing is:
1) an aggregated view of the top protocols on the graph
2) an easy way from the historical explorer to aggregate per host or per 
protocol to be able to see and sort bewteen accurate statistics

>>     On 16 May 2017, at 4:01 am, Andrew Hilborne <[email protected]
>>     <mailto:[email protected]><mailto:[email protected]>
>>     <mailto:[email protected]>> wrote:
>>     ​​
>>
>>     On 15 May 2017 at 17:10, Emanuele Farandafaranda-at-ntop.org
>>     <http://faranda-at-ntop.org><http://faranda-at-ntop.org>
>>     <http://faranda-at-ntop.org>  |ntop-flugle| <[email protected]
>>     <mailto:[email protected]><mailto:[email protected]>
>>     <mailto:[email protected]>> wrote:
>>
>>     You are right, network stats are calculated every minute, whereas interface stats are updated each second.
>>
>>     Please note that these stats are dumped to RRD files, not to the MySQL database.
>>
>>     ​Would it be possible t change this? Is the issue storage space in the MySQL database? This is what I want to know (initially):
>     No, this is something different. MySQL database exports /flows/ as
>     data, whereas RRD is /timeseries/ database, so they play different
>     roles.
>
>
> ​I do understand the difference between flows and and the RDD 
> timeseries. However, typical 5-minute RDD data is useless for 
> investigating traffic peaks; I think you may know this, because n2disk 
> can now detect 'micro-bursts'​. I don't suggest that storing 
> sufficient information to provide a near real-time breakdown of 
> traffic is easy, but it would be interesting. If I am reduced to going 
> back to using RDDtool type data, there are better tools than ntopng 
> for that purpose.
>
> Maybe I'm not using ntopng properly?
Interface traffic statistics are stored with 1 second resolution, 
whereas network traffic statistics with 1 minute resolution 
(ingress/egress not the protocols, which are dumped each 5 minutes). 
It's a trade off between space/time taken for data dump and time 
resolution you get. The idea is that raw data is kept in MySQL database, 
so this is where you land when you need precise data.

We know there is room for improvements, and we appreciate our users 
feedback. So please, if you feel there is a use case interesting which 
is not covered/could be better implemented into ntopng, open a feature 
request on our github page https://github.com/ntop/ntopng .

Please see also these links:
http://www.ntop.org/ntopng/exploring-historical-data-using-ntopng/
http://www.ntop.org/ntopng/exploring-historical-data-using-ntopng-part-2/

Emanuele

>
> Andrew
>
>
> _______________________________________________
> Ntop mailing list
> [email protected]
> http://listgateway.unipi.it/mailman/listinfo/ntop

_______________________________________________
Ntop mailing list
[email protected]
http://listgateway.unipi.it/mailman/listinfo/ntop