Re: starting ntop errors and warnings
Simone Mainardi <[email protected]>
| Newsgroups | gmane.linux.ntop.general |
|---|---|
| Message-ID | <[email protected]> |
Spiros, > On 4 Sep 2017, at 21:48, Spiros Papageorgiou <[email protected]> wrote: > > On 4/9/2017 10:47 πμ, Simone Mainardi wrote: >> Spiros, >> >>> On 2 Sep 2017, at 06:43, Spiros Papageorgiou <[email protected] <mailto:[email protected]>> wrote: >>> >>> Hi Luca, >>> >>> I updated everything and the error messages seem to be gone. I can see that huge pages have been allocated: >>> >>> # cat /sys/devices/system/node/node*/meminfo | grep Huge F10Quit >>> Node 0 AnonHugePages: 100352 kB >>> Node 0 HugePages_Total: 2048 >>> Node 0 HugePages_Free: 1852 >>> Node 0 HugePages_Surp: 0 >>> >>> Thanx! >>> >>> Two more problems: >>> - What about the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" which is not working? Is the syntax wrong or something? >> >> What do you mean with 'is not working'? So you get empty pages? Please, report the issue using our GitHub issue tracker. >> >>> - I get "02/Sep/2017 07:34:50 [Lua.cpp:5629] [@/usr/share/ntopng/scripts/callbacks/5min.lua]:[161] ERROR: Cannot complete local hosts RRD dump in 5 minutes. Please check your RRD configuration.". Do I need a stronger server? I have a 4core VM and the disk (SAN) seems not to be stressed that much. The CPU though is at about 70%. >> >> This means that you have too many local hosts that there's not enough time to update their statistics in 5 minutes. How many local hosts do you have? Disabling data retention of L7 timeseries may alleviate the issue. >> The bottleneck here are the several multiple RRD updates made for every local host. > Ntopng reports about 10K local hosts and 50K remote. > How can i disable L7 timeseries data retention? Can I keep stats for only local hosts? What kind of control do i have on that aspect? See the ntopng preferences page, tab "Data Retention". > Is it possible to use somekind of caching, like rrdcache? no > >> >>> - The same goes when I export to ES. The ES is at about 30% CPU utilization but ntopng reports massive flow drops. >> >> ElasticSearch is not fast enough to inject flows at the required speed. Hence ntopng flows export queue grows and at some point it overflows resulting in flows drops. Other people experienced this in the past. You must configure your ES deployment to make sure it has enough intake. >> >>> >>> Regards, >>> Spiros >>> >>> >>> On 1/9/2017 9:26 πμ, Luca Deri wrote: >>>> Spiros >>>> can you please update ntopng and report? >>>> >>>> Thanks Luca >>>> >>>>> On 31 Aug 2017, at 20:10, Spiros Papageorgiou <[email protected] <mailto:[email protected]>> wrote: >>>>> >>>>> Hi Alfredo, >>>>> >>>>> Here is the output: >>>>> [root@server ~]# cat /proc/mounts | grep huge >>>>> cgroup /sys/fs/cgroup/hugetlb cgroup rw,nosuid,nodev,noexec,relatime,hugetlb 0 0 >>>>> hugetlbfs /dev/hugepages hugetlbfs rw,relatime 0 0 >>>>> >>>>> [root@server ~]# cat /sys/devices/system/node/node*/meminfo | grep Huge >>>>> Node 0 AnonHugePages: 2488320 kB >>>>> Node 0 HugePages_Total: 2048 >>>>> Node 0 HugePages_Free: 2048 >>>>> Node 0 HugePages_Surp: 0 >>>>> [root@server ~]# >>>>> >>>>> Regards, >>>>> Sp >>>>> >>>>> On 31/8/2017 8:06 μμ, Alfredo Cardigliano wrote: >>>>>> Hi Spiros >>>>>> please provide the output of: >>>>>> >>>>>> cat /proc/mounts / grep huge >>>>>> cat /sys/devices/system/node/node*/meminfo | grep Huge >>>>>> >>>>>> Alfredo >>>>>> >>>>>>> On 31 Aug 2017, at 18:53, Spiros Papageorgiou <[email protected]> <mailto:[email protected]> wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> >>>>>>> I have an updated ntopng/pfring-zc installation and when I start ntopng the following messages are produced: >>>>>>> >>>>>>> Aug 30 18:05:06 server logger: ntopng start >>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2] >>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2] >>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2] >>>>>>> Aug 30 18:05:08 server ntopng: [ViewInterface.cpp:50] ERROR: Internal Error: NULL interface [view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3][2] >>>>>>> Aug 30 18:05:08 server kernel: device ens160 entered promiscuous mode >>>>>>> Aug 30 18:05:09 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:05:11 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:05:11 server ntopng: Starting ntopng: Unable to start ntopng[FAILED] >>>>>>> Aug 30 18:05:11 server systemd: Started Start/stop ntopng program. >>>>>>> Aug 30 18:05:12 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:05:14 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:05:14 server kernel: traps: ntopng[2575] general protection ip:42df01 sp:7ff418f69b60 error:0 in ntopng[400000+264000] >>>>>>> Aug 30 18:05:15 server kernel: device ens160 left promiscuous mode >>>>>>> Aug 30 18:05:15 server systemd: ntopng.service: main process exited, code=killed, status=11/SEGV >>>>>>> >>>>>>> Why the view of all queues does not work? How should i write this line? >>>>>>> >>>>>>> When I delete from ntopng.conf, the "view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3" line, the I get another bunch of warnings: >>>>>>> >>>>>>> Aug 30 18:06:50 server systemd: Starting Start/stop ntopng program... >>>>>>> Aug 30 18:06:50 server logger: ntopng start >>>>>>> Aug 30 18:06:51 server kernel: device ens160 entered promiscuous mode >>>>>>> Aug 30 18:06:52 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:06:53 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:06:55 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:06:55 server ntopng: Starting ntopng: Unable to start ntopng[FAILED] >>>>>>> Aug 30 18:06:55 server systemd: Started Start/stop ntopng program. >>>>>>> Aug 30 18:06:56 server ntopng: [PF_RINGInterface.cpp:72] WARNING: Unable to set packet capture direction >>>>>>> Aug 30 18:06:57 server ZC[2647]: error opening /dev/hugepages/ens160@1-txrx: Permission denied >>>>>>> Aug 30 18:06:57 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB >>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@0-txrx: Permission denied >>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB >>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@2-txrx: Permission denied >>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB >>>>>>> Aug 30 18:06:58 server ZC[2647]: error opening /dev/hugepages/ens160@3-txrx: Permission denied >>>>>>> Aug 30 18:06:58 server ZC[2647]: error mmap'ing 49 hugepages of 2048 KB >>>>>>> Aug 30 18:07:01 server systemd: Started Session 6 of user root. >>>>>>> Aug 30 18:07:01 server systemd: Starting Session 6 of user root. >>>>>>> >>>>>>> The ntopng then, seems to be working but I would like to get rid of the errors. What should I do to avoid the hugepages errors? I'm running ntop as root (though I can see it drops privs). >>>>>>> >>>>>>> My conf: >>>>>>> >>>>>>> -G=/var/run/ntopng.pid >>>>>>> --data-dir=/mnt/data/ntopng >>>>>>> #--interface=zc:ens160 >>>>>>> --interface=zc:ens160@0 >>>>>>> --interface=zc:ens160@1 >>>>>>> --interface=zc:ens160@2 >>>>>>> --interface=zc:ens160@3 >>>>>>> #--interface=view:zc:ens160@0,zc:ens160@1,zc:ens160@2,zc:ens160@3 >>>>>>> --local-networks="xx.xx.0.0/16,xx.xx.cc <http://xx.xx.cc/>.0/24,xx.xx.ff.0/22,xx.xx.vv.0/24" >>>>>>> --dns-mode=3 >>>>>>> --disable-autologout >>>>>>> --max-num-flows=512000 >>>>>>> --max-num-hosts=256000 >>>>>>> --sticky-hosts=none >>>>>>> -g -1 >>>>>>> >>>>>>> # cat /etc/ntopng/ntopng.start >>>>>>> # >>>>>>> >>>>>>> # cat /etc/pf_ring/hugepages.conf >>>>>>> node=0 hugepages=1024 >>>>>>> # cat /etc/pf_ring/pf_ring.conf >>>>>>> transparent_mode=2 >>>>>>> >>>>>>> # cat /etc/pf_ring/pf_ring.start >>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.conf >>>>>>> RSS=0,0,0,0 >>>>>>> # cat /etc/pf_ring/zc/ixgbe/ixgbe.start >>>>>>> # >>>>>>> >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Spiros >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Ntop mailing list >>>>>>> [email protected] <mailto:[email protected]> >>>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Ntop mailing list >>>>>> [email protected] <mailto:[email protected]> >>>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>>> _______________________________________________ >>>>> Ntop mailing list >>>>> [email protected] <mailto:[email protected]> >>>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop> >>>> >>>> >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] <mailto:[email protected]> >>>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop> >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] <mailto:[email protected]> >>> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop> >> >> >> _______________________________________________ >> Ntop mailing list >> [email protected] <mailto:[email protected]> >> http://listgateway.unipi.it/mailman/listinfo/ntop <http://listgateway.unipi.it/mailman/listinfo/ntop> > _______________________________________________ > Ntop mailing list > [email protected] > http://listgateway.unipi.it/mailman/listinfo/ntop _______________________________________________ Ntop mailing list [email protected] http://listgateway.unipi.it/mailman/listinfo/ntop