Re: How to Analyse MikroTik Traffic Using nprobe -> ntopng on Windows
[email protected] Mon, 24 Dec 2018 23:32:29 +0200
| Newsgroups | gmane.linux.ntop.general |
|---|---|
| Message-ID | <[email protected]> |
Update to prev mail: Starting ntopng with: ntopng /c -i tcp://*:5556c and nprobe with: nprobe /c --zmq "tcp://127.0.0.1:5556" --zmq-probe-mode -i none -n none --collector-port 2055 -T "@NTOPNG@" Results in traffic being parsed to GUI running on: http://127.0.0.1:3000/lua/hosts_stats.lua However when selecting Hosts, Filter Hosts, Local Hosts, it outputs: No results found, yet we can see some of our local IPs listed under Hosts (main menu). Our primary requirement right now is analyzing / recording LAN users internet bandwidth usage. C:\Program Files\ntopng>ntopng /c -i tcp://*:5556c =================================================================== Starting ntopng Running ntopng. 24/Dec/2018 23:26:30 [Ntop.cpp:1545] Setting local networks to 127.0.0.0/8 24/Dec/2018 23:26:30 [Redis.cpp:132] Successfully connected to redis 127.0.0.1@0 24/Dec/2018 23:26:30 [Redis.cpp:132] Successfully connected to redis 127.0.0.1@0 24/Dec/2018 23:26:30 [NtopPro.cpp:310] [LICENSE] Reading license from Redis 24/Dec/2018 23:26:30 [NtopPro.cpp:451] WARNING: [LICENSE] Invalid or missing license 24/Dec/2018 23:26:30 [NtopPro.cpp:468] WARNING: [LICENSE] ntopng will now run in enterprise edition for 10 minutes 24/Dec/2018 23:26:30 [NtopPro.cpp:470] WARNING: [LICENSE] before returning to community mode 24/Dec/2018 23:26:30 [NtopPro.cpp:472] WARNING: [LICENSE] You can buy a permanent license at http://shop.ntop.org 24/Dec/2018 23:26:30 [NtopPro.cpp:474] WARNING: [LICENSE] or run ntopng in community mode starting 24/Dec/2018 23:26:30 [NtopPro.cpp:475] WARNING: [LICENSE] ntopng --community 24/Dec/2018 23:26:30 [Ntop.cpp:1639] Registered interface tcp://*:5556c [id: 9] 24/Dec/2018 23:26:31 [HTTPserver.cpp:945] HTTPS Disabled: missing SSL certificate C:\Program Files\ntopng\httpdocs/ssl/ntopng-cert.pem 24/Dec/2018 23:26:31 [HTTPserver.cpp:947] Please read https://github.com/ntop/ntopng/blob/dev/doc/README.SSL if you want to enable SSL. 24/Dec/2018 23:26:31 [HTTPserver.cpp:1114] Web server dirs [C:\Program Files\ntopng\httpdocs][C:\Program Files\ntopng\scripts] 24/Dec/2018 23:26:31 [HTTPserver.cpp:1117] HTTP server listening on 3000 24/Dec/2018 23:26:31 [main.cpp:393] Working directory: Z:\Cloud\OneDrive\MyPC\Documents\ntopng 24/Dec/2018 23:26:31 [main.cpp:395] Scripts/HTML pages directory: C:\Program Files\ntopng 24/Dec/2018 23:26:31 [Ntop.cpp:390] Welcome to ntopng x64 v.3.7.180929 - (C) 1998-18 ntop.org 24/Dec/2018 23:26:31 [Ntop.cpp:400] Built on Windows 24/Dec/2018 23:26:31 [NtopPro.cpp:633] [LICENSE] System Id: 2152224034-9206A1D8 24/Dec/2018 23:26:31 [NtopPro.cpp:634] [LICENSE] Edition: Enterprise 24/Dec/2018 23:26:31 [NtopPro.cpp:635] [LICENSE] License Type: Time-Limited License 24/Dec/2018 23:26:31 [NtopPro.cpp:644] [LICENSE] Validity: Until Mon Dec 24 23:36:30 2018 24/Dec/2018 23:26:31 [PeriodicActivities.cpp:68] Started periodic activities loop... 24/Dec/2018 23:26:32 [PeriodicActivities.cpp:109] Each periodic activity script will use 2 threads 24/Dec/2018 23:26:32 [NetworkInterface.cpp:2581] Started packet polling on interface tcp://*:5556c [id: 9]... 24/Dec/2018 23:26:32 [CollectorInterface.cpp:122] Collecting flows on tcp://*:5556c ====================================================================================== C:\Program Files\nProbe>nprobe /c --zmq "tcp://127.0.0.1:5556" --zmq-probe-mode -i none -n none --collector-port 2055 -T "@NTOPNG@" ============================================================================== Running nProbe for Windows. 24/Dec/2018 23:26:40 [nprobe.c:4168] Valid nProbe license found 24/Dec/2018 23:26:40 [nprobe.c:6092] WARNING: The output interfaceId is set to 0: did you forget to use -Q perhaps ? 24/Dec/2018 23:26:40 [nprobe.c:6095] WARNING: The input interfaceId is set to 0: did you forget to use -u perhaps ? 24/Dec/2018 23:26:40 [nprobe.c:6182] Welcome to nProbe v.8.6.181004 ($Revision: 4384 $) for Windows 24/Dec/2018 23:26:40 [nprobe.c:6192] Running on Windows 24/Dec/2018 23:26:40 [nprobe.c:6203] [LICENSE] nProbe SystemId: 2152224034-9206A1D8 24/Dec/2018 23:26:40 [nprobe.c:6270] Sample rate [packet: 1][flow collection/export: 1/1] 24/Dec/2018 23:26:40 [nprobe.c:8966] Welcome to nProbe v.8.6.181004 for Windows 24/Dec/2018 23:26:40 [nprobe.c:7870] WARNING: Adding %EXPORTER_IPV4_ADDRESS to the template as nProbe is working as collector 24/Dec/2018 23:26:40 [nprobe.c:7976] Using NetFlow Packet Payload Len: 1472 24/Dec/2018 23:26:40 [nprobe.c:7906] @NTOPNG@ expanded to " %L7_PROTO %IPV4_SRC_ADDR %IPV4_DST_ADDR %L4_SRC_PORT %L4_DST_PORT %IPV6_SRC_ADDR %IPV6_DST_ADDR %IP_PROTOCOL_VERSION %PROTOCOL %IN_BYTES %IN_PKTS %OUT_BYTES %OUT_PKTS %FIRST_SWITCHED %LAST_SWITCHED %SRC_VLAN %EXPORTER_IPV4_ADDRESS" 24/Dec/2018 23:26:40 [plugin.c:1238] 0 plugin(s) enabled 24/Dec/2018 23:26:40 [nprobe.c:8422] Each flow is 82 bytes long 24/Dec/2018 23:26:40 [nprobe.c:8423] The # flows per packet has been set to 16 24/Dec/2018 23:26:40 [nprobe.c:8426] IP TOS is accounted 24/Dec/2018 23:26:40 [nprobe.c:8452] Non IPv4/v6 traffic is discarded according to the template 24/Dec/2018 23:26:40 [nprobe.c:9231] Flows ASs will not be computed (missing libmxminddb support) 24/Dec/2018 23:26:40 [nprobe.c:9334] Not capturing packet from interface (collector mode) 24/Dec/2018 23:26:40 [util.c:4719] Initializing ZMQ as client 24/Dec/2018 23:26:40 [util.c:4738] Exporting flows towards ZMQ endpoint tcp://127.0.0.1:5556 24/Dec/2018 23:26:40 [collect.c:142] Flow collector listening on port 2055 (IPv4/v6) 24/Dec/2018 23:26:40 [nprobe.c:9582] nProbe started successfully 24/Dec/2018 23:30:26 [nprobe.c:567] Received shutdown request... [signal: 2] 24/Dec/2018 23:30:27 [nprobe.c:6317] Flushing active flows 24/Dec/2018 23:30:27 [engine.c:3169] About to flush hash (threadId 0) 24/Dec/2018 23:30:27 [engine.c:3171] Completed hash walk (thread 0) ================================================================================== Assistance greatly appreciated. Best, Johan. On 2018-12-24 20:50, [email protected] wrote: > Hi Emanuele, > > Both below Windows CMD terminals run as Administrator: > > C:\Program Files\ntopng>ntopng /c i -i tcp://*:5556c > > ============================================= > Starting ntopng > Running ntopng. > 24/Dec/2018 20:39:33 [Ntop.cpp:1545] Setting local networks to > 127.0.0.0/8 > 24/Dec/2018 20:39:33 [Redis.cpp:132] Successfully connected to redis > 127.0.0.1@0 > 24/Dec/2018 20:39:33 [Redis.cpp:132] Successfully connected to redis > 127.0.0.1@0 > 24/Dec/2018 20:39:33 [NtopPro.cpp:310] [LICENSE] Reading license from > Redis > 24/Dec/2018 20:39:33 [NtopPro.cpp:451] WARNING: [LICENSE] Invalid or > missing license > 24/Dec/2018 20:39:33 [NtopPro.cpp:468] WARNING: [LICENSE] ntopng will > now run in enterprise edition for 10 minutes > 24/Dec/2018 20:39:33 [NtopPro.cpp:470] WARNING: [LICENSE] before > returning to community mode > 24/Dec/2018 20:39:33 [NtopPro.cpp:472] WARNING: [LICENSE] You can buy > a permanent license at http://shop.ntop.org > 24/Dec/2018 20:39:33 [NtopPro.cpp:474] WARNING: [LICENSE] or run > ntopng in community mode starting > 24/Dec/2018 20:39:33 [NtopPro.cpp:475] WARNING: [LICENSE] ntopng > --community > 24/Dec/2018 20:39:34 [CollectorInterface.cpp:66] ERROR: Unable to bind > to ZMQ endpoint tcp://*:5556 [collector] > 24/Dec/2018 20:39:35 [main.cpp:239] ERROR: An exception occurred > during tcp://*:5556c interface creation[2]: No such file or directory > 24/Dec/2018 20:39:35 [main.cpp:293] ERROR: Startup error: missing > super-user privileges ? > > C:\Program Files\ntopng>ntopng /c i -i tcp://*:5556c > Starting ntopng > Running ntopng. > 24/Dec/2018 20:40:36 [Ntop.cpp:1545] Setting local networks to > 127.0.0.0/8 > 24/Dec/2018 20:40:36 [Redis.cpp:132] Successfully connected to redis > 127.0.0.1@0 > 24/Dec/2018 20:40:36 [Redis.cpp:132] Successfully connected to redis > 127.0.0.1@0 > 24/Dec/2018 20:40:36 [NtopPro.cpp:310] [LICENSE] Reading license from > Redis > 24/Dec/2018 20:40:36 [NtopPro.cpp:451] WARNING: [LICENSE] Invalid or > missing license > 24/Dec/2018 20:40:36 [NtopPro.cpp:468] WARNING: [LICENSE] ntopng will > now run in enterprise edition for 10 minutes > 24/Dec/2018 20:40:36 [NtopPro.cpp:470] WARNING: [LICENSE] before > returning to community mode > 24/Dec/2018 20:40:36 [NtopPro.cpp:472] WARNING: [LICENSE] You can buy > a permanent license at http://shop.ntop.org > 24/Dec/2018 20:40:36 [NtopPro.cpp:474] WARNING: [LICENSE] or run > ntopng in community mode starting > 24/Dec/2018 20:40:36 [NtopPro.cpp:475] WARNING: [LICENSE] ntopng > --community > 24/Dec/2018 20:40:37 [CollectorInterface.cpp:66] ERROR: Unable to bind > to ZMQ endpoint tcp://*:5556 [collector] > 24/Dec/2018 20:40:37 [main.cpp:239] ERROR: An exception occurred > during tcp://*:5556c interface creation[2]: No such file or directory > 24/Dec/2018 20:40:37 [main.cpp:293] ERROR: Startup error: missing > super-user privileges ? > ================================================ > > C:\Program Files\nProbe>nprobe /c my_nprobe --zmq > "tcp://<192.168.88.2>:5556" --zmq-probe-mode -i none -n none > --collector-port 2055 -T "@NTOPNG@" > > ============================================================ > Running nProbe for Windows. > 24/Dec/2018 20:41:38 [nprobe.c:4168] Valid nProbe license found > 24/Dec/2018 20:41:38 [nprobe.c:6092] WARNING: The output interfaceId > is set to 0: did you forget to use -Q perhaps ? > 24/Dec/2018 20:41:38 [nprobe.c:6095] WARNING: The input interfaceId is > set to 0: did you forget to use -u perhaps ? > 24/Dec/2018 20:41:38 [nprobe.c:6182] Welcome to nProbe v.8.6.181004 > ($Revision: 4384 $) for Windows > 24/Dec/2018 20:41:38 [nprobe.c:6192] Running on Windows > 24/Dec/2018 20:41:38 [nprobe.c:6203] [LICENSE] nProbe SystemId: > 2152224034-9206A1D8 > 24/Dec/2018 20:41:38 [nprobe.c:6270] Sample rate [packet: 1][flow > collection/export: 1/1] > 24/Dec/2018 20:41:38 [nprobe.c:8966] Welcome to nProbe v.8.6.181004 for > Windows > 24/Dec/2018 20:41:38 [nprobe.c:7870] WARNING: Adding > %EXPORTER_IPV4_ADDRESS to the template as nProbe is working as > collector > 24/Dec/2018 20:41:38 [nprobe.c:7976] Using NetFlow Packet Payload Len: > 1472 > 24/Dec/2018 20:41:38 [nprobe.c:7906] @NTOPNG@ expanded to " %L7_PROTO > %IPV4_SRC_ADDR %IPV4_DST_ADDR %L4_SRC_PORT %L4_DST_PORT %IPV6_SRC_ADDR > %IPV6_DST_ADDR %IP_PROTOCOL_VERSION %PROTOCOL %IN_BYTES %IN_PKTS > %OUT_BYTES %OUT_PKTS %FIRST_SWITCHED %LAST_SWITCHED %SRC_VLAN > %EXPORTER_IPV4_ADDRESS" > 24/Dec/2018 20:41:38 [plugin.c:1238] 0 plugin(s) enabled > 24/Dec/2018 20:41:38 [nprobe.c:8422] Each flow is 82 bytes long > 24/Dec/2018 20:41:38 [nprobe.c:8423] The # flows per packet has been > set to 16 > 24/Dec/2018 20:41:38 [nprobe.c:8426] IP TOS is accounted > 24/Dec/2018 20:41:38 [nprobe.c:8452] Non IPv4/v6 traffic is discarded > according to the template > 24/Dec/2018 20:41:38 [nprobe.c:9231] Flows ASs will not be computed > (missing libmxminddb support) > 24/Dec/2018 20:41:38 [nprobe.c:9334] Not capturing packet from > interface (collector mode) > 24/Dec/2018 20:41:38 [util.c:4719] Initializing ZMQ as client > 24/Dec/2018 20:41:38 [util.c:4736] ERROR: Unable to export flows > towards ZMQ endpoint tcp://<192.168.88.2>:5556: Invalid argument > 24/Dec/2018 20:41:38 [collect.c:142] Flow collector listening on port > 2055 (IPv4/v6) > 24/Dec/2018 20:41:38 [nprobe.c:9582] nProbe started successfully > 24/Dec/2018 20:46:29 [nprobe.c:567] Received shutdown request... > [signal: 2] > 24/Dec/2018 20:46:29 [nprobe.c:6317] Flushing active flows > 24/Dec/2018 20:46:31 [nprobe.c:3127] Processed packets: 0 (max bucket > search: 0) > 24/Dec/2018 20:46:31 [nprobe.c:3110] Fragment queue length: 0 > 24/Dec/2018 20:46:31 [nprobe.c:3137] Flow collection stats: > [collected pkts: 0][processed flows: 0] > 24/Dec/2018 20:46:31 [nprobe.c:3140] Flow export stats: [0 > bytes/0 pkts][0 flows/0 pkts sent] > 24/Dec/2018 20:46:31 [nprobe.c:3146] Flow export drop stats: [0 > bytes/0 pkts][0 flows] > 24/Dec/2018 20:46:31 [nprobe.c:3151] Total flow stats: [0 > bytes/0 pkts][0 flows/0 pkts sent] > ==================================================================== > > Am not sure what to do / try form here, assistance appreciated, > > Best, > > Johan. > > > On 2018-12-24 16:02, Emanuele Faranda wrote: >> Hi, >> >> Please try to replace /i with /c so that you can see the commands >> output. >> >> Regards, >> >> Emanuele >> >> On 12/24/18 12:17 AM, [email protected] wrote: >>> Update to the below, as per what Ive posted to the mailing list: >>> >>> We have Multiple nProbe sites with Mikrotik routers, and want to send >>> flows to one remote ntopng instance running on a Windows machine. >>> >>> Starting with the local site all behind the same Firewall / on same >>> LAN: >>> >>> Mikrotik is setup to send NetFlow to the IP of the host running >>> nprobe & ntopng: 192.168.88.2 >>> >>> ntopng started as service with the below CMD: >>> >>> ntopng /i -i tcp://*:5556c >>> >>> And nprobe with: >>> >>> nprobe /i my_nprobe --zmq "tcp://<192.168.88.2>:5556" >>> --zmq-probe-mode -i none -n none --collector-port 2055 -T "@NTOPNG@" >>> >>> As per the steps outlined here: >>> >>> https://www.ntop.org/nprobe/best-practices-for-the-collection-of-flows-with-ntopng-and-nprobe/ >>> However ntopng when loaded shows only: >>> >>> No packet has been received yet on interface tcp://*:5556c. Please >>> wait 6 seconds until this page reloads. >>> >>> Have also tried the steps outlined below to no avail: >>> >>> https://www.ntop.org/ntopng/how-to-analyse-mikrotik-traffic-using-ntopng/ >>> >>> Any help greatly appreciated, >>> >>> Johan. >>> >>> >>> >>> >>> >>> On 2018-12-23 13:12, [email protected] wrote: >>>> Hi there, >>>> >>>> We have one simple requirement: >>>> >>>> To accurately record how much bandwidth each user is using, across >>>> our >>>> several sites, over a day / week / month / year. Realtime data nice >>>> to >>>> have but not necessary. >>>> >>>> I say 'simple requirement' however having tried many ways to achieve >>>> this over years its been anything but simple. (For us anyhow.) >>>> >>>> With ntopng now being able to record historical data we're feeling >>>> encouraged to try ntop again. >>>> >>>> As such we've acquired the needed licenses, instructed our Mikrotik >>>> to >>>> send NetFlow to the Windows PC running nProbe & ntopng, and created >>>> the needed license file. >>>> >>>> However I cannot figure out how to start nprobe service to capture >>>> the >>>> Mikrotik flows and send them to ntopng. >>>> >>>> What are the correct Windows cmd's to start nprobe & ntopng, to >>>> capture NetFlow from Mikrotik please? >>>> >>>> Lots of tutorials like the one below for starting on Linux but no so >>>> much on Windows: >>>> >>>> https://www.ntop.org/ntopng/how-to-analyse-mikrotik-traffic-using-ntopng/ >>>> We seem to need the Windows equivalent of the below however: >>>> >>>> nprobe -i none -n none -3 2055 --zmq tcp://127.0.0.1:1234 >>>> ntopng -i tcp://127.0.0.1:1234 >>>> >>>> Help greatly appreciated, >>>> >>>> Best, >>>> >>>> Johan. >>>> >>>> _______________________________________________ >>>> Ntop mailing list >>>> [email protected] >>>> http://listgateway.unipi.it/mailman/listinfo/ntop >>> _______________________________________________ >>> Ntop mailing list >>> [email protected] >>> http://listgateway.unipi.it/mailman/listinfo/ntop >> _______________________________________________ >> Ntop mailing list >> [email protected] >> http://listgateway.unipi.it/mailman/listinfo/ntop