Re: Missing percentdiscards, percenterrors, discards, and errors graphs

Zhi-Wei Lu <[email protected]>
Newsgroups gmane.network.opennms.general
Message-ID <MWHPR08MB28310A439F56A40C8F4FD9E0CB790@MWHPR08MB2831.namprd08.prod.outlook.com> 
Thank you, Jose for confirming that you have seen similar behaviors.


I took sometime to install grafana and  opennms helm, I was able to put up a dashboard to plot Discards/Errors for one interface.  Unlike what you have witnessed that grafana graphs were fine, I didn't get that.  I am getting some zeros and some without data.


It happens that that interface has some InErrors and it increment slowly.


snmpget 10.99.1.254 IF-MIB::ifInErrors.1
IF-MIB::ifInErrors.1 = Counter32: 1831


I don't really know what it is actually being plotted in the OpenNMS resource graphs or in the grafana integration graphs, how do Opennms plot percentage errors and what are being stored in the newts database? Do they store the difference of ifInErrors or the actual number polled from SNMP?  If the difference is being stored, then if the previous number was NaN, how do they calculate the difference for points thereafter?  This might explain the problem that we are seeing.

New discards/errors graph works, but the existing graphs suffer uncertainties after we re-enabled discards/errors in SNMP polling.


I don't quite know how I can get to the bottom of that (it would take a me long while to read/understand those codes), the developer can probably explain how those numbers are stored and computed ( if[In|Out]Errors and if[In|Out]Discards) and the percentage graphs in OpenNMS as well as in the grafana integration graphs.


Thank you and have a great weekend.


Zhi-Wei Lu

IET-CR-Network Operations Center
University of California, Davis
(530) 752-0155
________________________________
From: José Manuel Agudo Cuesta <[email protected]>
Sent: Thursday, June 7, 2018 12:03:38 AM
To: General OpenNMS Discussion
Subject: Re: [opennms-discuss] Missing percentdiscards, percenterrors, discards, and errors graphs

Hi,

I also have problems with some of the graphs, looks like a problem with the visualization backend, because the same counters graph without errors in Grafana. Also the same counters in OpenNMS sometimes graph sometimes not. Its strange.

Best regards,

Jose





El mié., 6 jun. 2018 a las 22:39, Zhi-Wei Lu (<[email protected]<mailto:[email protected]>>) escribió:

Thank you, Jose!



I did restart opennms! Now, I noticed that if I start a new interface graph, those discards/errors would show up properly (all zeros in this case).  However, for those existing graphs, some show up as zeros (e.g. InErrors) , while the other as NaN (OutErrors).  I don’t quite understand the reason why. I guess that all those zeros might have something to do (e.g. handling divide by zero in percentage) with the issue that I saw. I am trying to learn cqlsh to search for those data directly (in Cassandra), why I am getting NaN for some, but not the other. I am not quite there yet.



Thanks again!



Zhi-Wei Lu

IET-CR-Network Operations Center

University of California, Davis

(530) 752-0155<tel:(530)%20752-0155>



From: José Manuel Agudo Cuesta <[email protected]<mailto:[email protected]>>
Sent: Wednesday, June 06, 2018 10:46 AM

To: General OpenNMS Discussion <[email protected]<mailto:[email protected]>>
Subject: Re: [opennms-discuss] Missing percentdiscards, percenterrors, discards, and errors graphs



Yes, restart OpenNMS when possible.

El mié., 6 jun. 2018 19:04, Zhi-Wei Lu <[email protected]<mailto:[email protected]>> escribió:

Thank you Jose!



I reload the Collectd and Pollerd, but the data (discards and errors) are not being collected.  Do I have to restart opennms?



Thank you.



Zhi-Wei Lu

IET-CR-Network Operations Center

University of California, Davis

(530) 752-0155<tel:(530)%20752-0155>



From: José Manuel Agudo Cuesta <[email protected]<mailto:[email protected]>>
Sent: Wednesday, June 06, 2018 7:38 AM
To: General OpenNMS Discussion <[email protected]<mailto:[email protected]>>
Subject: Re: [opennms-discuss] Missing percentdiscards, percenterrors, discards, and errors graphs



Hi,



I added the missing counters to the mib2-X-interfaces-pkts group and start graphing the counters again. I have newts storage because of that I don't have lost any information except the last days after the upgrade to version 22.



   <group name="mib2-X-interfaces-pkts" ifType="all">

      <mibObj oid=".1.3.6.1.2.1.31.1.1.1.7" instance="ifIndex" alias="ifHCInUcastPkts" type="Counter64"/>

      <mibObj oid=".1.3.6.1.2.1.31.1.1.1.8" instance="ifIndex" alias="ifHCInMulticastPkts" type="Counter64"/>

      <mibObj oid=".1.3.6.1.2.1.31.1.1.1.9" instance="ifIndex" alias="ifHCInBroadcastPkts" type="Counter64"/>

      <mibObj oid=".1.3.6.1.2.1.31.1.1.1.11" instance="ifIndex" alias="ifHCOutUcastPkts" type="Counter64"/>

      <mibObj oid=".1.3.6.1.2.1.31.1.1.1.12" instance="ifIndex" alias="ifHCOutMulticastPkt" type="Counter64"/>

      <mibObj oid=".1.3.6.1.2.1.31.1.1.1.13" instance="ifIndex" alias="ifHCOutBroadcastPkt" type="Counter64"/>

      <mibObj oid=".1.3.6.1.2.1.2.2.1.13" instance="ifIndex" alias="ifInDiscards" type="counter"/>

      <mibObj oid=".1.3.6.1.2.1.2.2.1.14" instance="ifIndex" alias="ifInErrors" type="counter"/>

      <mibObj oid=".1.3.6.1.2.1.2.2.1.19" instance="ifIndex" alias="ifOutDiscards" type="counter"/>

      <mibObj oid=".1.3.6.1.2.1.2.2.1.20" instance="ifIndex" alias="ifOutErrors" type="counter"/>

   </group>



Also modified the mib2 thresholding-config to take account of the changes and use the 64 bit counters:



      <expression description="Trigger an alert if incoming usage of any interface reaches or goes above 80% of its maximum speed for three consecutive measurement interv

als (only for interfaces that have a maximum speed value defined)" type="high" ds-type="if" value="80.0" rearm="65.0" trigger="3" ds-label="ifName" filterOperator="OR" ex

pression="ifHCInOctets * 8 / 1000000 / ifHighSpeed * 100">

         <resource-filter field="ifHighSpeed">^[1-9]+[0-9]*$</resource-filter>

      </expression>

      <expression description="Trigger an alert if outgoing usage of an interface reaches or goes above 80% of its maximum speed for three consecutive measurement interva

ls (only for interfaces that have a maximum speed value defined)" type="high" ds-type="if" value="80.0" rearm="65.0" trigger="3" ds-label="ifName" filterOperator="OR" exp

ression="ifHCOutOctets * 8 / 1000000 / ifHighSpeed * 100">

         <resource-filter field="ifHighSpeed">^[1-9]+[0-9]*$</resource-filter>

      </expression>



Best regards,



Jose















El mié., 6 jun. 2018 a las 15:35, Ronny Trommer (<[email protected]<mailto:[email protected]>>) escribió:

Hi,



I’ve looked up the configuration files and the issue is related to splitting the high speed counters, which removed those 32 bit counters. There are no 64bit counter equivalents for discards and errors. There are several approaches to fix this problem which depend if you use store-by-group or not.



1)

If you have store-by-group disabled, which is default. You can add a group, let’s say mib2-interface-errors[1] and include them in the system definition for the Enterprise sysoidMask.



2)

If you have store-by-group enabled, your OID groups are persisted in one RRD file. You can restore the old behaviour described in the release notes [2]. If you create a new group described in 1) you will end up with a new RRD file for mib2-interface-errors, but the allocated storage in the mib2-interface RRD files from your old setup is still there. You need to mangle the RRD files with rrdtune [3] to remove data sources for the discard and error messages.



Created also an issue[4] to track this problem. In hope this helps



Ronny



[1] https://gist.github.com/indigo423/2a86ab560065fc62d6dc2501d6cdc2bb<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgist.github.com%2Findigo423%2F2a86ab560065fc62d6dc2501d6cdc2bb&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432588931&sdata=Um7gFNUrrBj376jC1PkPpHf%2BhsLspumC277Q6ttVtGA%3D&reserved=0>

[2] http://docs.opennms.org/opennms/releases/22.0.0/releasenotes/releasenotes.html<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdocs.opennms.org%2Fopennms%2Freleases%2F22.0.0%2Freleasenotes%2Freleasenotes.html&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432598939&sdata=rN%2FaB5to%2Fs7%2FsiFUxvorDB%2BNw5Q%2BdX50vtbLFR4iaiw%3D&reserved=0>

[3] https://oss.oetiker.ch/rrdtool/doc/rrdtune.en.html<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Foss.oetiker.ch%2Frrdtool%2Fdoc%2Frrdtune.en.html&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432598939&sdata=bkMgWcHPKzWEg%2BVU4qrVSrmZgZrV%2FUGImiQmHaomHic%3D&reserved=0>

[4] https://issues.opennms.org/browse/NMS-10195<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.opennms.org%2Fbrowse%2FNMS-10195&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432608952&sdata=ncdUJ65mH4FQhnkBPgNkivcC%2Boi6b5BKvcYXhNr54Z0%3D&reserved=0>



On 6. Jun 2018, at 02:21, Zhi-Wei Lu <[email protected]<mailto:[email protected]>> wrote:



Hi all,



I recently upgraded from Horizon 21 to Horizon 22, all new nodes added since the upgrade don’t have percentdiscards, percenterrors, discards, and error graphs, while the old graphs have no new data for these.  I just wonder what happened, those data were not collected after the upgrade?



Thank you for your help.



Zhi-Wei Lu

IET-CR-Network Operations Center

University of California, Davis

(530) 752-0155<tel:(530)%20752-0155>



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fslashdot.org%2F&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432608952&sdata=KlaU7xdnxXZpKxZM%2FJrP4GQj%2Bek2hq%2BM0dz%2BYsMonKI%3D&reserved=0>! http://sdm.link/slashdot_______________________________________________<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot_______________________________________________&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432618960&sdata=pcbrMlc%2FDskrn%2BrUQIaCnHEZsbvXK13azaJnnQIvusA%3D&reserved=0>
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.opennms.org%2Findex.php%2FMailing_List_FAQ&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432628968&sdata=6fJIL7UIQ5QnisJejjxzgqXLzCEk8muR5NOo%2B6yJQRA%3D&reserved=0>

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fopennms-discuss&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432628968&sdata=fuULze3wcKPxvRDW6ZQ0FOkUSDl0rICyhEzV2OAwTmY%3D&reserved=0>



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432638977&sdata=H6YKEPmQ5biuZG5DR4RvWz5aAMtTV3fJuzkTRqr0QMs%3D&reserved=0>_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.opennms.org%2Findex.php%2FMailing_List_FAQ&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432638977&sdata=a6kzF3cU4X4ybgFHC9d3OfOkCnrybCpC9khSmEM%2FAY4%3D&reserved=0>

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fopennms-discuss&data=02%7C01%7C%7C7a287664444545501d4b08d5cbbb3ed1%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636638927432648985&sdata=Gazto2dAMA35L0k59W0inXpaHOoUDElzZA7eDr635aQ%3D&reserved=0>

--

JOSE MANUEL AGUDO CUESTA

Ingeniero de Redes y Seguridad

Servicios Informáticos - C.P.D.

Universidad de Salamanca

+34 923 294 500 ext. 1398<tel:+34%20923%2029%2045%2000>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7C%7C2876a701bef14e0266bb08d5cbd575c5%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636639040025911863&sdata=lSX51Ndt6wi4PWPAy0gpw2DkLf4Ls%2BH9DaIDSqrQ%2FT4%3D&reserved=0>_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.opennms.org%2Findex.php%2FMailing_List_FAQ&data=02%7C01%7C%7C2876a701bef14e0266bb08d5cbd575c5%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636639040025911863&sdata=ksaNdBR0CR%2BpaS3KaxuMAaW3uZB6TAnHfygH26I7KtA%3D&reserved=0>

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fopennms-discuss&data=02%7C01%7C%7C2876a701bef14e0266bb08d5cbd575c5%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636639040025911863&sdata=h1ZJX61065G9ft4gq8TmjnJLFafuk%2FWPkIkBZBaJ1jc%3D&reserved=0>

--

JOSE MANUEL AGUDO CUESTA

Ingeniero de Redes y Seguridad

Servicios Informáticos - C.P.D.

Universidad de Salamanca

+34 923 294 500 ext. 1398<tel:+34%20923%2029%2045%2000>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsdm.link%2Fslashdot&data=02%7C01%7C%7C358497476e11445c352508d5cc44e075%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636639518559950436&sdata=xvpZANdX1V7%2FSa%2BrN%2BX8%2F5JrlQDq8ix0ROHWEGv2UTI%3D&reserved=0>_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.opennms.org%2Findex.php%2FMailing_List_FAQ&data=02%7C01%7C%7C358497476e11445c352508d5cc44e075%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636639518559960441&sdata=JJr%2FWBB1gu5oZ9d3bR%2FQQN8xIxf1XS70eSwAII%2BtkbI%3D&reserved=0>

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fopennms-discuss&data=02%7C01%7C%7C358497476e11445c352508d5cc44e075%7Ca8046f6466c04f009046c8daf92ff62b%7C0%7C1%7C636639518559960441&sdata=Env0DYaqhlWqmnZSoYibiafxEkUKBRaWqUdZe1SAh9A%3D&reserved=0>
--
JOSE MANUEL AGUDO CUESTA
Ingeniero de Redes y Seguridad
Servicios Informáticos - C.P.D.
Universidad de Salamanca
+34 923 294 500 ext. 1398

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ

opennms-discuss mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.