Re: XML datacollection crash after upgrading to v20
"Showers, William" <[email protected]>
| Newsgroups | gmane.network.opennms.general |
|---|---|
| Message-ID | <[email protected]> |
I have attached the config files you mentioned and below is the output from the ES cluster that is being monitored. Thanks for looking at this!
-Bill
{"_nodes":{"total":1,"successful":1,"failed":0},"cluster_name":"logging","timestamp":1501609963943,"status":"yellow","indices":{"count":55,"shards":{"total":183,"primaries":183,"replication":0.0,"index":{"shards":{"min":1,"max":5,"avg":3.327272727272727},"primaries":{"min":1,"max":5,"avg":3.327272727272727},"replication":{"min":0.0,"max":0.0,"avg":0.0}}},"docs":{"count":94878152,"deleted":47173},"store":{"size_in_bytes":76727570953,"throttle_time_in_millis":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"query_cache":{"memory_size_in_bytes":0,"total_count":0,"hit_count":0,"miss_count":0,"cache_size":0,"cache_count":0,"evictions":0},"completion":{"size_in_bytes":0},"segments":{"count":821,"memory_in_bytes":204878986,"terms_memory_in_bytes":172514554,"stored_fields_memory_in_bytes":19730392,"term_vectors_memory_in_bytes":0,"norms_memory_in_bytes":40576,"points_memory_in_bytes":2245332,"doc_values_memory_in_bytes":10348132,"index_writer_memory_in_bytes":11990020,"version_map_memory_in_bytes":18885,"fixed_bit_set_memory_in_bytes":0,"max_unsafe_auto_id_timestamp":9223372036854775807,"file_sizes":{}}},"nodes":{"count":{"total":1,"data":1,"coordinating_only":0,"master":1,"ingest":1},"versions":["5.5.1"],"os":{"available_processors":8,"allocated_processors":8,"names":[{"name":"Linux","count":1}],"mem":{"total_in_bytes":33732866048,"free_in_bytes":9444581376,"used_in_bytes":24288284672,"free_percent":28,"used_percent":72}},"process":{"cpu":{"percent":4},"open_file_descriptors":{"min":821,"max":821,"avg":821}},"jvm":{"max_uptime_in_millis":6373642,"versions":[{"version":"1.8.0_131","vm_name":"OpenJDK 64-Bit Server VM","vm_version":"25.131-b11","vm_vendor":"Oracle Corporation","count":1}],"mem":{"heap_used_in_bytes":2025014664,"heap_max_in_bytes":17110138880},"threads":72},"fs":{"total_in_bytes":393675419648,"free_in_bytes":315810734080,"available_in_bytes":295789588480},"plugins":[],"network_types":{"transport_types":{"netty4":1},"http_types":{"netty4":1}}}}
From: Alejandro Galue [mailto:[email protected]]
Sent: Tuesday, August 1, 2017 1:40 PM
To: General OpenNMS Discussion <[email protected]>
Subject: Re: [opennms-discuss] XML datacollection crash after upgrading to v20
On Aug 1, 2017, at 1:35 PM, Showers, William <[email protected]<mailto:[email protected]>> wrote:
It allows the service to start but no data gets collected.
Maybe sharing the configuration of the XML collector an a sample XML output that is supposed to be consumed can give us some context about what is not working.
Here is a troubleshooting hint: as you’re running 20, you can test a single collection with the new shell commands:
http://docs.opennms.org/opennms/releases/20.0.1/guide-admin/guide-admin.html#ga-performance-mgmt-shell
Alejandro Galue
[email protected]<mailto:[email protected]>
PGP Key Fingerprint: 5293 6234 1E75 DF30 7821 1823 87AF 972E DAF8 BE2C
This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, notify the sender immediately by return email and delete the message and any attachments from your system.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/index.php/Mailing_List_FAQ
opennms-discuss mailing list
To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-discuss
elasticsearch-cluster-stats.xml
(text/xml, 7.1 KB)
<xml-groups>
<xml-group name="cluster" resource-type="node" resource-xpath="/">
<xml-object name="clusterName" type="STRING" xpath="cluster_name"/>
</xml-group>
<xml-group name="cluster-indices" resource-type="node" resource-xpath="/indices">
<xml-object name="indicesCount" type="GAUGE" xpath="count"/>
</xml-group>
<xml-group name="cluster-indices-shards-index-shards" resource-type="node" resource-xpath="/indices/shards/index/shards">
<xml-object name="shardsMin" type="GAUGE" xpath="min"/>
<xml-object name="shardsMax" type="GAUGE" xpath="max"/>
<xml-object name="shardsAvg" type="GAUGE" xpath="avg"/>
</xml-group>
<xml-group name="cluster-indices-shards-index-primaries" resource-type="node" resource-xpath="/indices/shards/index/primaries">
<xml-object name="primariesMin" type="GAUGE" xpath="min"/>
<xml-object name="primariesMax" type="GAUGE" xpath="max"/>
<xml-object name="primariesAvg" type="GAUGE" xpath="avg"/>
</xml-group>
<xml-group name="cluster-indices-shards-index-replication" resource-type="node" resource-xpath="/indices/shards/index/replication">
<xml-object name="replicationMin" type="GAUGE" xpath="min"/>
<xml-object name="replicationMax" type="GAUGE" xpath="max"/>
<xml-object name="replicationAvg" type="GAUGE" xpath="avg"/>
</xml-group>
<xml-group name="cluster-indices-docs" resource-type="node" resource-xpath="/indices/docs">
<xml-object name="docsCount" type="GAUGE64" xpath="count"/>
<xml-object name="docsDeleted" type="GAUGE64" xpath="deleted"/>
</xml-group>
<xml-group name="cluster-indices-store" resource-type="node" resource-xpath="/indices/store">
<xml-object name="storeSizeBytes" type="GAUGE64" xpath="size_in_bytes"/>
<xml-object name="throttleTimeMillis" type="GAUGE64" xpath="throttle_time_in_millis"/>
</xml-group>
<xml-group name="cluster-indices-fielddata" resource-type="node" resource-xpath="/indices/fielddata">
<xml-object name="memorySizeBytes" type="GAUGE64" xpath="memory_size_in_bytes"/>
<xml-object name="memoryEvictions" type="GAUGE64" xpath="evictions"/>
</xml-group>
<xml-group name="cluster-indices-filtercache" resource-type="node" resource-xpath="/indices/filter_cache">
<xml-object name="fltrCacheBytes" type="GAUGE64" xpath="memory_size_in_bytes"/>
<xml-object name="fltrCacheEvictions" type="GAUGE64" xpath="evictions"/>
</xml-group>
<xml-group name="cluster-indices-idcache" resource-type="node" resource-xpath="/indices/id_cache">
<xml-object name="idCacheBytes" type="GAUGE64" xpath="memory_size_in_bytes"/>
</xml-group>
<xml-group name="cluster-indices-completion" resource-type="node" resource-xpath="/indices/completion">
<xml-object name="completionSizeBytes" type="GAUGE64" xpath="memory_size_in_bytes"/>
</xml-group>
<xml-group name="cluster-indices-segments" resource-type="node" resource-xpath="/indices/segments">
<xml-object name="segCount" type="GAUGE64" xpath="count"/>
<xml-object name="segMemory" type="GAUGE64" xpath="memory_in_bytes"/>
<xml-object name="segWriterMem" type="GAUGE64" xpath="index_writer_memory_in_bytes"/>
<xml-object name="segWriterMemMax" type="GAUGE64" xpath="index_writer_max_memory_in_bytes"/>
<xml-object name="segVerMapMem" type="GAUGE64" xpath="version_map_memory_in_bytes"/>
<xml-object name="segFixBitSetMem" type="GAUGE64" xpath="fixed_bit_set_memory_in_bytes"/>
</xml-group>
<xml-group name="cluster-indices-percolate" resource-type="node" resource-xpath="/indices/percolate">
<xml-object name="percTotal" type="GAUGE" xpath="total"/>
<xml-object name="percTimeInMillis" type="GAUGE64" xpath="time_in_millis"/>
<xml-object name="percCurrent" type="GAUGE" xpath="current"/>
<xml-object name="percMemSizeBytes" type="GAUGE64" xpath="memory_size_in_bytes"/>
<xml-object name="percQueries" type="GAUGE" xpath="queries"/>
</xml-group>
<xml-group name="cluster-nodes-count" resource-type="node" resource-xpath="/nodes/count">
<xml-object name="nodesCntTotal" type="GAUGE" xpath="total"/>
<xml-object name="nodesCntMstOnly" type="GAUGE" xpath="master_only"/>
<xml-object name="nodesCntDataOnly" type="GAUGE" xpath="data_only"/>
<xml-object name="nodesCntMasterData" type="GAUGE" xpath="master_data"/>
<xml-object name="nodesClient" type="GAUGE" xpath="client"/>
</xml-group>
<xml-group name="cluster-nodes-os" resource-type="node" resource-xpath="/nodes/os">
<xml-object name="osAvailProc" type="GAUGE" xpath="available_processors"/>
</xml-group>
<xml-group name="cluster-nodes-os-mem" resource-type="node" resource-xpath="/nodes/os/mem">
<xml-object name="osMemTotalBytes" type="GAUGE64" xpath="total_in_bytes"/>
</xml-group>
<xml-group name="cluster-nodes-process-cpu" resource-type="node" resource-xpath="/nodes/process/cpu">
<xml-object name="procCpuPercent" type="GAUGE" xpath="percent"/>
</xml-group>
<xml-group name="cluster-nodes-process-open-file-descriptors" resource-type="node" resource-xpath="/nodes/process/open_file_descriptors">
<xml-object name="openFileDescMin" type="GAUGE" xpath="min"/>
<xml-object name="openFileDescMax" type="GAUGE" xpath="max"/>
<xml-object name="openFileDescAvg" type="GAUGE" xpath="avg"/>
</xml-group>
<xml-group name="cluster-nodes-jvm" resource-type="node" resource-xpath="/nodes/jvm">
<xml-object name="jvmMaxUptimeMillis" type="GAUGE64" xpath="max_uptime_in_millis"/>
</xml-group>
<xml-group name="cluster-nodes-jvm-mem" resource-type="node" resource-xpath="/nodes/jvm/mem">
<xml-object name="jvmMemHeapBytes" type="GAUGE64" xpath="heap_used_in_bytes"/>
<xml-object name="jvmMemMaxBytes" type="GAUGE64" xpath="heap_max_in_bytes"/>
</xml-group>
<xml-group name="cluster-nodes-jvm-threads" resource-type="node" resource-xpath="/nodes/jvm/threads">
<xml-object name="jvmThreads" type="GAUGE" xpath="threads"/>
</xml-group>
<xml-group name="cluster-nodes-fs" resource-type="node" resource-xpath="/nodes/fs">
<xml-object name="fsTotalBytes" type="GAUGE64" xpath="total_in_bytes"/>
<xml-object name="fsFreeBytes" type="GAUGE64" xpath="free_in_bytes"/>
<xml-object name="fsAvailBytes" type="GAUGE64" xpath="available_in_bytes"/>
<xml-object name="fsDskReads" type="GAUGE" xpath="disk_reads"/>
<xml-object name="fsDskWrites" type="GAUGE" xpath="disk_writes"/>
<xml-object name="fsDskIoOp" type="GAUGE" xpath="disk_io_op"/>
<xml-object name="fsDskReadSizeBytes" type="GAUGE64" xpath="disk_read_size_in_bytes"/>
<xml-object name="fsDskWriteSizeBytes" type="GAUGE64" xpath="disk_write_size_in_bytes"/>
<xml-object name="fsDskIoSizeBytes" type="GAUGE64" xpath="disk_io_size_in_bytes"/>
<xml-object name="fsDskQueue" type="GAUGE" xpath="disk_queue"/>
<xml-object name="fsDskSvcTime" type="GAUGE" xpath="disk_service_time"/>
</xml-group>
</xml-groups>
xml-datacollection-config.xml
(text/xml, 869 B)
<xml-datacollection-config rrdRepository="/var/lib/opennms/rrd/snmp/" xmlns="http://xmlns.opennms.org/xsd/config/xml-datacollection">
<!--
To understand how the XML Collector works, and how to configure it, please check the following link:
http://www.opennms.org/wiki/XML_Collector
-->
<xml-collection name="xml-elasticsearch-cluster-stats">
<rrd step="300">
<rra>RRA:AVERAGE:0.5:1:2016</rra>
<rra>RRA:AVERAGE:0.5:12:1488</rra>
<rra>RRA:AVERAGE:0.5:288:366</rra>
<rra>RRA:MAX:0.5:288:366</rra>
<rra>RRA:MIN:0.5:288:366</rra>
</rrd>
<xml-source url="http://es.ptcollege.edu:9200/_cluster/stats">
<import-groups>xml-datacollection/elasticsearch-cluster-stats.xml</import-groups>
</xml-source>
</xml-collection>
</xml-datacollection-config>