[OpenNMS/opennms] eb1a84: NMS-20203: track the Kafka sink consumer that was ...

  Branch: refs/heads/mm/NMS-20203-smoke
  Home:   https://github.com/OpenNMS/opennms
  Commit: eb1a8465927ab1b959d72f5b209740649037cdc5
      https://github.com/OpenNMS/opennms/commit/eb1a8465927ab1b959d72f5b209740649037cdc5
  Author: Marshall Massengill <[email protected]>
  Date:   2026-08-11 (Tue, 11 Aug 2026)

  Changed paths:
    M core/ipc/sink/kafka/itests/src/test/java/org/opennms/core/ipc/sink/kafka/itests/HeartbeatSinkIT.java
    M core/ipc/sink/kafka/server/src/main/java/org/opennms/core/ipc/sink/kafka/server/KafkaMessageConsumerManager.java

  Log Message:
  -----------
  NMS-20203: track the Kafka sink consumer that was actually started

startConsumingForModule() started one KafkaConsumerRunner per thread but
added a second, never-started one to the tracking list. stopConsumingForModule()
therefore called shutdown() on runners that had never subscribed, where
wakeup() has no poll to interrupt, while the running consumers never saw
closed=true.

The consequence is not just a leak. The orphaned consumers stay in the Kafka
group and keep receiving partition assignments, but their SinkModule key is no
longer registered, so AbstractMessageConsumerManager.dispatch() finds no
consumer and drops the record while auto-commit advances the offset. A
telemetryd or Sentinel adapter reload silently loses telemetry in proportion to
the partitions those consumers hold, compounding with each reload.

The unstarted duplicates never subscribe, so throughput is unaffected, but each
one holds 2 file descriptors and 6 JMX MBeans that are never released. The MBean
registration pins the object graph, so they survive GC.

Present since HZN-963 (2016).


  Commit: db0e04af32168bb8c14ee4c072cc295339b6d0ab
      https://github.com/OpenNMS/opennms/commit/db0e04af32168bb8c14ee4c072cc295339b6d0ab
  Author: Marshall Massengill <[email protected]>
  Date:   2026-08-11 (Tue, 11 Aug 2026)

  Changed paths:
    M core/ipc/sink/kafka/server/src/main/java/org/opennms/core/ipc/sink/kafka/server/KafkaMessageConsumerManager.java

  Log Message:
  -----------
  NMS-20203: stop the sink consumers when the manager shuts down

shutdown() called executor.shutdown(), which lets running tasks finish. The
KafkaConsumerRunner loop only exits once closed is set, so nothing ever ended
it and the consumers polled on past teardown of the Spring context or the
blueprint bundle, holding their group membership and their threads.

Shut the runners down first, then bound the wait on the executor and fall back
to shutdownNow(). The runners wake from a 100ms poll, so the timeout is only a
backstop.


Compare: https://github.com/OpenNMS/opennms/compare/eb1a8465927a%5E...db0e04af3216

To unsubscribe from these emails, change your notification settings at https://github.com/OpenNMS/opennms/settings/notifications


_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/wiki/index.php?page=MailingListFaq
opennms-cvs mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-cvs