[OpenNMS/opennms] dca94f: NMS-20203: Kafka sink consumer manager tracks unst...

  Branch: refs/heads/merge-foundation/foundation-2024-to-foundation-2025
  Home:   https://github.com/OpenNMS/opennms
  Commit: dca94f0b60895aa68011417d9003793ca3cd8f05
      https://github.com/OpenNMS/opennms/commit/dca94f0b60895aa68011417d9003793ca3cd8f05
  Author: Marshall Massengill <[email protected]>
  Date:   2026-08-13 (Thu, 13 Aug 2026)

  Changed paths:
    M core/ipc/sink/kafka/itests/src/test/java/org/opennms/core/ipc/sink/kafka/itests/HeartbeatSinkIT.java
    M core/ipc/sink/kafka/server/src/main/java/org/opennms/core/ipc/sink/kafka/server/KafkaMessageConsumerManager.java

  Log Message:
  -----------
  NMS-20203: Kafka sink consumer manager tracks unstarted consumers, leaking live ones on reload (#8778)

* NMS-20203: track the Kafka sink consumer that was actually started

startConsumingForModule() started one KafkaConsumerRunner per thread but
added a second, never-started one to the tracking list. stopConsumingForModule()
therefore called shutdown() on runners that had never subscribed, where
wakeup() has no poll to interrupt, while the running consumers never saw
closed=true.

The consequence is not just a leak. The orphaned consumers stay in the Kafka
group and keep receiving partition assignments, but their SinkModule key is no
longer registered, so AbstractMessageConsumerManager.dispatch() finds no
consumer and drops the record while auto-commit advances the offset. A
telemetryd or Sentinel adapter reload silently loses telemetry in proportion to
the partitions those consumers hold, compounding with each reload.

The unstarted duplicates never subscribe, so throughput is unaffected, but each
one holds 2 file descriptors and 6 JMX MBeans that are never released. The MBean
registration pins the object graph, so they survive GC.

Present since HZN-963 (2016).

* NMS-20203: stop the sink consumers when the manager shuts down

shutdown() called executor.shutdown(), which lets running tasks finish. The
KafkaConsumerRunner loop only exits once closed is set, so nothing ever ended
it and the consumers polled on past teardown of the Spring context or the
blueprint bundle, holding their group membership and their threads.

Shut the runners down first, then bound the wait on the executor and fall back
to shutdownNow(). The runners wake from a 100ms poll, so the timeout is only a
backstop.

* NMS-20203: drain the starter threads before stopping the sink consumers

shutdown() stopped the runners and cleared consumerRunnersByModule while
startupExecutor was still live, so a startup task landing in that window
registered consumers that nothing was left to stop. It also submitted to
an executor that was about to be shut down, which rejected the remaining
runners mid-loop and left the already-started ones untracked. Shutting the
starters down first and waiting for the in-flight task closes both.

Also share one AdminClient across the group-membership polls in the IT
rather than opening one per poll, and tolerate the transient
describeConsumerGroups failures that happen before the broker has elected
a coordinator for the group.


  Commit: f360a3718bad1ef1ef4f308a0a383ebdcd1b736a
      https://github.com/OpenNMS/opennms/commit/f360a3718bad1ef1ef4f308a0a383ebdcd1b736a
  Author: CI/CD System <[email protected]>
  Date:   2026-08-13 (Thu, 13 Aug 2026)

  Changed paths:
    M core/ipc/sink/kafka/itests/src/test/java/org/opennms/core/ipc/sink/kafka/itests/HeartbeatSinkIT.java
    M core/ipc/sink/kafka/server/src/main/java/org/opennms/core/ipc/sink/kafka/server/KafkaMessageConsumerManager.java

  Log Message:
  -----------
  Merge remote-tracking branch 'origin/foundation-2024' into foundation-2025


Compare: https://github.com/OpenNMS/opennms/compare/8d7a8491f45a...f360a3718bad

To unsubscribe from these emails, change your notification settings at https://github.com/OpenNMS/opennms/settings/notifications


_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/wiki/index.php?page=MailingListFaq
opennms-cvs mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-cvs