[OpenNMS/opennms] 03b6b6: NMS-20027: compute flow proportional sums with Pai...

  Branch: refs/heads/mm/NMS-20027-smoke
  Home:   https://github.com/OpenNMS/opennms
  Commit: 03b6b6dd8cfb3d3b90094530d7b2d10439fe0a48
      https://github.com/OpenNMS/opennms/commit/03b6b6dd8cfb3d3b90094530d7b2d10439fe0a48
  Author: Marshall Massengill <[email protected]>
  Date:   2026-07-16 (Thu, 16 Jul 2026)

  Changed paths:
    M docs/modules/operation/pages/deep-dive/flows/basic.adoc
    M docs/modules/releasenotes/pages/whatsnew.adoc
    M features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/ProportionalSumAggregation.java
    A features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/ProportionalSumQuery.java
    M features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/RawFlowQueryService.java
    M features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/SearchQueryProvider.java
    M features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/agg/AggregatedFlowQueryService.java
    M features/flows/elastic/src/main/java/org/opennms/netmgt/flows/elastic/agg/AggregatedSearchQueryProvider.java
    M features/flows/elastic/src/main/resources/OSGI-INF/blueprint/blueprint.xml
    M features/flows/elastic/src/main/resources/org/opennms/netmgt/flows/elastic/agg/series_top_n.ftl
    M features/flows/elastic/src/main/resources/org/opennms/netmgt/flows/elastic/agg/series_totals.ftl
    M features/flows/elastic/src/main/resources/org/opennms/netmgt/flows/elastic/series_for_missing.ftl
    M features/flows/elastic/src/main/resources/org/opennms/netmgt/flows/elastic/series_for_others.ftl
    M features/flows/elastic/src/main/resources/org/opennms/netmgt/flows/elastic/series_for_terms.ftl
    A features/flows/elastic/src/test/java/org/opennms/netmgt/flows/elastic/ProportionalSumQueryTest.java
    M features/flows/itests/pom.xml
    M features/flows/itests/src/test/java/org/opennms/netmgt/flows/elastic/AggregatedFlowQueryIT.java
    M features/flows/itests/src/test/java/org/opennms/netmgt/flows/elastic/FlowQueryIT.java
    M pom.xml

  Log Message:
  -----------
  NMS-20027: compute flow proportional sums with Painless instead of the drift plugin

Replace the proportional_sum aggregation provided by the
elasticsearch-drift-plugin with an equivalent scripted_metric
aggregation built from inline Painless scripts. This removes the need
to install a plugin build that exactly matches the Elasticsearch
version, and makes flow queries work against stock Elasticsearch and
OpenSearch clusters.

The scripted_metric state is a flat double[] indexed by bucket when
the window/step combination yields at most 4096 buckets (any UI-driven
query), which benchmarked 2.3x faster than the plugin on whole-window
series queries at 40M flows; a HashMap-based form with identical
output covers arbitrarily fine-grained windows, since the dense state
is allocated per terms cell up front. Field names are inlined into the
script source: per-document params lookups benchmarked ~1.5x slower.

The previous plugin-based queries remain available by setting
proportionalSumStrategy=plugin in
org.opennms.features.flows.persistence.elastic.cfg for clusters that
disallow inline scripting and already run the plugin.

The script-based implementation intentionally does not reproduce the
defect tracked as NMS-20001: for query windows whose start is not a
multiple of the step (the common case), the plugin mixes two bucket
grids in series results and stops counting window totals at the first
epoch-grid boundary after the window start, undercounting totals.
Totals may therefore increase after this change.

FlowQueryIT and AggregatedFlowQueryIT now run against a stock
Elasticsearch container without the plugin installed; the drift plugin
build-time dependency is removed. The testcontainers version in the
flow itests is aligned with the version managed in the root pom.



To unsubscribe from these emails, change your notification settings at https://github.com/OpenNMS/opennms/settings/notifications


_______________________________________________
Please read the OpenNMS Mailing List FAQ:
http://www.opennms.org/wiki/index.php?page=MailingListFaq
opennms-cvs mailing list

To *unsubscribe* or change your subscription options, see the bottom of this page:
https://lists.sourceforge.net/lists/listinfo/opennms-cvs