Re: [PATCH v4 0/9] Add python, PCP support via libdtrace interfaces

As a preliminary review, here is what Codex has to say about the series.
I run every posted series through it as a first level of review, and to
evaluate how valid its findings are.

Here goes:

• Found four issues in the nine commits:

  - P1 — --without python RPM builds still force Python bindings.
    dtrace.spec:174 and :190 use %{?with_python:...}; %bcond_without python
    defines that macro even when its value is 0. Thus a Python-disabled build
    still runs WITH_PYTHON=y/install-python, despite omitting Python build
    requirements. Use %if %{with python} around those arguments.

  - P1 — Dynamic PMDA aggregation instances cannot recover after disappearing.
    pcp/pmdadtrace.python:560 removes the instance from the PMDA indom and ID
    map but leaves _data_instances[instance]. When that aggregation key returns,
    pcp/pmdadtrace.python:541 returns early, so it is never re-added and fetches
    yield no value. Remove the name-map entry too, or restore its indom/ID
    mapping.

  - P2 — Changing the PMDA domain as the installation instructions require has
    no effect. pcp/pmdadtrace.python:1219 parses domain= from Install, but pcp/
    pmdadtrace.python:1228 instantiates the PMDA with hard-coded 487. A
    collision workaround described in the README leaves pmcd and metric PMIDs on
    different domains. Pass domain instead.

  - P2 — The multiple-session regression test does not close the sessions it
    creates. test/unittest/python/tst.multi-session.sh:50 loops over j but
    always closes dts[i], excludes the final session, and leaves the others
    open. It therefore fails to test the close/refcount lifecycle that the
    preceding BTF-sharing commit changes.


On Wed, Aug 12, 2026 at 02:22:01PM +0100, Alan Maguire wrote:
> Having python bindings for libdtrace interfaces to compile, run
> and collect information from DTrace programs is valuable because
> it could help integration into metric collection frameworks like
> PCP.  This series adds python bindings for DTrace (patch 4)
> and provides tests for them (patch 7).  The support is
> similar to what cmd/dtrace.c can do; compile a program,
> enable probes, run it and collect data.  Support is also
> added to grab or create processes as is done by
> dtrace -p, -c options.  Aggregation snapshot walk is also
> supported, with aggregation values filled out as raw values,
> ints, lists of function names for stack keys and dicts for
> quantized aggregations.
> 
> For detailed description/usage see the README in patch 4 and the
> tests.
> 
> Patch 8 then provides a consumer of python support - a Performance
> Co-Pilot (PCP) Performance Metric Data Agent (PMDA) which enables
> flexible metric collection via DTrace scripts.  See the README.md
> in patch 8 for more details.
> 
> In order to support an environment where we have multiple handles
> per process, some prep work is required.  Patch 1 fixes some issues
> with multiple handle use, while patch 2 improves sharing for
> vmlinux BTF (and CTF generated from it if needed) which multiple
> handles in a process image will benefit from.  The upper bound
> of per-process handles is now limited by the maximum number of
> BPF programs attachable to a uprobe which is 64 (BPF_TRACE_MAX_PROGS)
> which is encountered with attaches to the BEGIN probe.
> 
> Patch 3 makes available some of the functions used in calculating
> stddev() values which is useful for the python bindings since they
> do not use the print callbacks to do this.
> 
> Patch 4 is the cpython bindings themselves; the README.md describes
> their usage.  Patch 5 packages them and patches 6 and 7 facilitate
> testing them.
> 
> Patch 8 then builds the PCP PMDA using the python bindings while
> patch 9 packages them in a pcp-pmda-dtrace package.
> 
> Changes since v3:
> 
> - Fixed up issues identified by Kris around custom BTF path (patch 2)
> - Added PCP patches 8/9
> 
> Changes since v2:
> 
> - Fixed some handle lifetime issues and related potential segfaults
>   in error paths (patch 4)
> - Fixed up packaging to derive python package version from DTrace
>   version (patch 5)
> 
> Changes since v1:
> 
> - Fixed multi-handle issues (patches 1/2)
> - Improved aggregation representation of stacks, added stddev and
>   *quantize representations (patch 4)
> 
> Alan Maguire (9):
>   libdtrace: Support multiple DTrace handles per process
>   libdtrace: share vmlinux BTF/CTF globally to support faster startup
>   libdtrace: Refactor math functions into dt_math.h
>   python: Add cpython bindings for libdtrace
>   dtrace.spec: add python bindings packaging
>   runtest.sh: Export PYTHONPATH when running tests in-tree
>   test: add tests for python bindings
>   dtrace: Add a PCP PMDA to expose DTrace data as metrics
>   dtrace.spec: Add optional PCP PMDA packaging pcp-pmda-dtrace
> 
>  GNUmakefile                                   |    2 +
>  bindings/Build                                |   35 +
>  bindings/python/README.md                     |  167 ++
>  bindings/python/pyproject.toml                |    3 +
>  bindings/python/setup.py                      |   67 +
>  bindings/python/src/pydtrace_module.c         | 1997 +++++++++++++++++
>  configure                                     |    4 +-
>  dtrace.spec                                   |   58 +-
>  libdtrace/dt_aggregate.c                      |    1 +
>  libdtrace/dt_bpf.c                            |    6 +-
>  libdtrace/dt_btf.c                            |   79 +-
>  libdtrace/dt_btf.h                            |    1 +
>  libdtrace/dt_consume.c                        |  340 +--
>  libdtrace/dt_impl.h                           |    7 +-
>  libdtrace/dt_math.h                           |  358 +++
>  libdtrace/dt_open.c                           |    5 +-
>  libdtrace/dt_printf.c                         |    1 +
>  libdtrace/dt_prov_dtrace.c                    |    6 +-
>  pcp/Build                                     |   56 +
>  pcp/Install                                   |   37 +
>  pcp/README.md                                 |  260 +++
>  pcp/Remove                                    |   27 +
>  pcp/autostart.d/.gitkeep                      |    0
>  pcp/dtrace.conf                               |   12 +
>  pcp/examples/packet_drop_reasons.d            |   14 +
>  pcp/examples/packet_drop_reasons.json         |    8 +
>  pcp/examples/syscall_counts.d                 |   11 +
>  pcp/examples/syscall_counts.json              |    8 +
>  pcp/pmdadtrace.python                         | 1232 ++++++++++
>  runtest.sh                                    |    2 +
>  test/unittest/python/tst.aggr-actions.sh      |  133 ++
>  test/unittest/python/tst.aggr-change.sh       |   75 +
>  test/unittest/python/tst.aggr-stack.sh        |   61 +
>  test/unittest/python/tst.aggr.sh              |   58 +
>  test/unittest/python/tst.exit.sh              |   64 +
>  test/unittest/python/tst.multi-session-end.sh |   75 +
>  test/unittest/python/tst.multi-session.sh     |   58 +
>  test/unittest/python/tst.proc-create.sh       |   56 +
>  test/unittest/python/tst.proc-exit.sh         |   57 +
>  test/unittest/python/tst.proc-grab.sh         |   81 +
>  40 files changed, 5165 insertions(+), 357 deletions(-)
>  create mode 100644 bindings/Build
>  create mode 100644 bindings/python/README.md
>  create mode 100644 bindings/python/pyproject.toml
>  create mode 100644 bindings/python/setup.py
>  create mode 100644 bindings/python/src/pydtrace_module.c
>  create mode 100644 libdtrace/dt_math.h
>  create mode 100644 pcp/Build
>  create mode 100755 pcp/Install
>  create mode 100644 pcp/README.md
>  create mode 100755 pcp/Remove
>  create mode 100644 pcp/autostart.d/.gitkeep
>  create mode 100644 pcp/dtrace.conf
>  create mode 100644 pcp/examples/packet_drop_reasons.d
>  create mode 100644 pcp/examples/packet_drop_reasons.json
>  create mode 100644 pcp/examples/syscall_counts.d
>  create mode 100644 pcp/examples/syscall_counts.json
>  create mode 100755 pcp/pmdadtrace.python
>  create mode 100755 test/unittest/python/tst.aggr-actions.sh
>  create mode 100755 test/unittest/python/tst.aggr-change.sh
>  create mode 100755 test/unittest/python/tst.aggr-stack.sh
>  create mode 100755 test/unittest/python/tst.aggr.sh
>  create mode 100755 test/unittest/python/tst.exit.sh
>  create mode 100755 test/unittest/python/tst.multi-session-end.sh
>  create mode 100755 test/unittest/python/tst.multi-session.sh
>  create mode 100755 test/unittest/python/tst.proc-create.sh
>  create mode 100755 test/unittest/python/tst.proc-exit.sh
>  create mode 100755 test/unittest/python/tst.proc-grab.sh
> 
> -- 
> 2.43.5
>