git: 0bb238104c - main - Status/2026Q2/hpc-ports-modernization.adoc

Lorenzo Salvadore <[email protected]> Mon, 20 Jul 2026 12:00:06 +0000
Newsgroups gmane.os.freebsd.devel.cvs.doc
Message-ID <[email protected]>
The branch main has been updated by salvadore:

URL: https://cgit.FreeBSD.org/doc/commit/?id=0bb238104c9e0644331f7fda6adb2e70e0e9448a

commit 0bb238104c9e0644331f7fda6adb2e70e0e9448a
Author:     GenericRikka <[email protected]>
AuthorDate: 2026-07-08 16:35:45 +0000
Commit:     Lorenzo Salvadore <[email protected]>
CommitDate: 2026-07-20 11:59:34 +0000

    Status/2026Q2/hpc-ports-modernization.adoc
    
    Pull Request:   https://github.com/freebsd/freebsd-doc/pull/695
---
 .../hpc-ports-modernization.adoc                   | 46 ++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/website/content/en/status/report-2026-04-2026-07/hpc-ports-modernization.adoc b/website/content/en/status/report-2026-04-2026-07/hpc-ports-modernization.adoc
new file mode 100644
index 0000000000..907f41eb7a
--- /dev/null
+++ b/website/content/en/status/report-2026-04-2026-07/hpc-ports-modernization.adoc
@@ -0,0 +1,46 @@
+=== FreeBSD HPC Ports Modernization: Slurm 26.05, UCX 1.20 Upstreaming, and MPI-Parallel File Utilities
+
+Links: +
+link:https://cgit.freebsd.org/ports/tree/sysutils/slurm-wlm/[sysutils/slurm-wlm] URL: link:https://cgit.freebsd.org/ports/tree/sysutils/slurm-wlm/[] +
+link:https://cgit.freebsd.org/ports/tree/net/ucx/[net/ucx] URL: link:https://cgit.freebsd.org/ports/tree/net/ucx/[] +
+link:https://cgit.freebsd.org/ports/tree/sysutils/mpifileutils/[sysutils/mpifileutils] URL: link:https://cgit.freebsd.org/ports/tree/sysutils/mpifileutils/[] +
+link:https://cgit.freebsd.org/ports/tree/devel/libcircle/[devel/libcircle] URL: link:https://cgit.freebsd.org/ports/tree/devel/libcircle/[] +
+link:https://cgit.freebsd.org/ports/tree/devel/lwgrp/[devel/lwgrp] URL: link:https://cgit.freebsd.org/ports/tree/devel/lwgrp/[] +
+link:https://cgit.freebsd.org/ports/tree/devel/dtcmp/[devel/dtcmp] URL: link:https://cgit.freebsd.org/ports/tree/devel/dtcmp/[] +
+link:https://cgit.freebsd.org/ports/tree/benchmarks/py-reframe-hpc/[benchmarks/py-reframe-hpc] URL: link:https://cgit.freebsd.org/ports/tree/benchmarks/py-reframe-hpc/[] +
+link:https://github.com/openucx/ucx/pull/11354[openucx/ucx#11354: UCS/TYPE portability fixes for non-glibc/Clang environments (merged)] URL: link:https://github.com/openucx/ucx/pull/11354[] +
+link:https://github.com/openucx/ucx/pull/11549[openucx/ucx#11549: UCS/SYS portability fixes for non-Linux platforms (in review)] URL: link:https://github.com/openucx/ucx/pull/11549[] +
+link:https://github.com/hpc/mpifileutils/pull/664[hpc/mpifileutils#664: Portability fixes for FreeBSD/non-Linux builds (in review)] URL: link:https://github.com/hpc/mpifileutils/pull/664[] +
+link:https://kavocado.net/reports/[Kavocado Monthly Status Reports – FreeBSD HPC notes] URL: link:https://kavocado.net/reports/[]
+
+Contact: Generic Rikka <[email protected]>
+
+This report continues the FreeBSD HPC Ports Modernization initiative.
+Previous quarters focused on bringing the Slurm + PMIx + PRRTE + UCX stack up to date and filling gaps in the surrounding ecosystem.
+This quarter's work centered on three things: keeping the core scheduler and communication libraries current against fast-moving upstream releases, upstreaming the FreeBSD portability fixes accumulated while maintaining these ports so the local patchsets keep shrinking, and adding a new port for MPI-parallel file utilities that are common on large HPC filesystems.
+
+==== Work completed
+
+* Updated package:sysutils/slurm-wlm[] twice this quarter, from 25.11.4 to 25.11.5 and then from 25.11.5 to 26.05.1, tracking the latest upstream releases.
+* Fixed several FreeBSD-specific runtime issues in package:net/ucx[]: hardened async thread state handling, corrected UCM relocation handling, fixed mm signal socket binding, and resolved other FreeBSD runtime portability issues.
+* Updated package:net/ucx[] from 1.20.0 to 1.20.1, and separately fixed a libucm early-init panic together with several gtest suite failures uncovered while validating the update.
+* Added package:sysutils/mpifileutils[] as a new port, along with its dependency stack: package:devel/libcircle[], package:devel/lwgrp[], and package:devel/dtcmp[]. These provide MPI-parallel file utilities (copy, remove, checksum, etc.) commonly used on large HPC filesystems.
+* Updated package:benchmarks/py-reframe-hpc[] twice this quarter to track upstream releases of the regression testing framework.
+* Upstreamed "UCS/TYPE: Portability fixes for non-glibc/Clang environments" to UCX, which was reviewed and merged.
+* Opened "UCS/SYS: portability fixes for non-Linux platforms" against UCX, covering hostname buffer sizing, MAC address retrieval via `getifaddrs()`/`AF_LINK`, shared-memory error reporting, `mmap`/`munmap`-based reallocation, and `cpuset_getaffinity()`-based thread affinity queries; currently addressing review feedback.
+* Opened a portability PR against mpifileutils guarding Linux-only APIs for FreeBSD and other non-Linux builds; currently in review.
+* Collaborated with SchedMD on a Slurm configure-time probe for `H5PTopen`, used to detect the HDF5 high-level library, which was merged upstream.
+
+==== Work in progress
+
+* Addressing reviewer feedback on the open UCX and mpifileutils portability pull requests to get them merged upstream and reduce the FreeBSD ports' local patchsets further.
+* Continuing collaboration with SchedMD to upstream additional portability fixes discovered while maintaining package:sysutils/slurm-wlm[] on FreeBSD, including BSD socket-handling improvements.
+* Evaluating a `proctrack/freebsd_reaper` Slurm plugin design, made possible by recently landed FreeBSD kernel support for exposing reaper metadata in process information.
+* Continuing to track upstream Slurm, UCX, and PMIx/PRRTE releases closely to keep the FreeBSD ports current with minimal local patching.
+
+==== Future plans
+
+* Land the outstanding UCX and mpifileutils upstream pull requests, and continue identifying further portability issues to upstream rather than patch locally.
+* Prototype and submit the Slurm `proctrack/freebsd_reaper` plugin once the underlying kernel interface work is finalized.
+* Continue expanding the HPC software ecosystem available in the FreeBSD Ports Collection and reduce local patchsets across the stack wherever upstream acceptance is possible.
+* Document a reference Slurm + OpenMPI + PMIx + PRRTE + UCX deployment on FreeBSD to lower the barrier for new sites experimenting with FreeBSD in an HPC context.