RE: Simultaneous access to same CPU/counter across processes crashes hwpmc (TSC and other classes)

Public

> -----Original Message-----
> From: [email protected] <owner-freebsd-
> [email protected]> On Behalf Of Alexander Leidinger
> Sent: Thursday, August 20, 2026 7:55 AM
> To: Nascimento, Anderson Eduardo
> <[email protected]>
> Cc: [email protected]
> Subject: Re: Simultaneous access to same CPU/counter across processes
> crashes hwpmc (TSC and other classes)
>
> Am 2026-08-19 16:15, schrieb Nascimento, Anderson Eduardo:
> > Public
> >
> > Hello all,
>
> Hello. I'm working on some fixes for several issues I found in the PMC code.
> Your finding is not one of them, but I want to provide a bit of info... at least as
> far as I understand the issue.

Hello Alexander,

That seems great. Good to know that.

>
> > I am working on developing a new class of PMCs for the hwpmc subsystem
> > and I ran into a kernel crash while testing my implementation. I
> > believe I have identified the reason the crash happens, and I could
> > try to fix it on the affected classes, but so far it's still not clear
> > to me what model the hwpmc subsystem intends to enforce. Should
> > simultaneous access to the same CPU and counter by different processes
> > be allowed?
> >
> > The issue happens when pmcstat is executed by two different processes
> > on the same counter and CPU core. The affected classes contain a
> > KASSERT() call that is triggered on a kernel built with INVARIANTS.
> > The command below, when executed on two different shells, triggers it
> > regardless of the CPU vendor:
> >
> > pmcstat -c 0 -s tsc-cycles -w 1
> >
> > While investigating the issue, I noticed that the function
> > pmc_can_allocate_rowindex() prevents the same process from
> > simultaneously accessing the rowindex, but this check is bypassed when
> > a different process is doing so. Another mechanism present in hwpmc,
> > as far as I can tell, is the PMC_PHW_FLAG_IS_SHAREABLE flag. In the
> > Intel IAF implementation, the crash doesn't happen because it doesn't
> > allow simultaneous access by different threads. The command below
> > returns an error when there's already an equivalent pmcstat process
> running:
> >
> > pmcstat -c 0 -s unhalted-core-cycles -w 1
> >
> > Besides not having the affected KASSERT that is present in the
> > affected classes, the Intel IAF implementation also doesn't set the
> > PMC_PHW_FLAG_IS_SHAREABLE flag.
> >
> > The questions are:
> >
> > 1 - Should the TSC and the other affected classes allow simultaneous
> > access by different processes?
> > 2 - If the answer is NO, I can test a patch that removes the
> > PMC_PHW_FLAG_IS_SHAREABLE flag from the affected classes and see if
> > this solves the problem.
> > 3 - If the answer is YES, how can we implement simultaneous access by
> > different processes in a safe way?
> >
> > Regarding question 3, I did some tests by removing
> > PMC_PHW_FLAG_IS_SHAREABLE from the Intel IAF implementation and
> adding
> > the equivalent KASSERT(), and in this case the problem could be
> > triggered the same way. So I believe there's nothing special about
> > Intel IAF that avoids this from happening, other than the flag and the
> > missing KASSERT(). That said, I also tested other events, such as
> > uops_issued.any, and simultaneous access by different processes seemed
> > to work fine. However, I'm not certain whether it is actually working
> > correctly or whether something is going wrong without being detected -
> > I didn't investigate this further for that event.
>
> Deleting the assertion would not be enough. pmc_release_pmc_descriptor()
> asserts phw->phw_pmc == pm and then calls pcd_config_pmc(cpu, adjri,
> NULL) unconditionally. With two owners on one row, the second allocation
> silently overwrites phw_pmc, and the first release unconfigures the survivor's
> counter. The slot is single-occupancy in structure, not just in the KASSERT —
> so his option 3 ("implement sharing safely") means refcounting or listing
> phw_pmc, not relaxing a check. Option 2 (drop the flag from TSC and RAPL) is
> the self-consistent small fix.
>
> It is root-only and refused in a jail. System-mode allocation needs
> PRIV_PMC_SYSTEM unless security.bsd.unprivileged_syspmcs, which defaults
> to 0. So this can not be triggered by a normal user.
>

Thanks for your explanation. It's well aligned with my understanding. If nobody else replies, I'll test and send a patch over the next few days removing the flag from the affected classes. This forbids simultaneous access, but today it's not working anyway.

> Bye,
> Alexander.
>
> --
> http://www.Leidinger.net [email protected]: PGP
> 0x8F31830F9F2772BF
> http://www.FreeBSD.org    [email protected]  : PGP
> 0x8F31830F9F2772BF