Re: [RESEND PATCH v4 05/15] x86,fs/resctrl: Introduce architecture hooks to program kernel-mode

Hi Reinette,

On 8/10/2026 10:14 PM, Reinette Chatre wrote:
> Hi Babu,
> 
> On 7/7/26 2:50 PM, Babu Moger wrote:
>> Kernel-mode policies defined by enum resctrl_kernel_mode must be applied to
> 
> "Kernel-mode policies" -> "Kernel modes"?

Ack.
> 
>> each affected CPU whenever a policy is selected or its scope changes.
> 
> What is an "affected CPU"?
> 

Kernel mode policies defined by enum resctrl_kernel_mode must be applied 
to each CPU whenever user space changes the kernel mode or modifies the 
CPUs associated with the active kernel mode.


> 
> policy -> "mode"? or rather: "whenever a policy is selected or its scope changes" ->
> "whenever user space switches the kernel mode or changes which CPUs are associated
> with the active kernel mode"?
> 
>> Generic resctrl therefore requires an architecture-specific interface to
>> program allocation and monitoring associations in hardware across a given
>> CPU mask.
>>
>> Introduce a helper, resctrl_arch_configure_kmode(),  to handle kernel-mode
>> programming. On x86/AMD systems, this helper programs the
>> MSR_IA32_PQR_PLZA_ASSOC register on all online CPUs in the specified mask
>> via on_each_cpu_mask(). Also provide a no-op stub for MPAM systems.
> 
> No need to describe the code details, please just make it high level of what
> the code accomplished as opposed to describing the code self.

ok.

> 
>>
>> Generic resctrl does not invoke this hook yet; it will be used when user
>> space selects a kernel-mode policy or updates the associated CPU set.
> 
> Looking ahead how this arch helper is used it really is a "one size fits all"
> based on what AMD requires. Specifically, as I see it this architecture helper
> is called under three very different scenarios:
> - A new kernel mode is activated
> - CPUs are added/removed from an active kernel mode
> - A kernel mode is de-activated.
> 
> There is no way for an architecture to distinguish these three scenarios. An architecture
> that, for example, needs to do some arch-specific init to support a particular mode will
> not know when it should do this.
> 
> This "one size fits all" may be ok for an initial approach until we learn what other
> architectures require, but the API needs to be clear on when and how architecture can
> expect it to be called from resctrl fs.

ack.
> 
> Consider, for example, the API description containing text/contract like:
> - If a per-cpu kernel mode is active when user space switches to a new per-cpu kernel
>    mode then resctrl_arch_configure_kmode() will first be called to de-activate the
>    active kernel mode on all CPUs that the kernel mode is active on.
> - When user space switches to a new per-cpu kernel mode then resctrl_arch_configure_kmode() is
>    called with cpu_online_mask.
> - When user space adds a CPU to an active per-cpu kernel mode ...
> - When user space removes a CPU from to an active per-cpu kernel mode ...
> - resctrl fs will always provide the same closid, rmid, and "assign_mon" parameters when
>    activating a kernel mode, all interactions (adding/removing CPU) while the kernel mode is
>    active, as well as when de-activating the kernel mode.

Will add these texts. Thanks.

> 
>>
>> Signed-off-by: Babu Moger <[email protected]>
>> ---
>> v4: Added assign_mon parameter in resctrl_arch_configure_kmode() to program the RMID
>>      as discussed in below.
>>      https://lore.kernel.org/lkml/[email protected]/
>>      Changed cpumask type to "const struct cpumask *cpu_mask".
>>      Added MPAM stub to avoid any linking issues when resctrl_arch_configure_kmode()
>>      is called from FS layer. Thanks to Qinyun.
>>      Re-wrote the changelog to be generic.
>>      Updated code comments.
>>
>> v3: Removed task based PLZA implementation so related changes are removed.
>>      Removed handling of rmid_en as it is not required. The group type assigned
>>      will be different so the monitoring part is already taken care.
>>      Updated the change log with details.
>>      Removed resctrl_arch_set_kmode() as arch only provides the modes supported.
>>      It is FS which decided which mode to apply.
>>
>> v2: Updated the commit message to include the sequence of steps to enable PLZA.
>>      Added mode code comments for clarity.
>>      Added kmode to functin names to be generic.
>> ---
>>   arch/x86/kernel/cpu/resctrl/ctrlmondata.c | 36 +++++++++++++++++++++++
>>   drivers/resctrl/mpam_resctrl.c            |  5 ++++
>>   include/linux/resctrl.h                   | 15 ++++++++++
>>   3 files changed, 56 insertions(+)
>>
>> diff --git a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
>> index b20e705606b8..025f139434f2 100644
>> --- a/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
>> +++ b/arch/x86/kernel/cpu/resctrl/ctrlmondata.c
>> @@ -131,3 +131,39 @@ int resctrl_arch_io_alloc_enable(struct rdt_resource *r, bool enable)
>>   
>>   	return 0;
>>   }
>> +
>> +static void resctrl_kmode_set_one_amd(void *arg)
>> +{
>> +	union msr_pqr_plza_assoc *plza = arg;
>> +
>> +	wrmsrq(MSR_IA32_PQR_PLZA_ASSOC, plza->full);
>> +}
>> +
>> +/*
>> + * Program Privilege Level Zero Association (PLZA) on @cpu_mask.
>> + *
>> + * When @enable is true, CPL 0 allocation traffic on the targeted CPUs uses
>> + * @closid from MSR_IA32_PQR_PLZA_ASSOC instead of the CLOSID from
>> + * MSR_IA32_PQR_ASSOC. Monitoring is redirected to @rmid only when
>> + * @assign_mon is true; otherwise kernel-mode monitoring continues to use the
>> + * RMID associated with the current task.
>> + *
>> + * @cpu_mask:	CPUs whose PLZA MSR should be updated.
>> + * @closid:	CLOSID to use for kernel-mode allocation when PLZA is enabled.
>> + * @rmid:	RMID to use for kernel-mode monitoring when @assign_mon is true.
>> + * @assign_mon:	Whether PLZA should provide the kernel-mode RMID.
>> + * @enable:	Whether PLZA should provide the kernel-mode association.
>> + */
>> +void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid, u32 rmid,
>> +				  bool assign_mon, bool enable)

Need to add "assign_ctrl" also.

>> +{
>> +	union msr_pqr_plza_assoc plza = { 0 };
>> +
>> +	plza.split.rmid = rmid;
>> +	plza.split.rmid_en = assign_mon;
>> +	plza.split.closid = closid;
>> +	plza.split.closid_en = 1;
>> +	plza.split.plza_en = enable;
>> +
>> +	on_each_cpu_mask(cpu_mask, resctrl_kmode_set_one_amd, &plza, 1);
>> +}
>> diff --git a/drivers/resctrl/mpam_resctrl.c b/drivers/resctrl/mpam_resctrl.c
>> index 226ff6f532fa..630b6cfc0269 100644
>> --- a/drivers/resctrl/mpam_resctrl.c
>> +++ b/drivers/resctrl/mpam_resctrl.c
>> @@ -158,6 +158,11 @@ bool resctrl_arch_get_io_alloc_enabled(struct rdt_resource *r)
>>   	return false;
>>   }
>>   
>> +void resctrl_arch_configure_kmode(const struct cpumask *cpu_mask, u32 closid,
>> +				  u32 rmid, bool assign_mon, bool enable)
>> +{
>> +}
>> +
>>   void resctrl_arch_pre_mount(void)
>>   {
>>   }
>> diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h
>> index c7abed51cd5f..47db34dd167e 100644
>> --- a/include/linux/resctrl.h
>> +++ b/include/linux/resctrl.h
>> @@ -734,6 +734,21 @@ enum resctrl_kernel_mode {
>>   
>>   #define RESCTRL_NUM_KERNEL_MODES (RESCTRL_KMODE_LAST + 1)
>>   
>> +/**
>> + * resctrl_arch_configure_kmode() - Program kernel-mode association on CPUs
>> + * @cpu_mask:	CPUs to update; the architecture applies the change on the
>> + *		online subset of this mask.
> 
> Could architecture not expect cpu_mask to only contain online CPUs ?

Yes. It should be only online CPUs.  Let me change the text little bit.

> 
>> + * @closid:	Allocation class for kernel-mode traffic. On x86 this is the
> 
> "Allocation class" ? Care should only be taken when using "closid" outside of
> kernel. Please see all the other examples of arch API in this file that uses closid.

Sure.

> 
>> + *		CLOSID programmed when allocation is assigned for kernel work.
>> + * @rmid:	Monitoring context for kernel-mode traffic. On x86 this is the
> 
> "Monitoring context" ? At this time it is quite clear to architectures how to do
> needed mapping.
> 
> Some archs will use closid/rmid separately, others will consider them together.

Yes. Will change based on other text example.

> 
>> + *		RMID programmed when monitoring is assigned for kernel work.
>> + * @assign_mon:	true to assign @rmid for kernel work; false to inherit
> 
> This is where the only distinction is required when considering other architectures.
> Note that, from user space perspective, it is a monitoring group, potentially identified
> with both closid/rmid  that is assigned, not just an rmid.
> This is thus not a request to "assign @rmid for kernel work" but instead something like
> "kernel work should be monitored by resource group identified by @rmid, or both @closid
> and @rmid, depending on the architecture"

Sure.

> 
>> + *		monitoring from the user task.
>> + * @enable:	true to enable kernel-mode association on the targeted CPUs.
> 
> "the targeted CPUs" -> CPUs in @cpu_mask?
> Please be explicit what is expected from architecture when "false" is provided.

Sure. Will add that text.

Thanks
Babu