Re: Question about SCHED_DEADLINE and cgroupv2/cpuset interaction

Hi Tobias,

giving my 2 cents here.

On Tue, 2026-04-28 at 15:52 +0200, Tobias Jakobi wrote:
> For the RT work we have these slices.
> 
> root@cm4-fat:/etc/systemd/system# cat workload.slice
> [Slice]
> AllowedCPUs=2-3
> 
> root@cm4-fat:/etc/systemd/system# cat workload-core2.slice
> [Slice]
> AllowedCPUs=2
> 

Applying the configuration above, I get both slices as member
partitions (not root):

head /sys/fs/cgroup/workload.slice/{,*/}cpuset.cpus.partition 
==> /sys/fs/cgroup/workload.slice/cpuset.cpus.partition <==
member

==> /sys/fs/cgroup/workload.slice/workload-core2.slice/cpuset.cpus.partition <==
member

Running your example in either slice doesn't work in my case.
That's because in the admission control we "Don't allow tasks with an
affinity mask smaller than the entire root_domain to become
SCHED_DEADLINE" [1].

In short that's to make the admission control able to give guarantees.

Setting the partition as root: 

echo root > /sys/fs/cgroup/workload.slice/cpuset.cpus.partition
echo root > /sys/fs/cgroup/workload.slice/workload-core2.slice/cpuset.cpus.partition

Fixes the issue in my case.
I don't really know how to specify that in systemd and I'm assuming
your version/configuration of systemd was somehow setting workload-
slice as root automatically, that's likely why it was working.

Could you try this on your side?

Cheers,
Gabriele

[1] -
https://elixir.bootlin.com/linux/v7.0.1/source/kernel/sched/syscalls.c#L627

> Note that core2 is a child of workload, i.e. on the cgroup level it 
> looks like this.
> 
> root@cm4-fat:~# find /sys/fs/cgroup/ -name workload\*.slice
> /sys/fs/cgroup/workload.slice
> /sys/fs/cgroup/workload.slice/workload-core2.slice
> 
> So what's exactly this problem I'm seeing? It's this here.
> 
> root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty chrt 
> --sched-runtime 100000 --sched-period 200000 --deadline 0 echo this
> works
> Running as unit: run-p676-i976.service; invocation ID: 
> d59dc443f66f46a6ba2dfd125e04f830
> Press ^] three times within 1s to disconnect TTY.
> chrt: failed to set pid 0's policy: Operation not permitted
> 
> What works however is this here.
> 
> root@cm4-fat:~# systemd-run --slice=workload.slice --pty chrt 
> --sched-runtime 100000 --sched-period 200000 --deadline 0 echo this
> works
> Running as unit: run-p670-i970.service
> Press ^] three times within 1s to disconnect TTY.
> this works
> 
> I double checked the slice settings, and they apply.
> 
> root@cm4-fat:~# systemd-run --slice=workload.slice --pty bash -c
> 'grep 
> Cpus_allowed_list /proc/$BASHPID/status'
> Running as unit: run-p727-i1027.service
> Press ^] three times within 1s to disconnect TTY.
> Cpus_allowed_list:      2-3
> 
> Once again for the core2 slice.
> 
> root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty bash -
> c 
> 'grep Cpus_allowed_list /proc/$BASHPID/status'
> Running as unit: run-p732-i1032.service
> Press ^] three times within 1s to disconnect TTY.
> Cpus_allowed_list:      2
> 
> This really puzzled here. Why would this work for AllowedCPUs=2-3,
> but 
> not for AllowedCPUs=2?
> 
> I then dove in the documentation and the code, trying to understand 
> where the -EPERM was coming from. I finally decided to disable the 
> bandwidth check that the kernel performs.
> 
> root@cm4-fat:~# echo -1 > /proc/sys/kernel/sched_rt_runtime_us
> root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty chrt 
> --sched-runtime 100000 --sched-period 200000 --deadline 0 echo this
> works
> Running as unit: run-p760-i1060.service
> Press ^] three times within 1s to disconnect TTY.
> this works
> 
> And voila, suddenly everything works. I fail to understand though why
> the bandwidth check fails here. I mean, 100000 / 200000, that's fifty
> percent, so the check should pass.
> 
> And apparantly the ratio itself doesn't seem to have any effect. I 
> enabled the check again and reduced the runtime so that the ratio is 
> just 5%. Still the same.
> 
> root@cm4-fat:~# echo 950000 > /proc/sys/kernel/sched_rt_runtime_us
> root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty chrt 
> --sched-runtime 1000 --sched-period 200000 --deadline 0 echo this
> works
> Running as unit: run-p838-i1138.service
> Press ^] three times within 1s to disconnect TTY.
> chrt: failed to set pid 0's policy: Operation not permitted
> 
> I looked around some more, made sure that CONFIG_RT_GROUP_SCHED=n,
> but 
> the more I think about, the more I get the impression that I totally 
> misunderstand how the bandwidth check works.
> 
> To be explicit, by bandwidth check I mean the line that is found in 
> section 4.1. here: 
> https://www.kernel.org/doc/Documentation/scheduler/sched-deadline.rst
> 
> Would be nice if someone could shed some light on this. Thanks in
> advance!
> 
> With best wishes,
> Tobias Jakobi
>