Question about SCHED_DEADLINE and cgroupv2/cpuset interaction

Hi there,

I'm currently pondering on an odd behaviour that I'm seeing when using 
SCHED_DEADLINE in combination with cgroupv2/cpuset.

A bit about my setup. The hardware is a Raspi CM4 module on some custom 
mainboard.

Kernel is rpi-6.12.y from here: https://github.com/raspberrypi/linux
Although I don't think that this matters much. There's barely any 
difference between vanilla and the Raspi fork when it comes to the 
scheduler. I've also tried this whole thing out with rpi-6.18.y, but I'm 
getting the same behavior there.
Kernel config is nothing special, except for the fact that 
CONFIG_PREEMPT_RT=y.

root@cm4-fat:~# uname -a
Linux cm4-fat 6.12.81+ #1 SMP PREEMPT_RT Tue Apr 14 13:06:22 CEST 2026 
aarch64 GNU/Linux

Anyway, for our application (in the sense of "how do we use the CM4 
module") we have partitioned the CPU resources into two groups. CPUs 0 
and 1 are for all the "menial" tasks, and CPUs 2 and 3 are for 
everything real-time. We use systemd slices for the partitioning, i.e. 
systemd does the cpuset controller setup for us.

Basically it's this setup: 
https://documentation.suse.com/sle-rt/15-SP7/html/SLE-RT-all/cha-shielding-with-systemd.html

We have assigned CPUs 0 and 1 to the machine and the user slice.

Also we have some kernel cmdline arguments that might be relevant for 
this: irqaffinity=0-1 isolcpus=2-3 nohz_full=2-3

For the RT work we have these slices.

root@cm4-fat:/etc/systemd/system# cat workload.slice
[Slice]
AllowedCPUs=2-3

root@cm4-fat:/etc/systemd/system# cat workload-core2.slice
[Slice]
AllowedCPUs=2

Note that core2 is a child of workload, i.e. on the cgroup level it 
looks like this.

root@cm4-fat:~# find /sys/fs/cgroup/ -name workload\*.slice
/sys/fs/cgroup/workload.slice
/sys/fs/cgroup/workload.slice/workload-core2.slice

So what's exactly this problem I'm seeing? It's this here.

root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty chrt 
--sched-runtime 100000 --sched-period 200000 --deadline 0 echo this works
Running as unit: run-p676-i976.service; invocation ID: 
d59dc443f66f46a6ba2dfd125e04f830
Press ^] three times within 1s to disconnect TTY.
chrt: failed to set pid 0's policy: Operation not permitted

What works however is this here.

root@cm4-fat:~# systemd-run --slice=workload.slice --pty chrt 
--sched-runtime 100000 --sched-period 200000 --deadline 0 echo this works
Running as unit: run-p670-i970.service
Press ^] three times within 1s to disconnect TTY.
this works

I double checked the slice settings, and they apply.

root@cm4-fat:~# systemd-run --slice=workload.slice --pty bash -c 'grep 
Cpus_allowed_list /proc/$BASHPID/status'
Running as unit: run-p727-i1027.service
Press ^] three times within 1s to disconnect TTY.
Cpus_allowed_list:      2-3

Once again for the core2 slice.

root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty bash -c 
'grep Cpus_allowed_list /proc/$BASHPID/status'
Running as unit: run-p732-i1032.service
Press ^] three times within 1s to disconnect TTY.
Cpus_allowed_list:      2

This really puzzled here. Why would this work for AllowedCPUs=2-3, but 
not for AllowedCPUs=2?

I then dove in the documentation and the code, trying to understand 
where the -EPERM was coming from. I finally decided to disable the 
bandwidth check that the kernel performs.

root@cm4-fat:~# echo -1 > /proc/sys/kernel/sched_rt_runtime_us
root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty chrt 
--sched-runtime 100000 --sched-period 200000 --deadline 0 echo this works
Running as unit: run-p760-i1060.service
Press ^] three times within 1s to disconnect TTY.
this works

And voila, suddenly everything works. I fail to understand though why 
the bandwidth check fails here. I mean, 100000 / 200000, that's fifty 
percent, so the check should pass.

And apparantly the ratio itself doesn't seem to have any effect. I 
enabled the check again and reduced the runtime so that the ratio is 
just 5%. Still the same.

root@cm4-fat:~# echo 950000 > /proc/sys/kernel/sched_rt_runtime_us
root@cm4-fat:~# systemd-run --slice=workload-core2.slice --pty chrt 
--sched-runtime 1000 --sched-period 200000 --deadline 0 echo this works
Running as unit: run-p838-i1138.service
Press ^] three times within 1s to disconnect TTY.
chrt: failed to set pid 0's policy: Operation not permitted

I looked around some more, made sure that CONFIG_RT_GROUP_SCHED=n, but 
the more I think about, the more I get the impression that I totally 
misunderstand how the bandwidth check works.

To be explicit, by bandwidth check I mean the line that is found in 
section 4.1. here: 
https://www.kernel.org/doc/Documentation/scheduler/sched-deadline.rst

Would be nice if someone could shed some light on this. Thanks in advance!

With best wishes,
Tobias Jakobi