[PATCH bpf-next v1 05/14] bpf, s390: JIT arena kfunc argument rebasing
Kumar Kartikeya Dwivedi <[email protected]>
| Newsgroups | org.kernel.vger.bpf |
|---|---|
| Message-ID | <[email protected]> |
Kfuncs with __arena arguments expect kernel addresses, while BPF programs carry arena pointers as zero-extended 32-bit offsets. The s390 JIT does not translate those offsets at the call boundary, so advertising arena kfunc argument support would pass invalid addresses to the callee. Use the kfunc BTF model after the existing argument width normalization to find arena arguments. Load kern_vm_start from the existing arena literal once per call, truncate each tagged argument with llgfr, and add the base. For __arena__nullable, test the truncated value and skip the add so offset zero remains NULL. Advertise the kfunc-specific arena argument capability. Struct_ops trampoline conversion is independent and remains disabled until its own support is added. Cc: Ilya Leoshkevich <[email protected]> Cc: Heiko Carstens <[email protected]> Cc: Vasily Gorbik <[email protected]> Signed-off-by: Kumar Kartikeya Dwivedi <[email protected]> --- arch/s390/net/bpf_jit_comp.c | 49 ++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/arch/s390/net/bpf_jit_comp.c b/arch/s390/net/bpf_jit_comp.c index c46872b071ce..20b6e53999b3 100644 --- a/arch/s390/net/bpf_jit_comp.c +++ b/arch/s390/net/bpf_jit_comp.c @@ -946,6 +946,48 @@ static int sign_zero_extend(struct bpf_jit *jit, int r, u8 size, u8 flags) } } +/* + * Rebase the __arena args of a kfunc call to arena kernel addresses, + * rN = kern_vm_start + (u32)rN. A nullable arg preserves NULL by skipping + * the add, tested on the truncated value as arena NULL is offset 0. + */ +static int emit_kfunc_arena_args(struct bpf_jit *jit, struct bpf_prog *fp, + const struct btf_func_model *m) +{ + bool base_loaded = false; + int i; + + for (i = 0; i < min_t(int, m->nr_args, MAX_BPF_FUNC_REG_ARGS); i++) { + u8 flags = m->arg_flags[i]; + int arg = BPF_REG_1 + i; + + if (!(flags & BTF_FMODEL_ARENA_ARG)) + continue; + if (WARN_ON_ONCE(!fp->aux->arena)) + return -EINVAL; + + if (!base_loaded) { + /* lgrl %w0,kern_arena */ + EMIT6_PCREL_RILB(0xc4080000, REG_W0, jit->kern_arena); + base_loaded = true; + } + + /* llgfr %arg,%arg: truncate and clear the upper 32 bits */ + EMIT4(0xb9160000, arg, arg); + if (flags & BTF_FMODEL_NULLABLE_ARG) { + /* ltgr %arg,%arg */ + EMIT4(0xb9020000, arg, arg); + /* brc 8,1f */ + EMIT4_PCREL_RIC(0xa7040000, 8, jit->prg + 8); + } + /* agr %arg,%w0 */ + EMIT4(0xb9080000, arg, REG_W0); + /* 1: */ + } + + return 0; +} + /* * Compile one eBPF instruction into s390x code * @@ -1867,6 +1909,8 @@ static noinline int bpf_jit_insn(struct bpf_jit *jit, struct bpf_prog *fp, m->arg_flags[j])) return -1; } + if (emit_kfunc_arena_args(jit, fp, m)) + return -1; } if ((void *)func == arch_bpf_timed_may_goto) { @@ -2459,6 +2503,11 @@ bool bpf_jit_supports_kfunc_call(void) return true; } +bool bpf_jit_supports_arena_kfunc_args(void) +{ + return true; +} + bool bpf_jit_supports_far_kfunc_call(void) { return true; -- 2.53.0