[PATCH bpf-next v3] bpf, x86: Add support for BPF_SDIV and BPF_SMOD in the i386 JIT
Nicholas Dudar <[email protected]>
| Newsgroups | org.kernel.vger.bpf,org.kernel.vger.linux-kernel |
|---|---|
| Message-ID | <[email protected]> |
emit_ia32_div_mod_r() emits an unsigned divide (xor edx,edx + div ecx) for BPF_DIV and BPF_MOD regardless of the instruction's signedness, so the i386 JIT does not implement signed BPF_SDIV and BPF_SMOD (off == 1), which get an unsigned quotient and remainder rather than the verifier's and the interpreter's signed result for negative operands. Add signed support. Pass the instruction to emit_ia32_div_mod_r() and, on the signed path (off == 1), emit cdq + idiv ecx instead of xor edx,edx + div ecx, mirroring the cdq/idiv-vs-xor/div split the x86_64 JIT uses. bpf_do_misc_fixups() rewrites the zero-divisor and INT_MIN/-1 cases out of the instruction stream before the JIT runs. The RV32 JIT has the same gap. Signed-off-by: Nicholas Dudar <[email protected]> Assisted-by: Claude:claude-opus-4-8 --- v3: - Correct the incomplete recipient list. v2: - Frame as adding support and drop the Fixes tag, per Pu Lehui [1]. - Pass struct bpf_insn * to emit_ia32_div_mod_r() rather than separate op and is_signed parameters, per Kuan-Wei Chiu. v2: https://lore.kernel.org/bpf/[email protected]/ v1: https://lore.kernel.org/bpf/[email protected]/ [1]: https://lore.kernel.org/bpf/[email protected]/ arch/x86/net/bpf_jit_comp32.c | 25 +++++++++++++++++-------- 1 file changed, 17 insertions(+), 8 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp32.c b/arch/x86/net/bpf_jit_comp32.c index 852baf2e4db4..012161da3e82 100644 --- a/arch/x86/net/bpf_jit_comp32.c +++ b/arch/x86/net/bpf_jit_comp32.c @@ -432,11 +432,13 @@ static inline void emit_ia32_to_be_r64(const u8 dst[], s32 val, * ALU operation (32 bit) * dst = dst (div|mod) src */ -static inline void emit_ia32_div_mod_r(const u8 op, const u8 dst, const u8 src, - bool dstk, bool sstk, u8 **pprog) +static inline void emit_ia32_div_mod_r(const struct bpf_insn *insn, const u8 dst, + const u8 src, bool dstk, bool sstk, u8 **pprog) { u8 *prog = *pprog; int cnt = 0; + const u8 op = BPF_OP(insn->code); + const bool is_signed = (insn->off == 1); if (sstk) /* mov ecx,dword ptr [ebp+off] */ @@ -454,10 +456,17 @@ static inline void emit_ia32_div_mod_r(const u8 op, const u8 dst, const u8 src, /* mov eax,dst */ EMIT2(0x8B, add_2reg(0xC0, dst, IA32_EAX)); - /* xor edx,edx */ - EMIT2(0x31, add_2reg(0xC0, IA32_EDX, IA32_EDX)); - /* div ecx */ - EMIT2(0xF7, add_1reg(0xF0, IA32_ECX)); + if (is_signed) { + /* cdq */ + EMIT1(0x99); + /* idiv ecx */ + EMIT2(0xF7, add_1reg(0xF8, IA32_ECX)); + } else { + /* xor edx,edx */ + EMIT2(0x31, add_2reg(0xC0, IA32_EDX, IA32_EDX)); + /* div ecx */ + EMIT2(0xF7, add_1reg(0xF0, IA32_ECX)); + } if (op == BPF_MOD) { if (dstk) @@ -1795,14 +1804,14 @@ static int do_jit(struct bpf_prog *bpf_prog, int *addrs, u8 *image, case BPF_ALU | BPF_MOD | BPF_X: switch (BPF_SRC(code)) { case BPF_X: - emit_ia32_div_mod_r(BPF_OP(code), dst_lo, + emit_ia32_div_mod_r(insn, dst_lo, src_lo, dstk, sstk, &prog); break; case BPF_K: /* mov ecx,imm32*/ EMIT2_off32(0xC7, add_1reg(0xC0, IA32_ECX), imm32); - emit_ia32_div_mod_r(BPF_OP(code), dst_lo, + emit_ia32_div_mod_r(insn, dst_lo, IA32_ECX, dstk, false, &prog); break; base-commit: 863f3ddd0b8ac65abfb50d3be0869268ac0e277b