bpf: Add x86-64 JIT support for PROBE_MEM32 pseudo instructions.
authorAlexei Starovoitov <ast@kernel.org>
Fri, 8 Mar 2024 01:08:01 +0000 (17:08 -0800)
committerAndrii Nakryiko <andrii@kernel.org>
Mon, 11 Mar 2024 22:37:24 +0000 (15:37 -0700)
commit2fe99eb0ccf2bb73df65ebcbbf2f2ff70e63547b
tree9bc8df3820cfa576dfbff871acff07e3145e1e1d
parent667a86ad9b71d934c444eec193cf3508016f35c5
bpf: Add x86-64 JIT support for PROBE_MEM32 pseudo instructions.

Add support for [LDX | STX | ST], PROBE_MEM32, [B | H | W | DW] instructions.
They are similar to PROBE_MEM instructions with the following differences:
- PROBE_MEM has to check that the address is in the kernel range with
  src_reg + insn->off >= TASK_SIZE_MAX + PAGE_SIZE check
- PROBE_MEM doesn't support store
- PROBE_MEM32 relies on the verifier to clear upper 32-bit in the register
- PROBE_MEM32 adds 64-bit kern_vm_start address (which is stored in %r12 in the prologue)
  Due to bpf_arena constructions such %r12 + %reg + off16 access is guaranteed
  to be within arena virtual range, so no address check at run-time.
- PROBE_MEM32 allows STX and ST. If they fault the store is a nop.
  When LDX faults the destination register is zeroed.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/bpf/20240308010812.89848-4-alexei.starovoitov@gmail.com
arch/x86/net/bpf_jit_comp.c
include/linux/bpf.h
include/linux/filter.h