bpf: move sleepable flag from bpf_prog_aux to bpf_prog
authorAndrii Nakryiko <andrii@kernel.org>
Sat, 9 Mar 2024 00:47:39 +0000 (16:47 -0800)
committerAlexei Starovoitov <ast@kernel.org>
Mon, 11 Mar 2024 23:41:25 +0000 (16:41 -0700)
commit66c8473135c62f478301a0e5b3012f203562dfa6
tree38552c4bba5b6de12904aa40539c2d14950c17a8
parentd6170e4aaf86424c24ce06e355b4573daa891b17
bpf: move sleepable flag from bpf_prog_aux to bpf_prog

prog->aux->sleepable is checked very frequently as part of (some) BPF
program run hot paths. So this extra aux indirection seems wasteful and
on busy systems might cause unnecessary memory cache misses.

Let's move sleepable flag into prog itself to eliminate unnecessary
pointer dereference.

Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Message-ID: <20240309004739.2961431-1-andrii@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
include/linux/bpf.h
kernel/bpf/bpf_iter.c
kernel/bpf/core.c
kernel/bpf/syscall.c
kernel/bpf/trampoline.c
kernel/bpf/verifier.c
kernel/events/core.c
kernel/trace/bpf_trace.c
net/bpf/bpf_dummy_struct_ops.c