BPF verifier throws error "expected=map

I'm writing a little eBPF program to set a flag whenever the tracepoint consume_skb is triggered. When I try to load it however, the BPF verifier complains, saying R1 type=inv expected=map_ptr when trying to access the map. I tried to follow the samples given in the linux kernel as closely as possible, but I am failing at even reading anything from the map.

eBPF program

#include <uapi/linux/bpf.h>
#include <bpf/bpf_helpers.h>

struct {
    __uint(type, BPF_MAP_TYPE_ARRAY);
    __type(key, u32);
    __type(value, u32);
    __uint(max_entries, 1);
} my_map SEC(".maps");

SEC("tracepoint/skb/consume_skb")
int bpf_sys(void *ctx)
{
    int index = 0;
    int *value;
    value = bpf_map_lookup_elem(&my_map, &index);
    if (!value)
    {
        return 1;
    }
    return 0;
}
char _license[] SEC("license") = "GPL";

loader, until point of failure

int main()
{
    int bfd;
    unsigned char buf[1024] = {};
    struct bpf_insn *insn;
    union bpf_attr attr = {};
    unsigned char log_buf[4096] = {};
    int pfd;
    int n;

    bfd = open("./test_bpf", O_RDONLY);
    if (bfd < 0)
    {
        printf("open eBPF program error: %s\n", strerror(errno));
        exit(-1);
    }

    n = read(bfd, buf, 1024);
    close(bfd);

    insn = (struct bpf_insn*)buf;
    attr.prog_type = BPF_PROG_TYPE_TRACEPOINT;
    attr.insns = (unsigned long)insn;
    attr.insn_cnt = n / sizeof(struct bpf_insn);
    attr.license = (unsigned long)"GPL";
    attr.log_size = sizeof(log_buf);
    attr.log_buf = (unsigned long)log_buf;
    attr.log_level = 1;

    pfd = syscall(SYS_bpf, BPF_PROG_LOAD, &attr, sizeof(attr));
    if (pfd < 0)
    {
        printf("bpf syscall error: %s\n", strerror(errno));
        printf("log_buf = %s\n", log_buf);
        exit(-1);
    }
...

llvm output

Disassembly of section tracepoint/skb/consume_skb:

0000000000000000 bpf_sys:
; {
       0:   b7 01 00 00 00 00 00 00 r1 = 0
;   int index = 0;
       1:   63 1a fc ff 00 00 00 00 *(u32 *)(r10 - 4) = r1
       2:   bf a2 00 00 00 00 00 00 r2 = r10
       3:   07 02 00 00 fc ff ff ff r2  = -4
;   value = bpf_map_lookup_elem(&my_map, &index);
       4:   18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll
       6:   85 00 00 00 01 00 00 00 call 1
       7:   bf 01 00 00 00 00 00 00 r1 = r0
       8:   b7 00 00 00 01 00 00 00 r0 = 1
;   if (!value)
       9:   15 01 01 00 00 00 00 00 if r1 == 0 goto  1 <LBB0_2>
      10:   b7 00 00 00 00 00 00 00 r0 = 0

0000000000000058 LBB0_2:
; }
      11:   95 00 00 00 00 00 00 00 exit

Error when loading

bpf syscall error: Permission denied
log_buf = 0: (b7) r1 = 0
1: (63) *(u32 *)(r10 -4) = r1
last_idx 1 first_idx 0
regs=2 stack=0 before 0: (b7) r1 = 0
2: (bf) r2 = r10
3: (07) r2  = -4
4: (18) r1 = 0x0
6: (85) call bpf_map_lookup_elem#1
R1 type=inv expected=map_ptr
processed 6 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0

Also, is there a better way to debug BPF error messages?

CodePudding user response：

Issues

Parsing the bytecode

Your loader does not work, at least not directly on the object file compiled from your C code. It just reads from the file, does not parse for an ELF section where the bytecode would be contained. Apparently you're getting the verifier output for the correct eBPF bytecode, though; so I suspect you've been using it in a different way, or with some intermediate step.

Setting up the maps

With regards to your issue, your loader also misses the creation of a map and the ELF relocation of the FD for that map into the bytecode instructions, which would be why you get your error. Typically, when you load a program, several steps occur including:

The loader program (often via libbpf) creates all eBPF maps needed by the eBPF program
Alternatively, it reuses some existing maps. In both cases (existing or new maps), it retrieves a file descriptor for each map (e.g. via their pinned path for existing maps, or returned by the syscall when creating new maps).
Then it inserts this file descriptor into the eBPF bytecode, as an argument to the helper functions used for map access.
At last the loader program loads the updated bytecode into the kernel.
The bytecode goes through the verifier, where the file descriptors are replaced with pointers to the map areas in the kernel memory space.

Your loader is not creating maps (or retrieving FDs for existing, compatible maps), and does not update the bytecode accordingly. So the eBPF program sent to the kernel doesn't say what map it tries to access: we just see, from the verifier logs, 4: (18) r1 = 0x0, where at this stage it should be a memory pointer, such as 4: (18) r1 = 0xffff8ca2c29e3200.

Workarounds

You can obviously extend your loader application to handle maps and ELF relocation. But here are a few other suggestions.

Adding new programs directly amongst the kernel samples is not always ideal. The build system for these samples is rather involved, and relies on a number of header inclusions that are not really standard outside of the kernel repository. On older versions it would also not rely on libbpf but would reimplement most of the loading components, although I think this has improved.

So I would suggest building outside of the kernel repository, if this is not what you have been doing already. In particular, if you're working with C, you should have a look at how libbpf can help handling (loading, attaching, etc.) eBPF programs and other objects.

One of the best starting point would also be libbpf-boostrap, which provides a framework with templates for building eBPF programs and corresponding user space applications very quickly. I would definitely recommend looking at it.

Alternatively, you should be able to load your program with e.g. bpftool prog load <object_file.o> /sys/fs/bpf/<some_path>, although you can't attach it to tracepoints with bpftool only.