Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sample usage for BPF_PROG_TYPE_NETFILTER #98

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

zq-david-wang
Copy link

BPF_PROG_TYPE_NETFILTER was introduced in 6.4, now with a new kernel, a bpf program could attach to netfilter hooks and handles package in a similiar way as iptables/nftables. By now, 6.5.0, there is no bpf kfunc implemented yet for DNAT/SNAT, and the only thing a bpf program can do is to decide whether to DROP the package or not.

  • netfilter_ip4_blocklist.c/netfilter_ip4_blocklist.bpf.c

This sample code implements a simple ipv4 blacklist.
The bpf program drops package if destination ip address hits a match in the map of type BPF_MAP_TYPE_LPM_TRIE,
The userspace code would load the bpf program, attach it to netfilter's FORWARD/OUTPUT hook, and then write ip patterns into the bpf map.

@tohojo
Copy link
Member

tohojo commented Sep 5, 2023

Instead of directing people to copy the files to the kernel tree, could you please add a Makefile to build the sample as part of this repo? You can copy the Makefile from one of the other directories and adjust appropriately :)

@zq-david-wang
Copy link
Author

Instead of directing people to copy the files to the kernel tree, could you please add a Makefile to build the sample as part of this repo? You can copy the Makefile from one of the other directories and adjust appropriately :)

The problem is with those new enum defined in kernel headers and the "new" vmlinux.h file.
Is it acceptable if I made a copy of vmlinux.h and redefine those enums, I think this can make it build within this repo.

@tohojo
Copy link
Member

tohojo commented Sep 6, 2023

The problem is with those new enum defined in kernel headers and the "new" vmlinux.h file. Is it acceptable if I made a copy of vmlinux.h and redefine those enums, I think this can make it build within this repo.

Sure. Have a look at the existing header files in headers/vmlinux in the project root dir. You can just copy the enum you need from a locally generated vmlinux.h into one of those files; vmlinux_net.h is probably appropriate for the netfilter definition :)

@zq-david-wang
Copy link
Author

New types can be easily dealt with, but the "changed" type bpf_attr in libbpf is hard to make a workaround....

netfilter_ip4_blocklist.c: In function ‘main’:
netfilter_ip4_blocklist.c:58:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   58 |         attr.link_create.netfilter.pf = NFPROTO_IPV4;
      |                         ^
netfilter_ip4_blocklist.c:59:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   59 |         attr.link_create.netfilter.hooknum = NF_INET_FORWARD;
      |                         ^
netfilter_ip4_blocklist.c:60:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   60 |         attr.link_create.netfilter.priority = -128;
      |                         ^
netfilter_ip4_blocklist.c:67:25: error: ‘struct <anonymous>’ has no member named ‘netfilter’
   67 |         attr.link_create.netfilter.hooknum = NF_INET_LOCAL_OUT;

@zq-david-wang
Copy link
Author

I turns out that lastet https://github.com/xdp-project/libbpf has netfilter added to link_create, any chance to make a update to the libbpf submodule?

@tohojo
Copy link
Member

tohojo commented Sep 7, 2023 via email

@zq-david-wang
Copy link
Author

Sad story.... Even I git submodule update --remote lib/libbpf to latest commit, replace headers/linux/bpf.h with lib/libbpf/include/uapi/linux/bpf.h and fix the missing defination for enum BPF_NETFILTER, I still could not get it through and got the following runtime error:

$ sudo ./netfilter_ip4_blocklist 
libbpf: prog 'netfilter_ip4block': BPF program load failed: Invalid argument
libbpf: prog 'netfilter_ip4block': failed to load: -22
libbpf: failed to load object './netfilter_ip4_blocklist.bpf.o'
loading BPF object file failed

The bpf program works when I replace lib/install/lib/libbpf.a with 6.5.0 in-tree version.....

@zq-david-wang
Copy link
Author

Could https://github.com/xdp-project/libbpf.git resync with in-tree version?....

@tohojo
Copy link
Member

tohojo commented Sep 8, 2023 via email

@zq-david-wang
Copy link
Author

nat64-bpf has build error, but it failed the same way before this commit

$ make
sh configure
clang: 18.0.0 (https://github.com/llvm/llvm-project.git 8dce4c56dd760b34b222d3188a0fe26232928406)
libmnl support: yes
libbpf support: submodule
Submodule 'lib/libbpf' (https://github.com/xdp-project/libbpf.git) registered for path 'lib/libbpf'
Submodule 'lib/xdp-tools' (https://github.com/xdp-project/xdp-tools) registered for path 'lib/xdp-tools'
Cloning into '/home/linan/codes/git/bpf-examples/bpf-examples/lib/libbpf'...
Cloning into '/home/linan/codes/git/bpf-examples/bpf-examples/lib/xdp-tools'...
Submodule path 'lib/libbpf': checked out 'a6d7530cb7dff87ac1e64a540e63b67ddde2e0f9'
Submodule path 'lib/xdp-tools': checked out '344b241da22a5358c714d6db1ea6f225f951dbdb'
ELF support: yes
zlib support: yes
libxdp support: submodule
Configuring libxdp to use our libbpf submodule
Found clang binary 'clang' with version 18 (from 'clang version 18.0.0 (https://github.com/llvm/llvm-project.git 8dce4c56dd760b34b222d3188a0fe26232928406)')
using emacs: GNU Emacs 26.1
using bpftool v5.8.0
libbpf support: custom v1.0.0
  perf_buffer__consume support: yes
  btf__load_from_kernel_by_id support: yes
  btf__type_cnt support: yes
  bpf_object__next_map support: yes
  bpf_object__next_program support: yes
  bpf_program__insn_cnt support: yes
  bpf_map_create support: yes
  perf_buffer__new_raw support: yes
  bpf_xdp_attach support: yes
  bpf_map__set_autocreate support: yes
  bpf_prog_test_run_opts support: yes
zlib support: yes
ELF support: yes
pcap support: yes
secure_getenv support: yes

lib

  libbpf
    CC       install/lib/libbpf.a
    INSTALL  install/lib/libbpf.a

  libxdp
    CC       install/lib/libxdp.a
    INSTALL  install/lib/libxdp.a

  util
    CC       json_writer.o
    CC       logging.o

encap-forward
    CLANG    xdp_encap.o
    LLC      xdp_encap.o
    CLANG    tc_bpf_encap.o
    LLC      tc_bpf_encap.o

ktrace-CO-RE
    CC       ktrace01
    CLANG    ktrace01_kern.o
    LLC      ktrace01_kern.o

lsm-nobpf
    CC       lsm-nobpf
    CLANG    lsm-nobpf-kern.o
    LLC      lsm-nobpf-kern.o

nat64-bpf
    CLANG    nat64_kern.o
    LLC      nat64_kern.o
    GEN      nat64_kern.skel.h
libbpf: bad map relo against section 2
Error: failed to open BPF object file: Relocation failed
make[1]: *** [../lib/common.mk:99: nat64_kern.skel.h] Error 255
make: *** [Makefile:37: nat64-bpf] Error 2

And second make yield following error:

$ make

lib

  util
make[2]: Nothing to be done for 'all'.

encap-forward
make[1]: Nothing to be done for 'all'.

ktrace-CO-RE
make[1]: Nothing to be done for 'all'.

lsm-nobpf
make[1]: Nothing to be done for 'all'.

nat64-bpf
    CC       nat64
nat64.c: In function ‘main’:
nat64.c:381:15: warning: implicit declaration of function ‘nat64_kern__open’ [-Wimplicit-function-declaration]
  381 |         obj = nat64_kern__open();
      |               ^~~~~~~~~~~~~~~~
nat64.c:381:13: warning: assignment to ‘struct nat64_kern *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
  381 |         obj = nat64_kern__open();
      |             ^
nat64.c:391:12: error: invalid use of undefined type ‘struct nat64_kern’
  391 |         obj->bss->config = cfg.c;
      |            ^~
nat64.c:392:37: error: invalid use of undefined type ‘struct nat64_kern’
  392 |         bpf_map__set_max_entries(obj->maps.v6_state_map, num_addr);
      |                                     ^~
nat64.c:393:37: error: invalid use of undefined type ‘struct nat64_kern’
  393 |         bpf_map__set_max_entries(obj->maps.v4_reversemap, num_addr);
      |                                     ^~
nat64.c:394:37: error: invalid use of undefined type ‘struct nat64_kern’
  394 |         bpf_map__set_max_entries(obj->maps.reclaimed_addrs, num_addr);
      |                                     ^~
nat64.c:396:15: warning: implicit declaration of function ‘nat64_kern__load’ [-Wimplicit-function-declaration]
  396 |         err = nat64_kern__load(obj);
      |               ^~~~~~~~~~~~~~~~
nat64.c:408:58: error: invalid use of undefined type ‘struct nat64_kern’
  408 |                 err = bpf_map_update_elem(bpf_map__fd(obj->maps.allowed_v6_src),
      |                                                          ^~
nat64.c:417:53: error: invalid use of undefined type ‘struct nat64_kern’
  417 |         attach_ingress.prog_fd = bpf_program__fd(obj->progs.nat64_ingress);
      |                                                     ^~
nat64.c:424:52: error: invalid use of undefined type ‘struct nat64_kern’
  424 |         attach_egress.prog_fd = bpf_program__fd(obj->progs.nat64_egress);
      |                                                    ^~
nat64.c:460:9: warning: implicit declaration of function ‘nat64_kern__destroy’ [-Wimplicit-function-declaration]
  460 |         nat64_kern__destroy(obj);
      |         ^~~~~~~~~~~~~~~~~~~
make[1]: *** [../lib/common.mk:83: nat64] Error 1
make: *** [Makefile:37: nat64-bpf] Error 2

Copy link
Member

@tohojo tohojo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bunch of minor-ish things, mostly on the userspace code.

Also, please add the netfilter-bpf directory to the toplevel Makefile.

struct bpf_dynptr ptr;
struct iphdr *p, iph = {};
struct ipv4_lpm_key key;
__u32 *pvalue;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please follow the kernel networking style of declaring variables in reverse x-mas tree order

__u32 *pvalue;

if (skb->len <= 20 || bpf_dynptr_from_skb(skb, 0, &ptr))
return NF_ACCEPT;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, let's have an empty line after this early return as well as the other ones.

int err;
struct bpf_object *obj;
struct bpf_program *prog;
union bpf_attr attr = { };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverse x-mas tree here as well, please.

{
struct ipv4_lpm_key key;
__u32 value = 0;
__u8 *p = (__u8 *) &key.data;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this casting to an array of u8, just do a htonl(0xc0a80b6b) with a comment explaining what it means.

Also, drop the block and just move the variables to the top of the function definition.

}
/* attach to netfilter output handler */
attr.link_create.netfilter.hooknum = NF_INET_LOCAL_OUT;
err = sys_bpf(BPF_LINK_CREATE, &attr, sizeof(attr));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the map be populated before attaching the program?

if (libbpf_get_error(obj)) {
printf("fail to open bpf file\n");
return 1;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From here on down, there should be proper error handling on the error paths. I.e., close the bpf object on exit, close the link fd, etc

printf("fail to find bpf program\n");
return 1;
}
bpf_program__set_type(prog, BPF_PROG_TYPE_NETFILTER);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed, the bpf code already has a SEC("netfilter") declaration?

printf("loading BPF object file failed\n");
return 1;
}
map_fd = bpf_object__find_map_fd_by_name(obj, "ipv4_lpm_map");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should use the high-level libbpf API; i.e. bpf_object__find_map_by_name() and bpf_map__update_elem().

return 1;
}
/* attach to netfilter forward handler */
prog_fd = bpf_program__fd(prog);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, this should use bpf_program__attach_netfilter()

static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr, unsigned int size)
{
return syscall(__NR_bpf, cmd, attr, size);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not needed when using the highlevel libbpf APIs, see below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants