iovisor / iovisor/bcc

libbpf-tools: some bio tools fails to run on kernel v6.1

Open
#4,638 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
22.7k
Forks
4.1k
Avg merge
10d 4h
Merged PRs (30d)
3

Description

I'm testing bio tools on riscv cpu with kenel 6.1, and get failures
```
~ # uname -a
Linux formosa 6.1.31+ #8 SMP Thu Jun 15 09:10:38 CST 2023 riscv64 GNU/Linux
~ # biosnoop
libbpf: prog 'blk_account_io_start': failed to attach: ERROR: strerror_r(-524)=22
failed to attach blk_account_io_start: Unknown error 524
~ # biostacks
libbpf: prog 'blk_account_io_done': failed to find kernel BTF type ID of 'blk_account_io_done': -3
libbpf: prog 'blk_account_io_done': failed to prepare load attributes: -3
libbpf: prog 'blk_account_io_done': failed to load: -3
libbpf: failed to load object 'biostacks_bpf'
libbpf: failed to load BPF skeleton 'biostacks_bpf': -3
failed to load BPF object: -3
~ # biotop
trace_kprobe: Could not probe notrace function blk_account_io_start
libbpf: prog 'blk_account_io_start': failed to create kprobe 'blk_account_io_start+0x0' perf event: Invalid argument
failed to load attach blk_account_io_start
```
only blk_account_io_start can be found in `/proc/kallsyms`, and they are both not existed in `tracing/available_filter_functions`
```
~ # cat /proc/kallsyms | grep blk_account_io
ffffffff801eed42 t blk_account_io_merge_bio
ffffffff801f2a9e t blk_account_io_completion
ffffffff801f2c74 t blk_account_io_start
~ # cat /sys/kernel/tracing/available_filter_functions | grep blk_account_io
blk_account_io_merge_bio
blk_account_io_completion
```
These 2 funstions are all inline functions, see
https://elixir.bootlin.com/linux/v6.1.31/source/block/blk-mq.c#L1001
https://elixir.bootlin.com/linux/v6.1.31/source/block/blk-mq.c#L971
inline is defined in `include/linux/compiler_types.h`
```
/*
* Prefer gnu_inline, so that extern inline functions do not emit an
* externally visible function. This makes extern inline behave as per gnu89
* semantics rather than c99. This prevents multiple symbol definition errors
* of extern inline functions at link time.
* A lot of inline functions can cause havoc with function tracing.
*/
#define inline inline __gnu_inline __inline_maybe_unused notrace
```
they also become notrace functions, is there any way to fix it?

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the failures in biosnoop, biostacks, and biotop on the reported riscv64 Linux 6.1 environment. Start with the bio-tool attachment definitions and the kernel references in block/blk-mq.c and include/linux/compiler_types.h, then compare /proc/kallsyms with tracing/available_filter_functions. Done means the unsupported blk_account_io functions are handled appropriately and the bio tools no longer fail unexpectedly.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
observability, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.