iovisor / iovisor/bcc

tools/offcputime.py crashes when it runs out space for new stack traces

Open
#526 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
22.7k
Forks
4.1k
Avg merge
10d 4h
Merged PRs (30d)
3

Description

`BPF_STACK_TRACE(_name, _size)` will allocate space for _size stack traces on each CPU (see [here](https://github.com/torvalds/linux/blob/master/kernel/bpf/stackmap.c#L30-L50)).
If we've already used all of this space, subsequent calls to `bpf_get_stackid()` will return -ENOMEM (see [here](https://github.com/torvalds/linux/blob/master/kernel/bpf/stackmap.c#L173-L176)).
This causes our BPF bytecode to store this value in `key_t.stack_id` and subsequently causes our python application to crash due to a `KeyError` when invoking `stack_traces.walk(k.stack_id)`.

How do we feel about...
1. `key_t.stack_id` having a value of -ENOMEM
2. `stack_traces.walk(k.stack_id)` allowing the `KeyError` to bubble up
3. handling `KeyError` explicitly at all call sites of `stack_traces.walk(k.stack_id)`

1bd74c18664716c2cbf4922661e0f8cf60d504b5 shows an implementation of (3) above. If this is the preferred way to handle this then we should also handle this edge case in the other tools that use `bpf_get_stackid()`.

cc @brendangregg

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.