iovisor / iovisor/bcc

Batch map lookups may fail with ENOSPC

Open
#5,273 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
22.7k
Forks
4.1k
Avg merge
5d 13h
Merged PRs (30d)
3

Description

`items_lookup_batch` and `items_lookup_and_delete_batch` may fail with ENOSPC, when the map is close to its `max_entries`.

Batch lookups are inherently non-atomic, as the map may change between reads, making it possible to go over `max_entries`, as long as BPF programs keep updating the map as it is read.

Limiting [`count`](https://github.com/iovisor/bcc/blob/29403046f7a5d86600bced4962c1ecfd994b7f87/src/python/bcc/table.py#L596) appears to be insufficient, as internally iteration is over [buckets](https://github.com/torvalds/linux/blob/master/kernel/bpf/hashtab.c#L1751) and if the requested `count` is smaller than the size of the bucket ENOSPC is [returned](https://github.com/torvalds/linux/blob/master/kernel/bpf/hashtab.c#L1774). This happens when [`_items_lookup_and_optionally_delete_batch`](https://github.com/iovisor/bcc/blob/29403046f7a5d86600bced4962c1ecfd994b7f87/src/python/bcc/table.py#L563) has accumulated items close to `max_entries` and has only few elements worth of allocated space left, which turns out to be less than the size of the next bucket in the iteration.

Perhaps, ENOSPC can be simply swallowed, although in hypothetical severe cases that would mean last buckets may be repeatedly left behind.

A more mathematically correct way would be to always allocate more and continue fetching elements until the map has been traversed in full. Theoretical maximum number of elements would be `max_entries * n_buckets`, but that is almost impossible in practice, so not a big concern, perhaps.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in src/python/bcc/table.py at _items_lookup_and_optionally_delete_batch, items_lookup_batch, and items_lookup_and_delete_batch, then read the linked kernel hashtab.c bucket iteration and ENOSPC handling. Determine how batch traversal should behave when the map changes during reads, and verify that the chosen behavior avoids failed lookups or repeatedly missed final buckets.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux, python
Domain
operating-systems, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.