kernelci / kernelci/kernelci-core
kbuild: skip building kselftest when not required by jobfilter (speed up bisection)
- Dominant language
- Python
- Stars
- 120
- Forks
- 107
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 21
Description
## Goal
To speed up bisection, do not build kselftest when it isn't needed by the requested jobs (kselftest builds are expensive and dominate bisection turnaround).
This was originally attempted in #2729, but that approach needs more design work before it can land — see below. Converting to an issue to track the proper solution.
## Current attempt (#2729)
In `kernelci/kbuild.py`, when a `jobfilter` is present, kselftest is disabled unless `-kselftest` literally appears in the filter:
```python
if node['jobfilter'] and self._kfselftest is True:
kselftest_name = node['name'] + "-kselftest"
if kselftest_name not in node['jobfilter']:
self._kfselftest = False
```
## Why this isn't enough
1. **Downstream tests are invisible to the build.** `kbuild.py` runs in the build container and only receives `node` + `params` + a flat `jobfilter` list of strings. It has no knowledge of which *tests* consume kselftest artifacts. So a jobfilter containing a kselftest test (but not the kselftest build name) would silently produce a build without kselftest, and the test would fail mysteriously. The dependency knowledge lives in kernelci-pipeline:
- test → needs-kselftest is identifiable from `config/jobs.yaml` (`test_method: kselftest`, `kcidb_test_suite: kselftest.*`)
- build → test link is the `event` field in `config/scheduler.yaml`
2. **Possible naming bug.** Dedicated kselftest build variants are already named `kbuild-...-kselftest` (e.g. `config/jobs.yaml`, `scheduler.yaml`). For such a node, `node['name'] + "-kselftest"` becomes `kbuild-...-kselftest-kselftest`, which never matches the filter — so kselftest would be disabled on the very build meant to produce it (this is exactly the bisection-of-a-kselftest-test path). Needs verification against real `node['name']` values.
## Proposed direction
Move the decision to kernelci-pipeline, which already has `self._configs` loaded and builds the jobfilter in `get_jobfilter()` (`src/lava_callback.py`):
- When any requested job in the jobfilter resolves to a kselftest test (`test_method == 'kselftest'`), auto-enable kselftest on the corresponding build (e.g. set a `kselftest: enable` param on the build node, or ensure the kselftest-enabled build name is in the filter).
- Otherwise, leave kselftest off so bisection builds stay fast.
The one non-trivial piece is reverse-resolving the scheduler graph (test → triggering build-node event → build job), since the build job doesn't inherently know its downstream tests. This is an in-memory config walk, not new infrastructure.
Closes #2729 (superseded by this issue).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with get_jobfilter() in src/lava_callback.py, then inspect config/jobs.yaml and config/scheduler.yaml to trace kselftest tests through event links to their build jobs. Compare this with the current filtering logic in kernelci/kbuild.py. Done means requested kselftest jobs retain the required artifacts while unrelated bisection builds skip kselftest.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, ci-cd
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100