kernelci / kernelci/kernelci-core

kbuild: skip building kselftest when not required by jobfilter (speed up bisection)

Open
#3,127 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
120
Forks
107
Avg merge
1d 2h
Merged PRs (30d)
21

Description

## Goal

To speed up bisection, do not build kselftest when it isn't needed by the requested jobs (kselftest builds are expensive and dominate bisection turnaround).

This was originally attempted in #2729, but that approach needs more design work before it can land — see below. Converting to an issue to track the proper solution.

## Current attempt (#2729)

In `kernelci/kbuild.py`, when a `jobfilter` is present, kselftest is disabled unless `-kselftest` literally appears in the filter:

```python
if node['jobfilter'] and self._kfselftest is True:
kselftest_name = node['name'] + "-kselftest"
if kselftest_name not in node['jobfilter']:
self._kfselftest = False
```

## Why this isn't enough

1. **Downstream tests are invisible to the build.** `kbuild.py` runs in the build container and only receives `node` + `params` + a flat `jobfilter` list of strings. It has no knowledge of which *tests* consume kselftest artifacts. So a jobfilter containing a kselftest test (but not the kselftest build name) would silently produce a build without kselftest, and the test would fail mysteriously. The dependency knowledge lives in kernelci-pipeline:
- test → needs-kselftest is identifiable from `config/jobs.yaml` (`test_method: kselftest`, `kcidb_test_suite: kselftest.*`)
- build → test link is the `event` field in `config/scheduler.yaml`

2. **Possible naming bug.** Dedicated kselftest build variants are already named `kbuild-...-kselftest` (e.g. `config/jobs.yaml`, `scheduler.yaml`). For such a node, `node['name'] + "-kselftest"` becomes `kbuild-...-kselftest-kselftest`, which never matches the filter — so kselftest would be disabled on the very build meant to produce it (this is exactly the bisection-of-a-kselftest-test path). Needs verification against real `node['name']` values.

## Proposed direction

Move the decision to kernelci-pipeline, which already has `self._configs` loaded and builds the jobfilter in `get_jobfilter()` (`src/lava_callback.py`):

- When any requested job in the jobfilter resolves to a kselftest test (`test_method == 'kselftest'`), auto-enable kselftest on the corresponding build (e.g. set a `kselftest: enable` param on the build node, or ensure the kselftest-enabled build name is in the filter).
- Otherwise, leave kselftest off so bisection builds stay fast.

The one non-trivial piece is reverse-resolving the scheduler graph (test → triggering build-node event → build job), since the build job doesn't inherently know its downstream tests. This is an in-memory config walk, not new infrastructure.

Closes #2729 (superseded by this issue).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with get_jobfilter() in src/lava_callback.py, then inspect config/jobs.yaml and config/scheduler.yaml to trace kselftest tests through event links to their build jobs. Compare this with the current filtering logic in kernelci/kbuild.py. Done means requested kselftest jobs retain the required artifacts while unrelated bisection builds skip kselftest.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, ci-cd
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.