randomparity / randomparity/kdive

External-build validation rejects a zstd-compressed x86_64 bzImage

Open
#2,476 1 comment 0 reactions 0 assignees View on GitHub
area:build-install effort:S priority:P1 status:ready type:bug
Dominant language
Python
Stars
0
Forks
0
Avg merge
1h 26m
Merged PRs (30d)
311

Description

## Problem

`runs.complete_build` rejects a real x86_64 kernel bundle whose `boot/vmlinuz` is a
**zstd-compressed** bzImage, with:

```
build_failure: boot/vmlinuz does not contain a supported gzip, bzip2, xz, or zstd ELF
kernel payload
```

`CONFIG_KERNEL_ZSTD=y` is the default on Fedora and Arch, so this is the ordinary case for an
agent uploading a kernel built from a distro config — not an exotic one. The same tree,
reconfigured to `CONFIG_KERNEL_GZIP=y` and relinked, validates.

The message names zstd as supported, which makes the rejection read as a malformed payload
rather than a gap in the decoder.

## Evidence

Source verified at `4883cfdce`. Kernel: linux 7.2.6, x86_64, built from a Fedora host config.

The payload is decodable. The zstd frame inside the rejected bzImage yields an ELF:

```python
src.seek(21196)
with zstd.ZstdFile(src, mode="rb") as s:
s.read(1 << 16) # -> 65536 bytes, magic b'\x7fELF'
```

Same tree, same `.config`, only `CONFIG_KERNEL_ZSTD` → `CONFIG_KERNEL_GZIP`, relinked:

```
zstd bzImage (16,953,856 B): _decoded_kernel -> build_failure
gzip bzImage (19,825,152 B): _decoded_kernel -> DECODED OK, magic b'\x7fELF'
```

### Root cause: gzip is member-bounded, the other three read to EOF

`src/kdive/build_artifacts/validation.py` `_decoded_kernel` treats one candidate differently
from the rest:

- gzip → `_copy_gzip_member_bounded`, which stops at the end of the gzip **member**;
- bzip2, xz, zstd → `_copy_kernel_bounded(source, ...)`, which reads the decompressor to **EOF**.

A bzImage is a compressed payload followed by further data. `ZstdFile` in multi-frame mode
finishes the kernel's frame, attempts to parse the trailing bytes as the next frame, and raises
`ZstdError: Unknown frame descriptor` — *after* having already produced ~67 MB of correct ELF.
The loop catches that, truncates `decoded`, and continues, so the one correct candidate is
discarded and the run ends on the generic "does not contain a supported … payload" message.

Traced candidate by candidate (offsets from the rejected bzImage):

```
cand 1 gzip off=450338 : zlib.error unknown header flags set
cand 2 gzip off=9388479: zlib.error incorrect header check
cand 3 zstd off=21196 : ZstdError Unknown frame descriptor <- correct payload, discarded
(budget 2147483648 -> 2080374784,
i.e. ~67 MB of ELF decoded first)
cand 4..12 zstd : Unknown frame descriptor / EOFError
```

### Second defect in the same function: the offset generator shares the stream it scans

`_magic_offsets(boot, magic)` is a generator reading from `boot`, while the loop body
`boot.seek(offset)`s and lets the opener consume from the same handle. The offsets it yields
therefore depend on what the previous candidate read. Probing the same bzImage twice produced
different lists:

```
gzip: [450338, 9838817] vs [450338, 9388479]
zstd: [21196, 9890700, 16811361, …] vs [21196, 4207253, 4207512, …]
```

That makes candidate enumeration order-dependent and is its own correctness problem,
independent of the EOF issue above.

### Why no test caught it

Every bundle fixture in the repository is gzip:

- `tests/mcp/complete_build_support.py:65` — `boot.extend(gzip.compress(elf))`
- `tests/providers/local_libvirt/test_validate_external_artifacts.py:80,434,446` —
`gzip.compress(kernel)`

`rg -n "zstd" tests/providers/local_libvirt/test_validate_external_artifacts.py` returns
nothing: the x86 zstd, xz and bzip2 decode paths have no coverage at all.

Fixture line 80 is `bytes(header) + gzip.compress(kernel) + compressed_trailer` — it explicitly
models a trailer, which is exactly the shape the gzip path handles and the other three do not.
So the one algorithm that is bounded correctly is also the only one exercised, and against the
only input shape that would have exposed the others.

## Expected

A kernel bundle whose `boot/vmlinuz` is compressed with any algorithm the error message names —
gzip, bzip2, xz, or zstd — validates, provided the payload decodes to an ELF. Trailing bytes
after the compressed payload do not cause a rejection, for any of the four.

Candidate enumeration does not depend on how much a previous candidate consumed.

## Proposed approach

- Bound the bzip2/xz/zstd decoders to a single frame or stream, the way
`_copy_gzip_member_bounded` already bounds the gzip member, so trailing bytes end the copy
instead of raising.
- Give `_magic_offsets` an independent handle, or materialize the offsets before the loop
consumes the stream.
- Add one validator fixture per compression algorithm, each carrying a trailer, so the three
untested paths gain the coverage gzip has.

Do not relax the decode budget or the candidate cap; neither is implicated. The budget was
untouched at rejection time (`2080374784` of `2147483648` remaining) and 12 candidates were
tried against a cap of 64.

## Provenance

Found while measuring external-build finalization for #2318 (part of #2314), on the first real
kernel bundle put through `runs.complete_build`. #2318 works around it by measuring a
`CONFIG_KERNEL_GZIP=y` build, recorded as such in its proof record; fixing the decoder is
outside that issue's frozen scope, and #2314 excludes changes to validation behavior.

Contributor guide

Open the contributing guide

Research direction

Start in src/kdive/build_artifacts/validation.py, reading _decoded_kernel, _copy_gzip_member_bounded, _copy_kernel_bounded, and _magic_offsets. Then inspect tests/providers/local_libvirt/test_validate_external_artifacts.py and tests/mcp/complete_build_support.py, and run the validator tests. Done means gzip, bzip2, xz, and zstd payloads with trailers decode to ELF, while candidate enumeration is independent of prior stream consumption.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.