apache / apache/iceberg-python

Cython Avro decoder reads past the buffer end on malformed input

Offen
#3,952 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Python
Sterne
1.1k
Forks
581
Ø Merge
1 T. 17 Std.
Gemergte PRs (30 T.)
78

Beschreibung

Two places in `pyiceberg/avro/decoder_fast.pyx` advance the read pointer using a value taken from the stream, without bounding it against `self._end`.

**1. `read_bytes` does not validate the decoded length**

```python
cpdef inline bytes read_bytes(self):
cdef uint64_t length;
if self._current >= self._end: # only confirms 1 byte is available
raise EOFError(f"EOF: read 1 bytes")

decode_zigzag_ints(&self._current, 1, &length)

if length <= 0:
return b""
cdef const unsigned char *r = self._current
self._current += length # not checked against self._end
return r[0:length]
```

The guard confirms one byte is available before decoding the length, but the decoded `length` is then used to slice and to advance `_current` with no check that `_current + length <= _end`. A length field larger than the remaining buffer reads beyond it.

**2. `decode_zigzag_ints` has no end pointer to bound against**

```c
void decode_zigzag_ints(const unsigned char **buffer, const uint64_t count, uint64_t *result);
```

The signature takes a buffer and a count but no end, so the varint walk cannot stop at the buffer boundary — a run of bytes with the continuation bit set keeps advancing. It is called from five sites in the decoder (lines 93, 101, 111, 124, 176), including from `read_bytes` above.

Both are reachable from a malformed or hostile Avro manifest.

---
Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Rechercherichtung

Start in pyiceberg/avro/decoder_fast.pyx, reading read_bytes and decode_zigzag_ints plus the five call sites listed in the issue. Exercise malformed or truncated Avro input through the decoder. Done means malformed input cannot advance past the buffer end and valid decoding remains covered by the existing decoder behavior.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
security
Issue-Typ
Bug
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Aktiv
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
72/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.