apache / apache/iceberg-python

Cython Avro decoder reads past the buffer end on malformed input

Aperta
#3,952 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug
Lingua principale
Python
Stelle
1.1k
Fork
581
Merge medio
1g 17h
PR unite (30g)
78

Descrizione

Two places in `pyiceberg/avro/decoder_fast.pyx` advance the read pointer using a value taken from the stream, without bounding it against `self._end`.

**1. `read_bytes` does not validate the decoded length**

```python
cpdef inline bytes read_bytes(self):
cdef uint64_t length;
if self._current >= self._end: # only confirms 1 byte is available
raise EOFError(f"EOF: read 1 bytes")

decode_zigzag_ints(&self._current, 1, &length)

if length <= 0:
return b""
cdef const unsigned char *r = self._current
self._current += length # not checked against self._end
return r[0:length]
```

The guard confirms one byte is available before decoding the length, but the decoded `length` is then used to slice and to advance `_current` with no check that `_current + length <= _end`. A length field larger than the remaining buffer reads beyond it.

**2. `decode_zigzag_ints` has no end pointer to bound against**

```c
void decode_zigzag_ints(const unsigned char **buffer, const uint64_t count, uint64_t *result);
```

The signature takes a buffer and a count but no end, so the varint walk cannot stop at the buffer boundary — a run of bytes with the continuation bit set keeps advancing. It is called from five sites in the decoder (lines 93, 101, 111, 124, 176), including from `read_bytes` above.

Both are reachable from a malformed or hostile Avro manifest.

---
Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Direzione di ricerca

Start in pyiceberg/avro/decoder_fast.pyx, reading read_bytes and decode_zigzag_ints plus the five call sites listed in the issue. Exercise malformed or truncated Avro input through the decoder. Done means malformed input cannot advance past the buffer end and valid decoding remains covered by the existing decoder behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
security
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Attiva
Chiarezza
Specificata chiaramente
Idoneità per principianti
72/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.