apache / apache/iceberg-python
Cython Avro decoder reads past the buffer end on malformed input
- Langage dominant
- Python
- Étoiles
- 1.1k
- Forks
- 581
- Merge moyen
- 1 j 17 h
- PR mergées (30 j)
- 78
Description
Two places in `pyiceberg/avro/decoder_fast.pyx` advance the read pointer using a value taken from the stream, without bounding it against `self._end`.
**1. `read_bytes` does not validate the decoded length**
```python
cpdef inline bytes read_bytes(self):
cdef uint64_t length;
if self._current >= self._end: # only confirms 1 byte is available
raise EOFError(f"EOF: read 1 bytes")
decode_zigzag_ints(&self._current, 1, &length)
if length <= 0:
return b""
cdef const unsigned char *r = self._current
self._current += length # not checked against self._end
return r[0:length]
```
The guard confirms one byte is available before decoding the length, but the decoded `length` is then used to slice and to advance `_current` with no check that `_current + length <= _end`. A length field larger than the remaining buffer reads beyond it.
**2. `decode_zigzag_ints` has no end pointer to bound against**
```c
void decode_zigzag_ints(const unsigned char **buffer, const uint64_t count, uint64_t *result);
```
The signature takes a buffer and a count but no end, so the varint walk cannot stop at the buffer boundary — a run of bytes with the continuation bit set keeps advancing. It is called from five sites in the decoder (lines 93, 101, 111, 124, 176), including from `read_bytes` above.
Both are reachable from a malformed or hostile Avro manifest.
---
Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Piste de recherche
Commencez dans pyiceberg/avro/decoder_fast.pyx en lisant read_bytes et decode_zigzag_ints, ainsi que les cinq sites d’appel listés dans l’issue. Testez des entrées Avro malformées ou tronquées via le décodeur. Le travail est terminé lorsque les entrées malformées ne peuvent pas avancer au-delà de la fin du tampon et que le décodage valide reste couvert par le comportement existant du décodeur.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python
- Domaine
- security
- Type d'issue
- Bug
- Difficulté
- 3/5
- Temps estimé
- 1-2 jours
- Activité
- Active
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 72/100