indygreg / indygreg/python-zstandard
SystemError / wrong behavior: 15+ sites conflate error-return sentinels with data (`PyObject_IsTrue`, `PyObject_IsInstance`, `PyLong_AsSsize_t`)
- Dominant language
- C
- Stars
- 642
- Forks
- 116
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 5
Description
## Summary
Three Python/C-API functions — `PyObject_IsTrue`, `PyObject_IsInstance`, and `PyLong_AsSsize_t` — return `-1` to signal an error *and* `-1` can be a legitimate data value. zstandard does not distinguish the two at ~15 sites. Observable consequences:
- `PyObject_IsTrue`: `-1` is stored as an `int` field value (`closefd = -1`, `read_across_frames = -1`, ...) or used in `!r` / `r == 0` comparisons, returning success with a pending exception → `SystemError: returned a result with an exception set` (fatal abort on debug builds).
- `PyObject_IsInstance`: `!r` maps `-1` to `0` (false), silently replacing the caller's exception with `TypeError("dict_data must be zstd.ZstdCompressionDict")`.
- `PyLong_AsSsize_t`: legitimate `-1` is conflated with the error sentinel without checking `PyErr_Occurred()`; `from_level(dict_size=-1)` takes the error branch without setting an exception → `SystemError: returned NULL without setting an exception` (fatal abort on debug builds).
## Impact
- **Severity:** Fatal `SystemError` on debug builds; silent wrong behavior / clobbered exceptions on release.
- **Reachability:** Standard Python — pass a `__bool__` that raises, a negative `dict_size`, or a `__instancecheck__` that raises. No unusual setup.
- **Version:** 0.25.0 (commit `7a77a75`).
- **Platform:** Confirmed Linux x86_64 / CPython 3.14 debug; bug is platform-independent.
## Reproducers
### `PyObject_IsTrue` — 11 sites, one trigger pattern
```python
import zstandard, io
class BadBool:
def __bool__(self):
raise ValueError("bad bool")
comp = zstandard.ZstdCompressor()
comp.stream_writer(io.BytesIO(), closefd=BadBool())
# Fatal Python error: returned a result with an exception set
# SystemError: returned a result with an exception set
```
All 11 confirmed sites:
```python
comp.stream_reader(source, closefd=BadBool())
comp.stream_writer(dest, closefd=BadBool())
comp.stream_writer(dest, write_return_read=BadBool())
decomp.stream_reader(source, closefd=BadBool())
decomp.stream_reader(source, read_across_frames=BadBool())
decomp.stream_writer(dest, closefd=BadBool())
decomp.stream_writer(dest, write_return_read=BadBool())
decomp.decompress(data, allow_extra_data=BadBool())
zstandard.ZstdCompressor(write_checksum=BadBool())
zstandard.ZstdCompressor(write_content_size=BadBool())
zstandard.ZstdCompressor(write_dict_id=BadBool())
```
### `PyLong_AsSsize_t` — `from_level(dict_size=-1)` aborts
```python
import zstandard
zstandard.ZstdCompressionParameters.from_level(3, dict_size=-1)
# Fatal Python error: returned NULL without setting an exception
# SystemError: returned NULL without setting an exception
```
### `PyObject_IsInstance` — 3 sites
Pattern: `!PyObject_IsInstance(obj, ...)` in validation for the `dict_data` kwarg on compressor / decompressor init. Confirmed by static review — reproducing requires a non-cooperative `__instancecheck__`, which is less idiomatic to construct than `BadBool`. Impact is exception clobbering rather than a crash: caller's exception from `__instancecheck__` is replaced with `TypeError("dict_data must be zstd.ZstdCompressionDict")`.
## Root cause
All three APIs return `-1` on error and `-1` can be a valid value. The correct patterns:
- `PyObject_IsTrue` / `PyObject_IsInstance`: check `< 0` *before* using the return as a boolean.
- `PyLong_AsSsize_t`: when the result is `-1`, check `PyErr_Occurred()` to distinguish a legitimate `-1` from an error.
zstandard stores the `PyObject_IsTrue` return directly into an `int` field (`result->closefd = PyObject_IsTrue(obj);`), uses `!PyObject_IsInstance(...)` without a prior `< 0` check, and takes a `goto cleanup` on `PyLong_AsSsize_t == -1` without consulting `PyErr_Occurred`.
## Affected sites
### `PyObject_IsTrue` — 11
| File | Line(s) | Kwarg / field |
|------|---------|---------------|
| `c-ext/compressor.c` | 487 | `write_checksum`, `write_content_size`, `write_dict_id` (init) |
| `c-ext/compressor.c` | 752-754 | `closefd`, `write_return_read` on `stream_reader` / `stream_writer` |
| `c-ext/decompressor.c` | 381 | `closefd` on `stream_reader` |
| `c-ext/decompressor.c` | 431 | `read_across_frames` |
| `c-ext/decompressor.c` | 562-563 | `closefd`, `write_return_read` on `stream_writer` |
| `c-ext/decompressor.c` | 614-615 | `allow_extra_data` on `decompress()` |
### `PyObject_IsInstance` — 3
| File | Line | Context |
|------|------|---------|
| `c-ext/compressor.c` | 128 | `dict_data` type check in `ZstdCompressor_init` |
| `c-ext/compressor.c` | 140 | `dict_data` type check (2nd occurrence) |
| `c-ext/decompressor.c` | 78 | `dict_data` type check in `Decompressor_init` |
### `PyLong_AsSsize_t` — reported site
| File | Line | Context |
|------|------|---------|
| `c-ext/compressionparams.c` | 271 | `dict_size` parameter to `from_level()` |
The `PyLong_AsSsize_t` calls in `copy_stream` (`c-ext/compressor.c` around 411; `c-ext/decompressor.c` around 247) interact with an OOM cleanup path to produce a separate `Py_DECREF(NULL)` — noted here for completeness; details in the full report linked below.
## Suggested fix
Three mechanical patterns; one per API.
**`PyObject_IsTrue`:**
```c
int r = PyObject_IsTrue(obj);
if (r < 0) goto except;
result->closefd = r ? 1 : 0;
```
**`PyObject_IsInstance`:**
```c
int r = PyObject_IsInstance(obj, (PyObject *)&ZstdCompressionDict_Type);
if (r < 0) goto except;
if (!r) {
PyErr_SetString(PyExc_TypeError, "dict_data must be zstd.ZstdCompressionDict");
goto except;
}
```
**`PyLong_AsSsize_t`:**
```c
Py_ssize_t dict_size_val = PyLong_AsSsize_t(dict_size);
if (dict_size_val == -1 && PyErr_Occurred()) goto cleanup;
if (dict_size_val < 0) {
PyErr_SetString(PyExc_ValueError, "dict_size must be non-negative");
goto cleanup;
}
```
All three can land in a single PR. If you prefer finer-grained review, happy to split into 3 PRs (one per API).
## Methodology
Found via [cext-review-toolkit](https://github.com/devdanzin/cext-review-toolkit) (Tree-sitter-based static analysis with structured naive/informed review passes). 11 of 11 `BadBool()` sites produce a `SystemError` fatal abort on CPython 3.14.3 debug. `from_level(dict_size=-1)` also verified on the same build. `PyObject_IsInstance` sites confirmed via static review. Happy to open a PR.
*Discovery, root-cause analysis, and issue drafting were performed by [Claude Code](https://www.anthropic.com/claude-code) and reviewed by a human before filing.*
## Full report
Complete multi-agent analysis (48 FIX findings across 13 categories, plus a reproducer appendix): https://gist.github.com/devdanzin/b86039ac097141579590c1a0f3a43605
Contributor guide
Research direction
Start with the affected entry points in c-ext/compressor.c, c-ext/decompressor.c, and c-ext/compressionparams.c, then run the BadBool and dict_size=-1 reproducers from the issue. Review all listed PyObject_IsTrue, PyObject_IsInstance, and PyLong_AsSsize_t sites. Done means the affected calls distinguish errors from valid -1 values and the reproducers no longer produce SystemError or clobbered exceptions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100