indygreg / indygreg/python-zstandard
`ZstdCompressionWriter()` constructs successfully but leads to `segmentation fault`
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 642
- Forks
- 116
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 5
Description
I've been fuzzing Python C extension modules for a small research project.
I found a sanitizer issue which is a null-pointer dereference in ZstdCompressionWriter_memory_size.
I reproduced it with the binary wheel from a plain pip install zstandard.
The process terminates with SIGSEGV there as well.
I'm not sure whether zero-argument construction of this type is considered supported (calling ZstdCompressionWriter() with no arguments),
but since it currently terminates the interpreter rather than raising a Python exception, I thought it was worth reporting.
Versions
zstandard 0.25.0, cext backend, CPython 3.12, Linux x86_64.
Reproducer
import zstandard
w = zstandard.ZstdCompressionWriter() # succeeds
w.memory_size() # SIGSEGV
Five calls across the two writer types behave the same way, each from a fresh direct construction:
| class | methods that segfault |
|---|---|
ZstdCompressionWriter |
memory_size(), close(), flush() |
ZstdDecompressionWriter |
memory_size(), flush() |
Sanitizer build
Same call, zstandard built with -fsanitize=address,undefined:
c-ext/compressionwriter.c:64:65: runtime error:
member access within null pointer of type 'ZstdCompressor'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
c-ext/compressionwriter.c:64:65
AddressSanitizer: SEGV on unknown address 0x000000000020
#0 ZstdCompressionWriter_memory_size c-ext/compressionwriter.c:64:65
ZstdDecompressionWriter.memory_size() reports the equivalent at
c-ext/decompressionwriter.c:50:67.
Expected behavior
I think that direct construction should be rejected with a Python exception, as it is by the CFFI backend.
Under PYTHON_ZSTANDARD_IMPORT_POLICY=cffi the same construction is refused outright:
TypeError: ZstdCompressionWriter.__init__() missing 5 required positional
arguments: 'compressor', 'writer', 'source_size', 'write_size',
and 'write_return_read'
Actual behavior
The C extension permits construction with no arguments, leaving internal fields NULL.
Several methods dereference those fields and terminate the process with SIGSEGV.
Although these objects are normally obtained through stream_writer(), I wonder whether the C extension should reject zero-argument construction, as the CFFI backend already does, rather than produce an uninitialized object.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Run the Python reproducer first, then inspect c-ext/compressionwriter.c:64 and c-ext/decompressionwriter.c:50 alongside the writer construction paths. Compare the C extension behavior with the CFFI backend for zero-argument construction. Done means direct construction no longer leaves NULL internal fields reachable and raises a Python exception instead of segfaulting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100