indygreg / indygreg/python-zstandard

`ZstdCompressionWriter()` constructs successfully but leads to `segmentation fault`

Open
#331 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
642
Forks
116
Avg merge
1d 14h
Merged PRs (30d)
5

Description

I've been fuzzing Python C extension modules for a small research project.
I found a sanitizer issue which is a null-pointer dereference in ZstdCompressionWriter_memory_size.

I reproduced it with the binary wheel from a plain pip install zstandard.
The process terminates with SIGSEGV there as well.

I'm not sure whether zero-argument construction of this type is considered supported (calling ZstdCompressionWriter() with no arguments),
but since it currently terminates the interpreter rather than raising a Python exception, I thought it was worth reporting.

Versions

zstandard 0.25.0, cext backend, CPython 3.12, Linux x86_64.

Reproducer
import zstandard

w = zstandard.ZstdCompressionWriter()   # succeeds
w.memory_size()                         # SIGSEGV

Five calls across the two writer types behave the same way, each from a fresh direct construction:

class methods that segfault
ZstdCompressionWriter memory_size(), close(), flush()
ZstdDecompressionWriter memory_size(), flush()
Sanitizer build

Same call, zstandard built with -fsanitize=address,undefined:

c-ext/compressionwriter.c:64:65: runtime error:
    member access within null pointer of type 'ZstdCompressor'
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior
    c-ext/compressionwriter.c:64:65

AddressSanitizer: SEGV on unknown address 0x000000000020
    #0 ZstdCompressionWriter_memory_size  c-ext/compressionwriter.c:64:65

ZstdDecompressionWriter.memory_size() reports the equivalent at
c-ext/decompressionwriter.c:50:67.

Expected behavior

I think that direct construction should be rejected with a Python exception, as it is by the CFFI backend.

Under PYTHON_ZSTANDARD_IMPORT_POLICY=cffi the same construction is refused outright:

TypeError: ZstdCompressionWriter.__init__() missing 5 required positional
arguments: 'compressor', 'writer', 'source_size', 'write_size',
and 'write_return_read'
Actual behavior

The C extension permits construction with no arguments, leaving internal fields NULL.
Several methods dereference those fields and terminate the process with SIGSEGV.


Although these objects are normally obtained through stream_writer(), I wonder whether the C extension should reject zero-argument construction, as the CFFI backend already does, rather than produce an uninitialized object.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run the Python reproducer first, then inspect c-ext/compressionwriter.c:64 and c-ext/decompressionwriter.c:50 alongside the writer construction paths. Compare the C extension behavior with the CFFI backend for zero-argument construction. Done means direct construction no longer leaves NULL internal fields reachable and raises a Python exception instead of segfaulting.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.