indygreg / indygreg/python-zstandard
Footgun with the `compressobj()` API
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 642
- Forks
- 116
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 5
Description
The fact that you can create multiple compressobj from a single ZstdCompressor allows for foot-guns such as:
a = zstandard.ZstdCompressor()
b = a.compressobj()
c = b.compress(b"prefix")
d = a.compressobj()
e = d.compress(b"foo")
e += d.flush(zstandard.FLUSH_BLOCK)
c = b.compress(b"foo")
c += b.flush(zstandard.FLUSH_BLOCK)
assert(c != e) # Assertion fails!
The API should protect users from interleaving usages of two compressobj. This could be accomplished via a counter that is atomically incremented in the ZstdCompressor. The compressobj would know what count it was created on and throw an error if compress() or flush() are called after the counter in the parent ZstdCompressor is incremented.
Thanks @thatch for identifying this issue and proposing the fix.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the interleaved compressobj() calls in the issue, then inspect the ZstdCompressor and compressobj implementation. Done means later operations on an older compressobj are rejected after another object is created, with the reported assertion no longer failing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100