Cyan4973 / Cyan4973/FiniteStateEntropy
FSE_compress returns 0 even when maxDstSize is still equal to or slightly larger than final compression size
- Dominant language
- C
- Stars
- 1.5k
- Forks
- 165
- PR merge metrics
- No merged PRs in 30d
Description
While adding more unit tests, I found a corner case when trying to compress a document with a `maxDstSize` which is exactly the expected compressed size (found by a previous compression attempt): in this test, `FSE_compress(..)` returns 0 (uncompressible data) while previously it was able to compress the same input (given a larger destination buffer).
I'm doing a two-step process:
1) Allocate a buffer large enough (using `FSE_compressBound`), and compress the source by calling `FSE_compress(..., maxDstSize: FSE_compressBound(ORIGINAL_SIZE))`, and measure its final compressed size `COMPRESSED_SIZE`.
2) Repeat the process on the same buffer and payload, but this time calling `FSE_compress(..., maxDstSize: COMPRESSED_SIZE)`. I get a result code of __0__ which means that data is uncompressible.
I tried probing the minimum size that will allow compressing the buffer (which is known to be compressible), and each time I need to call `FSE_compress(..)` with at least `COMPRESSED_SIZE + 8`. At first I thought it could be a pointer alignment issue, but it is always + 8 bytes whatever compressed size is (by changing the source a bit).
In my test, raw original size is __4,288__ bytes, `FSE_compressBound` return __4,833__ bytes, and the compressed size is __2,821__ bytes. I need to pass at least 2,823+8 = __2,829__ bytes for `FSE_compress` to succeed (return value > 0).
Is this expected behavior? I'm not sure if the "+8 rule" is true, or if this is random chance with the inputs I'm passing in.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.