AcademySoftwareFoundation / AcademySoftwareFoundation/openexr
Investigate additional compression: Zstd, LERC, webp, Rice, ZFP
- Dominant language
- C
- Stars
- 1.8k
- Forks
- 700
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 29
Description
libtiff 4.0 added support for zstd compression (lossless) and webp (lossy), and the newly-released 4.3 adds LERC compression (lossy, error-bounded). I've also heard that the astronomical imaging community have found "Rice" compression to be very good. Their data (FITS files) being floating point where lossless is important may indicate that their experience is also applicable to our typical data. ZFP is another format used for scientific data.
We should investigate whether any of these would be beneficial to add to OpenEXR, either for compression ratio or compress/decompress performance reasons. Most of the comparisons available (such as the [libtiff 4.0.10 release notes with zlib and webp benchmarks](http://libtiff.maptools.org/v4.0.10.html) and also [LERC benchmarks](https://github.com/Esri/lerc/blob/master/doc/LercBenchmarks_Feb_2016.pdf)) are for integer images; we should not assume their performance on typical OpenEXR half images without thorough testing.
References:
* https://github.com/facebook/zstd
* https://github.com/Esri/lerc
* https://awesomeopensource.com/project/Esri/lerc?categoryPage=41
* https://developers.google.com/speed/webp/
* https://www.webmproject.org/code/#libwebp-webp-image-library
* https://github.com/LLNL/zfp
This is a placeholder so we remember this topic and so that newcomers see that it's a potentially valuable project to take on. I'm not volunteering to implement it myself. Also note that each of these compression methods can be investigated or implemented separately and in isolation; it is not necessary for one person to do it all.
Contributor guide
Research direction
Review the linked libtiff, LERC, WebP, Zstd, and ZFP references, then benchmark the candidate methods on typical OpenEXR half images rather than relying on integer-image results. Done means documenting compression-ratio and compression/decompression-performance findings and recommending whether a specific method merits separate implementation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100