AcademySoftwareFoundation / AcademySoftwareFoundation/openexr

Investigate additional compression: Zstd, LERC, webp, Rice, ZFP

Open
#1,002 13 comments 1 reaction 0 assignees View on GitHub
needs attention
Dominant language
C
Stars
1.8k
Forks
700
Avg merge
3d 12h
Merged PRs (30d)
29

Description

libtiff 4.0 added support for zstd compression (lossless) and webp (lossy), and the newly-released 4.3 adds LERC compression (lossy, error-bounded). I've also heard that the astronomical imaging community have found "Rice" compression to be very good. Their data (FITS files) being floating point where lossless is important may indicate that their experience is also applicable to our typical data. ZFP is another format used for scientific data.

We should investigate whether any of these would be beneficial to add to OpenEXR, either for compression ratio or compress/decompress performance reasons. Most of the comparisons available (such as the [libtiff 4.0.10 release notes with zlib and webp benchmarks](http://libtiff.maptools.org/v4.0.10.html) and also [LERC benchmarks](https://github.com/Esri/lerc/blob/master/doc/LercBenchmarks_Feb_2016.pdf)) are for integer images; we should not assume their performance on typical OpenEXR half images without thorough testing.

References:
* https://github.com/facebook/zstd
* https://github.com/Esri/lerc
* https://awesomeopensource.com/project/Esri/lerc?categoryPage=41
* https://developers.google.com/speed/webp/
* https://www.webmproject.org/code/#libwebp-webp-image-library
* https://github.com/LLNL/zfp

This is a placeholder so we remember this topic and so that newcomers see that it's a potentially valuable project to take on. I'm not volunteering to implement it myself. Also note that each of these compression methods can be investigated or implemented separately and in isolation; it is not necessary for one person to do it all.

Contributor guide

Open the contributing guide

Research direction

Review the linked libtiff, LERC, WebP, Zstd, and ZFP references, then benchmark the candidate methods on typical OpenEXR half images rather than relying on integer-image results. Done means documenting compression-ratio and compression/decompression-performance findings and recommending whether a specific method merits separate implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.