AcademySoftwareFoundation / AcademySoftwareFoundation/OpenImageIO
[BUG] Slow read performance for multichannel EXRs in oiiotool
- Dominant language
- C++
- Stars
- 2.4k
- Forks
- 698
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 46
Description
**Describe the bug**
I'm getting very slow read performance on multi-channel EXRs where I just need to parse the RGBA channel and write out a `.png` file from it.
From 19s on my own machine (which is crazy for a 55 MB exr?) to 5s on the fastest machine I have around.
On my own slow machine, taking the same EXR (on no cold/hot cache) through Fusion to a PNG takes 1.3s
The 19 seconds (but 1.3s through fusion) is on:
- Windows 10
- 2x Intel(R) Xeon(R) CPU E5-2680 0 @2.70Ghz
- Dual Xeon in HP workstation
- 128 GB RAM
- 4 GB GPU
The 5 seconds is on:
- Windows 11
- AMD Ryzen 9 9900X 12-Core Processor (4.40 Ghz)
- 128 GB RAM
- 12 GB GPU
**OpenImageIO version and dependencies**
See test runs [here](https://gist.github.com/BigRoy/327f3c978b32182348c54ed9f30a5cbe).
```
OIIO 3.1.1.0dev | unknown arch?
Build compiler: MSVS 1943 | C++17/199711
HW features enabled at build: sse2
No CUDA support (disabled / unavailable at build time)
Dependencies: DCMTK NONE, expat 2.6.3, FFmpeg NONE, fmt 10.2.1, Freetype 2.13.2, GIF NONE, Imath 3.2.0, JPEG 80,
JXL NONE, libdeflate 1.23, Libheif NONE, libjpeg-turbo 3.0.4, LibRaw NONE, libuhdr 1.2.0, minizip-ng 4.0.7,
OpenColorIO 2.4.1, OpenCV NONE, OpenEXR 3.4.0, OpenJPEG NONE, OpenVDB NONE, PNG 1.6.47, Ptex NONE, Ptex NONE,
pystring 1.1.4, Robinmap 1.4.0, TBB 2022.2.0, TIFF 4.6.0, WebP 1.4.0, yaml-cpp 0.8.0, ZLIB 1.3.1, ZLIB 1.3.1
```
**To Reproduce**
Steps to reproduce the behavior:
1. `oiiotool -v -i:ch=R,G,B,A input.exr -o output.png`
I'm explicitly filtering `-i:ch=R,G,B,A` to avoid reading the rest of the file. Also adding `:now=1` doesn't make much of a difference.
Other options, like also passing `-ch` flag are included in the linked test runs with their timings. Using `i:ch=...` gave the fastest results, but still at 19s seems awful.
Here's the file I'm testing with: https://we.tl/t-Fjm4XVFw8z
It'll only remain available for about four weeks.
**Evidence**
See test runs [here](https://gist.github.com/BigRoy/327f3c978b32182348c54ed9f30a5cbe).
### Additional Context
This also came up on ASWF slack with a lengthy thread [here](https://academysoftwarefdn.slack.com/archives/C05782U3806/p1744995471959379)
Some notable comments from there:
#### Build options didn't make much of a difference?
> So, I've been testing some build variants.
> When enabling more hardware features like SSSE3, SSE4.1 SSE4.2, AVX etc. makes it so that the executable doesn't work on my machine - likely due to some AVX (it does work on newer machines but there was no real measurable performance difference on this simple read/write test cases) so I opted going back to just the sse2 default but adding OpenTBB which does run. I've added the performance on my slow machine to the gist (see link below)
> There is no noticable speed difference. It remains around 18/19 seconds on my machine
> For what it's worth, using `--frames 1001-1006` and `--parallel-frames` also takes about 20 seconds on my machine (so almost no time increase).
> Using `--frames 1001-1030` with `--parallel-frames` takes about 53 seconds on my machine.
>
> So I can get good gains with allowing it to read many files. However, it's still the majority the read performance taking a long time it seems.
Contributor guide
Research direction
Reproduce the issue with the supplied EXR and `oiiotool -v -i:ch=R,G,B,A input.exr -o output.png`, then compare timings with the linked test runs. Investigate the multichannel EXR read path and the effect of channel filtering; done means the RGBA-only conversion no longer exhibits the reported excessive read time and the relevant timings are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100