AcademySoftwareFoundation / AcademySoftwareFoundation/OpenImageIO

[BUG] I think ImageBufAlgo::make_texture() creates an unevictable tile in imagecache

Open
#3,334 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.4k
Forks
698
Avg merge
3d 9h
Merged PRs (30d)
48

Description

**Describe the bug**
We use `ImageBufAlgo::make_texture(.., string_view filename, ..)` to create a .tx file on the fly from a rather large PNG (30k x 15k 16-bit). When the conversion is done texture accesses to the newly created .tx are very slow. I observe that "Peak cache memory" is 2.6 GB after the call to make_texture and never goes down after that which leads to lots of redundant reads.

When we read the PNG to an ImageBuf and use the ImageBuf version of make_texture this problem does not occur.

**To Reproduce**
I have attached a sample program which creates the PNG, performs make_texture on it and then samples the .tx file from 8 threads. When setting `maketx_mode` to USE_IMAGEBUF the texture sampling takes 1s on my computer. With USE_IMAGEFILE the sampling takes 60s.

With USE_IMAGEFILE:
```
redundant reads: 529924 tiles, 12.1 GB
Peak cache memory : 2.6 GB
Image file statistics:
opens tiles MB read --redundant-- I/O time res File
1 1 1 2688.6 5.9s 30000x15662x3.u16 input.png UNTILED
2 1 537653 12601.2 (529924 12420.1) 8m 11.3s 30000x15662x3.u16 txfile.tx MIP-COUNT[339938,197715,0,0,0,0,0,0,0,0,0,0,0,0,0]

Tot: 2 537654 15289.8 (529924 12420.1) 8m 17.2s
1 not tiled, 1 not MIP-mapped
Broken or invalid files: 0
```

With USE_IMAGEBUF:
```
redundant reads: 0 tiles, 0 B
Peak cache memory : 181.3 MB
Image file statistics:
opens tiles MB read --redundant-- I/O time res File
1 1 0 0.0 0.0s 30000x15662x3.u16 input.png UNTILED
2 1 7729 181.1 1.0s 30000x15662x3.u16 txfile.tx MIP-COUNT[4951,2778,0,0,0,0,0,0,0,0,0,0,0,0,0]

Tot: 2 7729 181.1 ( 0 0.0) 1.0s
1 not tiled, 1 not MIP-mapped
Broken or invalid files: 0
```

**Expected behavior**
I expect the png data to vanish from the cache.

**Platform information:**
- OIIO branch/version: 2.3.12.0
- OS: Windows 10
- C++ compiler: Visual Studio 2019 16.11.8
- Any non-default build flags when you build OIIO: No
- libpng 1.6.37
- libtiff v4.3.0
- 9700K, 16 GB RAM
[oiio-repro.txt](https://github.com/OpenImageIO/oiio/files/8048098/oiio-repro.txt)

Contributor guide

Open the contributing guide

Research direction

Start with the attached oiio-repro.txt sample and compare ImageBufAlgo::make_texture using USE_IMAGEFILE versus USE_IMAGEBUF, focusing on imagecache behavior. Reproduce the cache-memory and redundant-read measurements on the reported large PNG, then verify that the generated texture no longer leaves the PNG data resident and that sampling avoids the excessive redundant reads.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.