AcademySoftwareFoundation / AcademySoftwareFoundation/OpenImageIO
[BUG] python3.10 spec.get_string_attribute() throws a runtime exception with non-utf-8 string headers
- Dominant language
- C++
- Stars
- 2.4k
- Forks
- 698
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 48
Description
**Describe the bug**
When looping over all key/value pairs in an EXR header using the python bindings to OpenImageIO, a field containing 0xFF values causes an awkward `RuntimeError: Could not allocate string object!`. This is very commonly seen in DI when the upstream writing tool doesn't want to set a value and instead just fills it with some default of `0xFF`.
**To Reproduce**
With the following header (as shown with iinfo -a -v):
```sh
AudioFramerate: 24
AudioInfo: "������������������������������������������������������������������������@"
AudioRunningLTC: "MOS"
```
Where the `�` character is `0xFF`.
Attempting to iterate over headers like so:
```python
>>> import OpenImageIO as oiio
>>> src = oiio.ImageBuf('./my_file.1000.exr')
>>> spec = src.spec()
>>> for a in spec.extra_attribs:
... print(f'{a.name} ({a.type}): {a.value} ({type(a.value)})')
```
Causes the following exception:
```python
AudioFileName (string): ()
AudioFramerate (float): 24.0 ()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "", line 2, in
RuntimeError: Could not allocate string object!
```
**Expected behavior**
It appears that strings are assumed to be in utf-8 unicode, which causes an error trying to decode non-unicode data into a python string.
The workaround is to not access `a.value` and instead ask for the attribute directly like so:
```python
>>> try:
... x = spec.get_string_attribute('AudioInfo')
... except RuntimeError:
... pass
```
However, I was hoping to have a more graceful exception of this error. For example, I cannot access the raw bytes of this value to determine if the offending field is actually usable. Perhaps adding an `a.raw_value` could return the information in the field as a bytestring so the above try/except has an opportunity to do something else special with the value. Since the accessing of an attribute here is forcing the coercion to unicode, I cannot get to the original data using python in any way.
**Platform information:**
- OIIO branch/version: `openimageio-2.4.5.0-r0`
- OS: `Alpine Linux 3.17 under Docker`
- C++ compiler:
- Any non-default build flags when you build OIIO:
Contributor guide
Research direction
Start with the Python binding entry points for spec.extra_attribs, a.value, and spec.get_string_attribute(), using the supplied non-UTF-8 EXR header to reproduce the failure. Determine how the original attribute bytes can be accessed without forced UTF-8 decoding; done means the offending header can be handled gracefully and its raw value remains available to Python.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100