AcademySoftwareFoundation / AcademySoftwareFoundation/OpenImageIO

[BUG] python3.10 spec.get_string_attribute() throws a runtime exception with non-utf-8 string headers

Open
#3,856 1 comment 0 reactions 0 assignees View on GitHub
python
Dominant language
C++
Stars
2.4k
Forks
698
Avg merge
3d 9h
Merged PRs (30d)
48

Description

**Describe the bug**

When looping over all key/value pairs in an EXR header using the python bindings to OpenImageIO, a field containing 0xFF values causes an awkward `RuntimeError: Could not allocate string object!`. This is very commonly seen in DI when the upstream writing tool doesn't want to set a value and instead just fills it with some default of `0xFF`.

**To Reproduce**
With the following header (as shown with iinfo -a -v):
```sh
AudioFramerate: 24
AudioInfo: "������������������������������������������������������������������������@"
AudioRunningLTC: "MOS"
```
Where the `�` character is `0xFF`.

Attempting to iterate over headers like so:

```python
>>> import OpenImageIO as oiio
>>> src = oiio.ImageBuf('./my_file.1000.exr')
>>> spec = src.spec()
>>> for a in spec.extra_attribs:
... print(f'{a.name} ({a.type}): {a.value} ({type(a.value)})')
```

Causes the following exception:

```python
AudioFileName (string): ()
AudioFramerate (float): 24.0 ()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 2, in
RuntimeError: Could not allocate string object!
```

**Expected behavior**
It appears that strings are assumed to be in utf-8 unicode, which causes an error trying to decode non-unicode data into a python string.

The workaround is to not access `a.value` and instead ask for the attribute directly like so:

```python
>>> try:
... x = spec.get_string_attribute('AudioInfo')
... except RuntimeError:
... pass
```

However, I was hoping to have a more graceful exception of this error. For example, I cannot access the raw bytes of this value to determine if the offending field is actually usable. Perhaps adding an `a.raw_value` could return the information in the field as a bytestring so the above try/except has an opportunity to do something else special with the value. Since the accessing of an attribute here is forcing the coercion to unicode, I cannot get to the original data using python in any way.

**Platform information:**
- OIIO branch/version: `openimageio-2.4.5.0-r0`
- OS: `Alpine Linux 3.17 under Docker`
- C++ compiler:
- Any non-default build flags when you build OIIO:

Contributor guide

Open the contributing guide

Research direction

Start with the Python binding entry points for spec.extra_attribs, a.value, and spec.get_string_attribute(), using the supplied non-UTF-8 EXR header to reproduce the failure. Determine how the original attribute bytes can be accessed without forced UTF-8 decoding; done means the offending header can be handled gracefully and its raw value remains available to Python.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.