iree-org / iree-org/iree

Exceptions with non-string arguments and pickling in multi-process context

Open
#11,655 2 comments 0 reactions 1 assignee Claimed by @jpienaar View on GitHub
bug 🐞
Dominant language
C++
Stars
3.9k
Forks
1k
Avg merge
4d 16h
Merged PRs (30d)
47

Description

### What happened?

I have this code that compiles an invalid program in subprocess. The raised exception `iree.compiler.tools.CompilerToolError` can not be correctly unpickled.
I am not sure if this is a bug in Python 3.10.9. It sure looks strange.

### Steps to reproduce your issue

```python
import iree.compiler
from concurrent.futures import ProcessPoolExecutor

def compile():
iree.compiler.compile_str(
"I'm a little teapot but not a valid program",
target_backends=iree.compiler.tools.DEFAULT_TESTING_BACKENDS)

with ProcessPoolExecutor() as executor:
executor.submit(compile).result()
```
The output is
```
concurrent.futures.process._RemoteTraceback:
'''
Traceback (most recent call last):
File "/home/petkantchin/ws/shark/shark_venv/lib/python3.10/site-packages/iree/compiler/tools/binaries.py", line 72, in __init__
errs = process.stderr.decode("utf-8")
AttributeError: 'str' object has no attribute 'stderr'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/lib/python3.10/concurrent/futures/process.py", line 387, in wait_result_broken_or_wakeup
result_item = result_reader.recv()
File "/usr/lib/python3.10/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
File "/home/petkantchin/ws/shark/shark_venv/lib/python3.10/site-packages/iree/compiler/tools/binaries.py", line 74, in __init__
errs = str(process.stderr) # Decode error or other: best we can do.
AttributeError: 'str' object has no attribute 'stderr'
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/petkantchin/ws/shark/experiment/compile_in_subprocess.py", line 10, in
executor.submit(compile).result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
```
I was expecting to get an exception telling me that the MLIR program I am trying to compile is invalid.
During unpickling it tries to recreate the exception with the message string, but `CompilerToolError.__init__` accepts `subprocess.CompletedProcess`.

### What component(s) does this issue relate to?

Python

### Version information

Git commit e5d71f5a0b2635cfaf5153364cc699434ac92eb7.

### Additional context

Here is another self-contained example that recreates the issue.
```python
from concurrent.futures import ProcessPoolExecutor

class CustomException(Exception):
def __init__(self, x):
super().__init__(x.some_member)

class SomeClass:
def __init__(self, some_member):
self.some_member = some_member

def raise_exception():
x = SomeClass('foo')
raise CustomException(x)

with ProcessPoolExecutor() as executor:
executor.submit(raise_exception).result()
```
```
concurrent.futures.process._RemoteTraceback:
'''
Traceback (most recent call last):
File "/usr/lib/python3.10/concurrent/futures/process.py", line 387, in wait_result_broken_or_wakeup
result_item = result_reader.recv()
File "/usr/lib/python3.10/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
File "/home/petkantchin/ws/shark/experiment/exception_in_subprocess.py", line 5, in __init__
super().__init__(x.some_member)
AttributeError: 'str' object has no attribute 'some_member'
'''

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/petkantchin/ws/shark/experiment/exception_in_subprocess.py", line 16, in
executor.submit(raise_exception).result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.