iree-org / iree-org/iree

[iree.build] torch export cleanup hang with multiprocessing ‘spawn’ context

Open
#20,102 2 comments 0 reactions 2 assignees Claimed by @stellaraccident View on GitHub
bug 🐞
Dominant language
C++
Stars
3.9k
Forks
1k
Avg merge
4d 16h
Merged PRs (30d)
47

Description

### What happened?

Related bandaid PR: https://github.com/iree-org/iree/pull/20101

Using iree.build to export torch modules (with turbine_generate build actions), we encounter an issue where weakref cleanup (?) seemingly can’t figure out what to do with the contents of the torch no.module exported, resulting in a hang during python exit.

This seemingly only happens when two conditions are satisfied:
1. We use multiprocessing ‘spawn’ context
2. The exported torch module creates tensors from class attributes or bare lists (observed with integer list) without registering them as buffers.

### Steps to reproduce your issue

coming soon…

### What component(s) does this issue relate to?

Compiler

### Version information

https://github.com/iree-org/iree/commit/028b90fd69a87b538ae8bf5e193785ffd2048f1e
(Top of main at post time)

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.