is there more stable way to use for 5090 in window?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 37.8k
- Forks
- 4.3k
- Avg merge
- 7m
- Merged PRs (30d)
- 1
Description
hi, I got it worked for 5090 in window but it still appears problems after a few generations. I reseach it and found out it is a known limitation of torch.compile + CUDA Graphs when the inference runs inside Gradio's background worker thread (anyio.to_thread.run_sync).
The first 1–2 generations sometimes succeed (fresh compile), but subsequent ones hit the thread-local storage assertion. is there anyway I can get this worked with optimize=True (torch.compile + CUDA Graphs) at maximum speed on RTX 5090 in window?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure on an RTX 5090 on Windows with optimize=True, using the Gradio background worker path described in the report. Investigate the interaction between torch.compile, CUDA Graphs, and anyio.to_thread.run_sync across repeated generations; done means multiple generations remain stable at the requested optimization level.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100