Woker erorr under python3.8: AttributeError: module 'asyncio' has no attribute 'to_thread'
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
```
/workspace# python3 -m fastchat.serve.model_worker
...
"POST /worker_generate HTTP/1.1" 500 Internal Server Error
2023-12-13 04:01:05 | ERROR | stderr | ERROR: Exception in ASGI application
....
2023-12-13 04:01:07 | ERROR | stderr | return await dependant.call(**values)
2023-12-13 04:01:07 | ERROR | stderr | File "/opt/conda/lib/python3.8/site-packages/fastchat/serve/base_model_worker.py", line 206, in api_generate
2023-12-13 04:01:07 | ERROR | stderr | output = await asyncio.to_thread(worker.generate_gate, params)
2023-12-13 04:01:07 | ERROR | stderr | AttributeError: module 'asyncio' has no attribute 'to_thread'
```
looks like `asyncio.to_thread` is only available in python 3.9+
https://github.com/lvxuan263/FastChat/blob/6ac7d76885cae2d06d76bfe7fd8ec5aac6602e6f/fastchat/serve/base_model_worker.py#L206
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with fastchat/serve/base_model_worker.py at the api_generate entry point and reproduce the failure using `python3 -m fastchat.serve.model_worker` under Python 3.8. Check the thread-dispatch call's supported Python versions and verify that a worker generation request no longer raises AttributeError on Python 3.8.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 50/100