bigscience-workshop / bigscience-workshop/petals
Ensure that SequenceManager is thread-safe
Open
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 650
- PR merge metrics
- No merged PRs in 30d
Description
We'd like to make a Petals client (mainly, SequenceManager) thread-safe. This is useful for web apps like [chat.petals.ml](https://github.com/borzunov/chat.petals.ml), as well as anything else sending requests to the model concurrently.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.