bigscience-workshop / bigscience-workshop/petals
Question about overlapped serving blocks
Open
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 650
- PR merge metrics
- No merged PRs in 30d
Description
Consider a case that a pre-trained model is only hosted on three servers: the first one hosts blocks 1-4, the second hosts blocks 2-64, and the third hosts blocks 32-128.
I want to know if the overlapping of serving blocks will affect a client's fine-tuning or inference.
Thanks.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.