bigscience-workshop / bigscience-workshop/petals
rpc_push may fail due to exceeding max message size
Open
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 650
- PR merge metrics
- No merged PRs in 30d
Description
This happens if inference step gives you(the handler) more than 4mb of tensors per step -- and you(the handler) proceed to rpc_push it to the next guy using rpc_push. We may choose not to forward such packages, but at least we should not try - and fail - to forward them every time.
This issue can also be resolved from converting rpc_push into a streaming rpc
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.