bigscience-workshop / bigscience-workshop/petals

rpc_push may fail due to exceeding max message size

Open
#372 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
650
PR merge metrics
No merged PRs in 30d

Description

This happens if inference step gives you(the handler) more than 4mb of tensors per step -- and you(the handler) proceed to rpc_push it to the next guy using rpc_push. We may choose not to forward such packages, but at least we should not try - and fail - to forward them every time.

This issue can also be resolved from converting rpc_push into a streaming rpc

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.