bigscience-workshop / bigscience-workshop/petals
Facing Issue while running 176b bloom-petals model
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 650
- PR merge metrics
- No merged PRs in 30d
Description
While running original bloom-petals model , I was successfully able to load all the 70 blocks , and also able to make path(span sequence) , but at the time of inference facing issue on client side i.e banning peer temporarily , caught exception while running inference, timeout Error().
Also on the 1st server side i could see the rpc_inference alloc and then getting error on server side i.e [hivemind.moe.server.runtime.run:104] Caught 1678, attempting to recover


Kindly help me resolve this issue as soon as possible
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing inference with the original BLOOM-Petals model and compare the client timeout and temporary peer ban with the server's hivemind.moe.server.runtime.run error during rpc_inference. Done means inference completes without the client timeout or peer ban and the server no longer reports the caught exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100