bigscience-workshop / bigscience-workshop/petals

Facing Issue while running 176b bloom-petals model

Open
#317 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
650
PR merge metrics
No merged PRs in 30d

Description

While running original bloom-petals model , I was successfully able to load all the 70 blocks , and also able to make path(span sequence) , but at the time of inference facing issue on client side i.e banning peer temporarily , caught exception while running inference, timeout Error().
Also on the 1st server side i could see the rpc_inference alloc and then getting error on server side i.e [hivemind.moe.server.runtime.run:104] Caught 1678, attempting to recover
![Screenshot (36)](https://github.com/bigscience-workshop/petals/assets/59815066/6cacb7bd-1678-4cc8-9e49-bd270b842000)
![Screenshot (37)](https://github.com/bigscience-workshop/petals/assets/59815066/71ff56e5-396b-479d-8df4-27871b01db46)
Kindly help me resolve this issue as soon as possible

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing inference with the original BLOOM-Petals model and compare the client timeout and temporary peer ban with the server's hivemind.moe.server.runtime.run error during rpc_inference. Done means inference completes without the client timeout or peer ban and the server no longer reports the caught exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
distributed-systems, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.