bigscience-workshop / bigscience-workshop/petals

Error when trying to launch private swarm using locally stored model

Open
#549 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
650
PR merge metrics
No merged PRs in 30d

Description

Hi,

I am trying to launch a private swarm using a model I have downloaded to a local directory, which I think in theory should work fine. However, it presents an error when I try to run the command to start the first server.

This is what I'm getting.

>python -m petals.cli.run_server lzlv --new_swarm
>Dec 21 19:41:15.890 [INFO] Running Petals 2.3.0.dev1
>Dec 21 19:41:16.231 [INFO] Make sure you follow the LLaMA's terms of use: https://bit.ly/llama2-license for LLaMA 2, >https://bit.ly/llama-license for LLaMA 1
>Traceback (most recent call last):
> File "/opt/conda/lib/python3.10/runpy.py", line 196, in _run_module_as_main
> return _run_code(code, main_globals, None,
> File "/opt/conda/lib/python3.10/runpy.py", line 86, in _run_code
> exec(code, run_globals)
> File "/opt/conda/lib/python3.10/site-packages/petals/cli/run_server.py", line 235, in
> main()
> File "/opt/conda/lib/python3.10/site-packages/petals/cli/run_server.py", line 219, in main
> server = Server(
> File "/opt/conda/lib/python3.10/site-packages/petals/server/server.py", line 119, in __init__
> assert UID_DELIMITER not in dht_prefix and CHAIN_DELIMITER not in dht_prefix, (
>TypeError: argument of type 'NoneType' is not iterable

My setup using the same command works fine when I launch it using the huggingface path for the same model. The folder I reference on my machine contains the model downloaded right from huggingface including config.json.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.