abetlen / abetlen/llama-cpp-python
Docker: Missing library transformers when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided
- Vorherrschende Sprache
- Python
- Sterne
- 10.6k
- Forks
- 1.4k
- PR-Merge-Kennzahlen
- PR-Kennzahlen ausstehend
Beschreibung
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.
# Expected Behavior
Running Functionary 2.4 Small in docker should work when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided
`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`
# Current Behavior
`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`
yields the following error when HF_PRETRAINED_MODEL_NAME_OR_PATH is set
```
File "/app/llama_cpp/llama_tokenizer.py", line 96, in from_pretrained
raise ImportError(
ImportError: The `transformers` library is required to use the `HFTokenizer`.You can install it with `pip install transformers`.
```
# Environment and Context
- Docker version: 23.0.4 - Community
- OS: Linux XYZ 5.15.0-70-generic 77-Ubuntu SMP Tue Mar 21 14:02:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
# Failure Information (for bugs)
Running Functionary 2.4 Small in docker does not work when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided.
```
File "/app/llama_cpp/llama_tokenizer.py", line 96, in from_pretrained
raise ImportError(
ImportError: The `transformers` library is required to use the `HFTokenizer`.You can install it with `pip install transformers`.
```
# Steps to Reproduce
Run
`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.