abetlen / abetlen/llama-cpp-python

Docker: Missing library transformers when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided

Open
#1,355 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior
Running Functionary 2.4 Small in docker should work when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided

`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`

# Current Behavior

`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`
yields the following error when HF_PRETRAINED_MODEL_NAME_OR_PATH is set

```
File "/app/llama_cpp/llama_tokenizer.py", line 96, in from_pretrained
raise ImportError(
ImportError: The `transformers` library is required to use the `HFTokenizer`.You can install it with `pip install transformers`.
```

# Environment and Context

- Docker version: 23.0.4 - Community
- OS: Linux XYZ 5.15.0-70-generic 77-Ubuntu SMP Tue Mar 21 14:02:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

# Failure Information (for bugs)

Running Functionary 2.4 Small in docker does not work when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided.

```
File "/app/llama_cpp/llama_tokenizer.py", line 96, in from_pretrained
raise ImportError(
ImportError: The `transformers` library is required to use the `HFTokenizer`.You can install it with `pip install transformers`.
```

# Steps to Reproduce

Run
`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.