abetlen / abetlen/llama-cpp-python

Docker: Missing library transformers when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided

未关闭
#1,355 1 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
10.6k
派生
1.4k
PR 合并指标
PR 指标待抓取

描述

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior
Running Functionary 2.4 Small in docker should work when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided

`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`

# Current Behavior

`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`
yields the following error when HF_PRETRAINED_MODEL_NAME_OR_PATH is set

```
File "/app/llama_cpp/llama_tokenizer.py", line 96, in from_pretrained
raise ImportError(
ImportError: The `transformers` library is required to use the `HFTokenizer`.You can install it with `pip install transformers`.
```

# Environment and Context

- Docker version: 23.0.4 - Community
- OS: Linux XYZ 5.15.0-70-generic 77-Ubuntu SMP Tue Mar 21 14:02:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

# Failure Information (for bugs)

Running Functionary 2.4 Small in docker does not work when HF_PRETRAINED_MODEL_NAME_OR_PATH is provided.

```
File "/app/llama_cpp/llama_tokenizer.py", line 96, in from_pretrained
raise ImportError(
ImportError: The `transformers` library is required to use the `HFTokenizer`.You can install it with `pip install transformers`.
```

# Steps to Reproduce

Run
`docker run --rm -it -p 8888:8000 -v /home/xstrama/llm/functionary-small-v2.4-GGUF:/models -e MODEL=/models/functionary-small-v2.4.Q4_0.gguf -e CHAT_FORMAT=functionary-v2 -e HF_PRETRAINED_MODEL_NAME_OR_PATH=/models ghcr.io/abetlen/llama-cpp-python:latest
`

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。