binary-husky / binary-husky/gpt_academic
[Bug]: 从魔塔社区下载的chatglm模型启动报错
- Dominant language
- Python
- Stars
- 71.4k
- Forks
- 8.3k
- PR merge metrics
- No merged PRs in 30d
Description
### Installation Method | 安装方法与平台
Pip Install (I used latest requirements.txt)
### Version | 版本
Latest | 最新版
### OS | 操作系统
Linux
### Describe the bug | 简述
# 背景
docker-compose启动的服务,因为网络问题无法从huggingface.co下载模型,是从魔塔社区下载的模型
modelscope download --model ZhipuAI/glm-4-9b-chat --local_dir ./models/THUDM/glm-4-9b-chat
# 1. docker-compose文件如下:
```
version: '3'
services:
gpt_academic_full_capability:
image: ghcr.io/binary-husky/gpt_academic_with_all_capacity:master
environment:
CHATGLM_LOCAL_MODEL_PATH: 'THUDM/glm-4-9b-chat'
LOCAL_MODEL_DEVICE: 'cuda'
LOCAL_MODEL_QUANT: 'FP16'
API_KEY: 'sk-xx'
DASHSCOPE_API_KEY: 'sk-xx'
USE_PROXY: 'False'
LLM_MODEL: 'gpt-4o'
AVAIL_LLM_MODELS: '["gpt-3.5-turbo", "gpt-4o", "qwen-max-latest", "chatglm4","deepseek-r1","deepseek-v3","chatglm3-6b"]'
ENABLE_AUDIO: 'False'
DEFAULT_WORKER_NUM: '20'
WEB_PORT: '18080'
ADD_WAIFU: 'False'
ALIYUN_APPKEY: 'RxPlZrM88DnAFkZK'
THEME: 'Chuanhu-Small-and-Beautiful'
LOCAL_MODEL_DEVICE: 'cuda'
API_URL_REDIRECT: >
{
"https://api.openai.com/v1/chat/completions": "https://api.gptsapi.net/v1/chat/completions",
"https://api.openai.com/v1/completions": "https://api.gptsapi.net/v1/completions",
"https://api.openai.com/v1/embeddings": "https://api.gptsapi.net/v1/embeddings"
}
CHATGLM_LOCAL_MODEL_PATH: '/models/THUDM/glm-4-9b-chat'
volumes:
- /root/models/THUDM/glm-4-9b-chat:/models/THUDM/glm-4-9b-chat:ro
runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
# network_mode: "host"
ports:
- "18080:18080"
command: >
bash -c "python3 -u main.py"
```
# 2. 后端服务日志:
```
14:02 | ..v_variable:33 | [ENV_VAR] 尝试加载CHATGLM_LOCAL_MODEL_PATH,默认值:THUDM/glm-4-9b-chat --> 修正值:/models/THUDM/glm-4-9b-chat
14:02 | ..v_variable:60 | [ENV_VAR] 成功读取环境变量CHATGLM_LOCAL_MODEL_PATH
14:02 | ..v_variable:33 | [ENV_VAR] 尝试加载LOCAL_MODEL_DEVICE,默认值:cpu --> 修正值:cuda
14:02 | ..v_variable:60 | [ENV_VAR] 成功读取环境变量LOCAL_MODEL_DEVICE
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 0%| | 0/10 [00:00
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at request_llms/bridge_chatglm4.py line 63 and follow the model-loading path into request_llms/local_llm_class.py. Reproduce the failure with the provided docker-compose configuration, then inspect the generation call and the referenced modeling_chatglm.py cache-handling path. Done means the downloaded GLM-4 model starts and generates a response without the unpacking error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker-compose, python, pytorch
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100