binary-husky / binary-husky/gpt_academic

[Bug]: 从魔塔社区下载的chatglm模型启动报错

Open
#2,206 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
71.4k
Forks
8.3k
PR merge metrics
No merged PRs in 30d

Description

### Installation Method | 安装方法与平台

Pip Install (I used latest requirements.txt)

### Version | 版本

Latest | 最新版

### OS | 操作系统

Linux

### Describe the bug | 简述

# 背景
docker-compose启动的服务,因为网络问题无法从huggingface.co下载模型,是从魔塔社区下载的模型

modelscope download --model ZhipuAI/glm-4-9b-chat --local_dir ./models/THUDM/glm-4-9b-chat

# 1. docker-compose文件如下:
```
version: '3'
services:
gpt_academic_full_capability:
image: ghcr.io/binary-husky/gpt_academic_with_all_capacity:master
environment:
CHATGLM_LOCAL_MODEL_PATH: 'THUDM/glm-4-9b-chat'
LOCAL_MODEL_DEVICE: 'cuda'
LOCAL_MODEL_QUANT: 'FP16'
API_KEY: 'sk-xx'
DASHSCOPE_API_KEY: 'sk-xx'
USE_PROXY: 'False'
LLM_MODEL: 'gpt-4o'
AVAIL_LLM_MODELS: '["gpt-3.5-turbo", "gpt-4o", "qwen-max-latest", "chatglm4","deepseek-r1","deepseek-v3","chatglm3-6b"]'
ENABLE_AUDIO: 'False'
DEFAULT_WORKER_NUM: '20'
WEB_PORT: '18080'
ADD_WAIFU: 'False'
ALIYUN_APPKEY: 'RxPlZrM88DnAFkZK'
THEME: 'Chuanhu-Small-and-Beautiful'
LOCAL_MODEL_DEVICE: 'cuda'
API_URL_REDIRECT: >
{
"https://api.openai.com/v1/chat/completions": "https://api.gptsapi.net/v1/chat/completions",
"https://api.openai.com/v1/completions": "https://api.gptsapi.net/v1/completions",
"https://api.openai.com/v1/embeddings": "https://api.gptsapi.net/v1/embeddings"
}
CHATGLM_LOCAL_MODEL_PATH: '/models/THUDM/glm-4-9b-chat'

volumes:
- /root/models/THUDM/glm-4-9b-chat:/models/THUDM/glm-4-9b-chat:ro

runtime: nvidia
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]

# network_mode: "host"

ports:
- "18080:18080"

command: >
bash -c "python3 -u main.py"
```

# 2. 后端服务日志:
```
14:02 | ..v_variable:33 | [ENV_VAR] 尝试加载CHATGLM_LOCAL_MODEL_PATH,默认值:THUDM/glm-4-9b-chat --> 修正值:/models/THUDM/glm-4-9b-chat
14:02 | ..v_variable:60 | [ENV_VAR] 成功读取环境变量CHATGLM_LOCAL_MODEL_PATH
14:02 | ..v_variable:33 | [ENV_VAR] 尝试加载LOCAL_MODEL_DEVICE,默认值:cpu --> 修正值:cuda
14:02 | ..v_variable:60 | [ENV_VAR] 成功读取环境变量LOCAL_MODEL_DEVICE
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Loading checkpoint shards: 0%| | 0/10 [00:00

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at request_llms/bridge_chatglm4.py line 63 and follow the model-loading path into request_llms/local_llm_class.py. Reproduce the failure with the provided docker-compose configuration, then inspect the generation call and the referenced modeling_chatglm.py cache-handling path. Done means the downloaded GLM-4 model starts and generates a response without the unpacking error.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, python, pytorch
Domain
backend, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.