DjangoPeng / DjangoPeng/LLM-quickstart

【AWQ_transformers.ipynb】下载facebook-opt-6.7模型到本地再加载报错

Open
#11 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
1.1k
Forks
590
PR merge metrics
No merged PRs in 30d

Description

### 作业: fine-tune-quickstart.ipynb

### **目标:**

因网络不稳定,打算把"facebook/opt-6.7b"模型下载到本地后再加载使用,进行量化。

### **代码如下:**

#下载模型
![b6e665c761cca3c138e6a088306c841](https://github.com/DjangoPeng/LLM-quickstart/assets/20674334/f39dd44f-0de4-4799-a0ae-50c1977ed868)

#加载模型
![48da132622d4042f1fb94ef1b8740bc](https://github.com/DjangoPeng/LLM-quickstart/assets/20674334/050217b3-4555-4d8c-92ef-9b758da806d2)

### **报错如下:**
![707458bbf54dedbf62497fa2061f006](https://github.com/DjangoPeng/LLM-quickstart/assets/20674334/5a49f109-3e42-4984-ae72-8eca490b4266)

**经查验本地存储内容为:**
![e3d6cc50f8e697d51643348763c27d0](https://github.com/DjangoPeng/LLM-quickstart/assets/20674334/308dfe78-f626-44db-aeda-5a5257b382a5)

完全和官网文件格式不一致,改为直接读取已经存在本地的缓存文件反而没有问题。

### **问题**

这是否说明:用transformers的from_pretrained方法从hf下载的原始模型到本地,用save_pretrained方法保存后,再用from_pretrained从本地指定路径读取这一方法是有使用限制的?

为什么会出现这样的问题?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with AWQ_transformers.ipynb and the referenced fine-tune-quickstart.ipynb, then compare the files produced by the local save with what the local from_pretrained call expects. Reproduce the error using the shown local path and determine whether the saved model format or loading arguments cause it. Done means explaining the limitation and documenting a reproducible local loading path.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, jupyter-notebook, python
Domain
ai, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.