LoRA微调和量化问题
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
在微调MOSS的时候遇到了以下问题：
由于MOSS太大了无法单卡微调，而INT4模型又不支持多卡微调，目前的方案是多卡LoRA微调fp16的模型。但是在部署推理的时候依旧需要转换成INT4的形式进行部署，目前的方案是
- 合并LoRA参数和MOSS参数
- 量化成INT4
请问有没有大佬做过这两步的，求教
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. First locate the LoRA training and INT4 quantization or deployment entry points, then determine whether merging the adapter with MOSS and quantizing the result are supported. Done would require a reproducible workflow or a clearly documented answer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100