训练MOSS所需的最低资源
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
请问训练160亿参数MOSS最少需要多少计算资源呢？
以下是我在开源的微调多轮对话数据zh_honesty.json中找到的信息，请问靠谱吗？因为我记得之前哪里看过8卡A100最大训练130亿参数来着？
“MOSS是一种基于深度学习的自然语言处理技术，它可以帮助人们快速、准确地完成各种自然语言处理任务，包括问答、文本翻译、文本摘要、文本生成、情绪分析和算法开发等。它的核心是8张A100 GPU训练出的160亿参数，可以模拟人类的思维，实现自然语言理解、文本生成等复杂任务。”
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file, test, or entry point is identified. Start by checking the training documentation and the referenced zh_honesty.json context, then compare the stated A100 requirements with the model's published training setup. Done means documenting a supported minimum-resource estimate and its assumptions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- machine-learning
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100