请将XAgentGen的tensor_parallel_size参数外放
Open
@AL-377 is already working on this.
Since Nov 29, 2023.
enhancement
- Dominant language
- Python
- Stars
- 8.5k
- Forks
- 905
- PR merge metrics
- No merged PRs in 30d
Description
Feature Description / 功能描述
加载XAgentLLaMa-34B-preview大模型时,一张4090无法加载,当前tensor_parallel_size默认是1
Goal / 目标
当一张GPU无法加载大模型时,可以通过多张GPU进行加载
Expected Behavior / 预期行为
请将tensor_parallel_size参数外放,并更新xagentteam/xagentgen的docker镜像
Alternative Solutions / 其他解决方案
无
Additional Notes / 其他备注
无
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.