Pipeline parallelism
未关闭
- 主要语言
- C
- 星标
- 22.3k
- 派生
- 2.1k
- 平均合并
- 1 天 3 小时
- 30 天内合并 PR
- 4
描述
I have a 2 node Strix halo 128GB setup (256GB total) running Deepseek V4 flash. I noticed that at all times one of the nodes is doing nothing waiting for the other node to complete its work. All the while requests are being piled up waiting to be processed.
Maybe we can do something so that multiple nodes can process different requests in parallel so they don't sit there doing nothing?
贡献指南
评估
这个 Issue 还没有评估数据。