Pipeline parallelism
Đang mở
- Ngôn ngữ chính
- C
- Star
- 22.3k
- Fork
- 2.1k
- Merge trung bình
- 1 ngày 3 giờ
- Pull request đã merge (30 ngày)
- 4
Mô tả
I have a 2 node Strix halo 128GB setup (256GB total) running Deepseek V4 flash. I noticed that at all times one of the nodes is doing nothing waiting for the other node to complete its work. All the while requests are being piled up waiting to be processed.
Maybe we can do something so that multiple nodes can process different requests in parallel so they don't sit there doing nothing?
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.