Pipeline parallelism
Offen
- Vorherrschende Sprache
- C
- Sterne
- 22.3k
- Forks
- 2.1k
- Ø Merge
- 1 T. 3 Std.
- Gemergte PRs (30 T.)
- 4
Beschreibung
I have a 2 node Strix halo 128GB setup (256GB total) running Deepseek V4 flash. I noticed that at all times one of the nodes is doing nothing waiting for the other node to complete its work. All the while requests are being piled up waiting to be processed.
Maybe we can do something so that multiple nodes can process different requests in parallel so they don't sit there doing nothing?
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.