inducer / inducer/pytato

Submit multiple GPU queues per DAG execution?

Open
#197 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
15
Forks
16
Avg merge
17m
Merged PRs (30d)
1

Description

Listening to @tomhers talk just now, I was wondering whether we get some benefit just by chunking up a DAG to submit two(-ish?) separate command queues, with the idea that the second queue would help cover the fill/drain latencies at kernel boundaries of the first. No scheduling at all, just relying on the GPU scheduler to figure things out. Given that we have #139 merged (and possibly with some additional widgets from #148, this could be very cheap to try.

Upon discussing with @kaushikcfd just now, we figured that we don't know whether this very simple thing would be remotely beneficial. A microbenchmark (e.g.: measure throughput of small-ish vector adds on one queue vs two) could maybe help provide some clarity. Update: @kaushikcfd unearthed [a paper](https://doi.org/10.1145/2490301.2451160). It could also serve as a point of comparison for the advanced things that @tomhers is gearing up to do.

cc @matthiasdiener @kaushikcfd @rwerger

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.