iree-org / iree-org/iree

Add post-host-wait timepoint elision to iree-stream-elide-timepoints pass.

Open
#9,747 0 comments 0 reactions 1 assignee View on GitHub

@benvanik is already working on this.

Since Jul 8, 2022.

compiler/dialects performance ⚡
Dominant language
C++
Stars
3.9k
Forks
1k
Avg merge
4d 16h
Merged PRs (30d)
47

Description

Today the pass is only looking at timepoint dependencies in order to elide redundant waits - we should also look for waits that can be elided due to a prior host wait on the code path. Unlike the basic timepoint elision there's no SSA value chain and instead we'll need to look at execution order/reachability/dominance. I know I've written that before but can't find it.

Example (from unidirectional_lstm):

%16 = stream.cmd.execute await(%4) => with(%5 as %arg12: !stream.resource<transient>{%6}, %15 as %arg13: !stream.resource<staging>{%c4}) {
  stream.cmd.copy %arg12[%7], %arg13[%c0], %c4 : !stream.resource<transient>{%6} -> !stream.resource<staging>{%c4}
  stream.cmd.flush %arg13[%c0 for %c4] : !stream.resource<staging>{%c4}
} => !stream.timepoint
%17 = stream.timepoint.await %16 => %15 : !stream.resource<staging>{%c4}
%18 = stream.resource.load %17[%c0] : !stream.resource<staging>{%c4} -> i32
%result_2, %result_timepoint_3 = stream.resource.alloca uninitialized await(%4) => !stream.resource<transient>{%c576} => !stream.timepoint

Here %4 is waited on to produce %16, which then the host waits on ensuring it is reached. The subsequent alloca waits on %4 again when instead it should be immediate.

It's kind of like liveness analysis in that we could walk by SCC, check block dominance for values outside the region, and then build a per-op bitmap of reached timepoints. Ideally we want to do this as part of the TimepointCoverage analysis so that we could propagate the information: above %result_timepoint_3 should cover %4. To keep things simpler, though, it's probably worth having its own ElideRedundantSynchronizationPass in the same fixed-point iteration pipeline as that should effectively do the same thing (at a higher cost).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.