microsoft / microsoft/DirectXShaderCompiler

[HLSL] LinAlg HLK: Add tolerance and ULP comparison modes to the matrix oracle

Open
#8,729 0 comments 0 reactions 1 assignee View on GitHub

@JoeCitizen is already working on this.

Since Aug 6, 2026.

Dominant language
C++
Stars
3.7k
Forks
900
Avg merge
2d 11h
Merged PRs (30d)
44

Description

Goal

Extend MatrixResultOracle in LinAlgTests.cpp beyond bit-exact comparison, so operations whose results the specification does not pin down uniquely can be verified without either weakening the test or enumerating an impractical number of candidate matrices.

Why

The oracle's three modes — Exact, PermittedResults and Excluded — all compare encoded component bits exactly. PermittedResults expresses "the spec permits this result or that result" as an enumerated set of complete candidate matrices; it is not a tolerance band.

That is correct and deliberate for the cases covered today, where every value is exactly representable in its component type and bit-exactness is both stronger than a tolerance and reproducible across implementations. It does not extend to operations where the specification leaves a range: matrix multiply accumulation order over a large K, conversion rounding, and results produced through the optimal layouts. Enumerating candidates there is not viable.

The harness already has the vocabulary for this. ValidationType::Epsilon and ValidationType::Ulp in HlslTestDataTypes.h are used by the pre-existing verifyFloatBuffer and verifyHalfBuffer paths in this same file. The oracle should adopt those rather than introduce a fourth comparison vocabulary.

Required work
  • Add tolerance-based and ULP-based comparison to MatrixResultOracle, reusing ValidationType and the existing doValuesMatch overloads.
  • Keep Exact the default; a case must opt in to a looser comparison and state why.
  • Require that any tolerance is derived from the specification or from an independent error analysis, and record that derivation at the call site.
  • Report the worst observed deviation on failure, not just the first mismatch.
Acceptance criteria
  • At least one operation whose result the spec does not uniquely determine is covered with a justified tolerance.
  • No existing case is relaxed from Exact as part of this work.
  • Tolerances are traceable to a public specification statement or a written derivation; none is fitted to an implementation's observed output.
Blocked on

Nothing, but it should land with the first case that genuinely needs it so the tolerance is chosen against a real result rather than guessed.

Public references
  • Proposal 0035 (hlsl-specs), Linear Algebra matrix operations.
  • Review thread on #8666 at L271.
Out of scope
  • Relaxing any currently exact expectation.
  • Templating the oracle, and the checked-arithmetic redesign (tracked separately).

Assisted-by: GitHub Copilot

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.