microsoft / microsoft/DirectXShaderCompiler
[HLSL] LinAlg HLK: Add tolerance and ULP comparison modes to the matrix oracle
@JoeCitizen is already working on this.
Since Aug 6, 2026.
- Dominant language
- C++
- Stars
- 3.7k
- Forks
- 900
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 44
Description
Goal
Extend MatrixResultOracle in LinAlgTests.cpp beyond bit-exact comparison, so operations whose results the specification does not pin down uniquely can be verified without either weakening the test or enumerating an impractical number of candidate matrices.
Why
The oracle's three modes — Exact, PermittedResults and Excluded — all compare encoded component bits exactly. PermittedResults expresses "the spec permits this result or that result" as an enumerated set of complete candidate matrices; it is not a tolerance band.
That is correct and deliberate for the cases covered today, where every value is exactly representable in its component type and bit-exactness is both stronger than a tolerance and reproducible across implementations. It does not extend to operations where the specification leaves a range: matrix multiply accumulation order over a large K, conversion rounding, and results produced through the optimal layouts. Enumerating candidates there is not viable.
The harness already has the vocabulary for this. ValidationType::Epsilon and ValidationType::Ulp in HlslTestDataTypes.h are used by the pre-existing verifyFloatBuffer and verifyHalfBuffer paths in this same file. The oracle should adopt those rather than introduce a fourth comparison vocabulary.
Required work
- Add tolerance-based and ULP-based comparison to
MatrixResultOracle, reusingValidationTypeand the existingdoValuesMatchoverloads. - Keep
Exactthe default; a case must opt in to a looser comparison and state why. - Require that any tolerance is derived from the specification or from an independent error analysis, and record that derivation at the call site.
- Report the worst observed deviation on failure, not just the first mismatch.
Acceptance criteria
- At least one operation whose result the spec does not uniquely determine is covered with a justified tolerance.
- No existing case is relaxed from
Exactas part of this work. - Tolerances are traceable to a public specification statement or a written derivation; none is fitted to an implementation's observed output.
Blocked on
Nothing, but it should land with the first case that genuinely needs it so the tolerance is chosen against a real result rather than guessed.
Public references
- Proposal 0035 (
hlsl-specs), Linear Algebra matrix operations. - Review thread on #8666 at L271.
Out of scope
- Relaxing any currently exact expectation.
- Templating the oracle, and the checked-arithmetic redesign (tracked separately).
Assisted-by: GitHub Copilot
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.