LinAlg HLK: Qualify the suite on WARP and representative third-party implementations

Goal

Qualify the completed LinAlg HLK suite through the existing DXIL HLK pipeline on WARP and agreed representative third-party compiler/driver implementations.

Qualification must prove that mandatory cases pass, capability-gated cases execute when advertised, unsupported legal permutations are classified correctly and the suite is stable enough for certification use.

Why

Local ExecutionTest success proves the harness and one runtime path, but it does not establish that the final packaged HLK correctly handles the capability differences and implementation behaviour of third-party devices.

Qualification is also needed to find tests that accidentally depend on one implementation's Wave size, matrix tile shape, floating-point behaviour or unsupported feature assumptions.

Required work

• Run the packaged suite from #8656 on the approved WARP/Agility configuration.
• Run it on the agreed representative third-party implementations without publishing private driver or lab details.
• Record the advertised LinAlg tier and operation capabilities before classifying results.
• Confirm mandatory tests execute and pass on every implementation that advertises the required base capability.
• For each capability-gated case, record the relevant public query category, query inputs and advertised result before execution is classified.
• Execute a case whenever its relevant public query advertises support. Do not use scope, dimensions, layout, alignment or another axis as a NotApplicable reason unless that axis is exposed by the corresponding public query.
• Confirm unsupported legal permutations follow #8647 and are not reported as passes or implementation failures.
• Repeat representative runs to identify nondeterminism, races or results outside the exact or specification-derived permitted expectations established by #8649 before qualification begins.
• Freeze each case's exact expectation or specification-derived permitted-result set before comparing WARP and third-party results; do not broaden acceptance after observing an implementation.
• Measure total suite runtime and individual outliers against the agreed HLK runtime budget.
• Triage failures as test defects, compiler defects, driver defects, capability-reporting defects or specification ambiguities.
• File separate public defects where appropriate without placing private implementation details in this issue.
• Exercise only the public dx::linalg API in HLK shaders except for documented operations with no public wrapper.

Acceptance criteria

• The full packaged suite passes on the approved WARP configuration.
• The suite has been exercised on the agreed representative third-party implementations.
• Every failure is classified and either resolved or tracked by a separate defect.
• Mandatory cases do not skip or report NotApplicable on implementations advertising the required capability.
• Every NotApplicable result identifies the public capability query and returned value that justified it; an unqueryable axis is never used as the skip reason.
• Capability-gated cases run whenever the corresponding public query advertises support.
• Unsupported permutations follow the reviewed #8647 policy and never appear as passes.
• Repeated runs show no unexplained nondeterminism or intermittent failures.
• Floating-point results satisfy the pre-reviewed #8649 exact or permitted-result model on every qualified implementation.
• No expectation, permitted-result set or tolerance is widened solely to accommodate a qualification failure.
• Total runtime and per-case outliers satisfy the agreed certification budget.
• Public qualification evidence records test revision, public runtime versions and result counts without exposing private driver, device or lab information.

Public references

• Parent tracker #7841
• Capability handling #8647
• Independent oracle #8649
• Existing-pipeline integration #8656
• D3D12 Linear Algebra Runtime Feature Support

Out of scope

• Publishing private drivers, device identifiers, contacts, schedules or lab topology.
• Comparing vendor performance.
• Expanding the coverage matrix during qualification instead of filing focused follow-up issues.
• Treating WARP-only success as final qualification.

Assisted-by: GitHub Copilot