microsoft / microsoft/DirectXShaderCompiler
IsHelperLane() emulation pre-SM 6.6 is broken
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 3.7k
- Forks
- 900
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 44
Description
IsHelperLane() emulation pre SM 6.6 is broken when using sample-rate shading.
The existing workaround checks for input coverage being non-zero to determine if the lane is a helper or not, but this breaks if there are sibling samples which are in-fact not helpers. A helper invocation will be treated as not being a helper by this check, which could lead to catastrophic hang behavior in some cases.
This test uncovers the broken behavior: https://github.com/HansKristian-Work/vkd3d-proton/blob/master/tests/d3d12_pso.c#L5154
D3D11.3 spec is a little unclear if sample-rate shading should receive coverage for samples which are not part of the current pixel, but all real world implementations seem to do that at least, including WARP. Vulkan defines the sample mask to only contain the currently shaded samples when per-sample shading is used, so that's an interesting behavioral difference (which would have avoided this bug), but not super relevant here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the regression test at tests/d3d12_pso.c around line 5154 in the linked vkd3d-proton repository and reproduce the failure involving sample-rate shading. Trace the pre-SM 6.6 IsHelperLane() emulation used by the compiler, then verify that helper-lane classification remains correct when sibling samples are not helpers. Done means the regression test passes without risking the described hang behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100