KhronosGroup / KhronosGroup/glslang
Mismatch between vertex output and hull shader input (validation error)
- Dominant language
- C++
- Stars
- 3.6k
- Forks
- 989
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 31
Description
Hi,
We've recently come across an issue where the vertex shader output is mismatched with the hull shader input, because the vertex shader flattens the output and sets location 0, location 1, etc. whereas the hull shader declares a single struct with location 0. This was all working until not long ago, but the newer SDK and validation layers have uncovered the problem.
The repro would be a very simple shader such as:
```hlsl
struct VertexOutput
{
float4 a : ATTRIBUTE0;
float4 b : ATTRIBUTE1;
};
VertexOutput VertexShader()
{
VertexOutput output = (VertexOutput)0;
return output;
}
struct HullConstantOutput
{
float edges[3] : SV_TessFactor;
float inside : SV_InsideTessFactor;
};
HullConstantOutput HullConstantFunction(InputPatch patches)
{
HullConstantOutput o = (HullConstantOutput)0;
return o;
}
[domain("tri")]
[partitioning("fractional_odd")]
[outputtopology("triangle_cw")]
[outputcontrolpoints(3)]
VertexOutput HullShader(InputPatch patches, uint pointID : SV_OutputControlPointID)
{
return patches[pointID];
}
```
If I compile the vertex shader, the glslang SPIRV output is as follows:
```hlsl
1: ExtInstImport "GLSL.std.450"
MemoryModel Logical GLSL450
EntryPoint Vertex 4 "VertexShader" 23 29
Source HLSL 500
Name 4 "VertexShader"
Name 23 "@entryPointOutput.a"
Name 29 "@entryPointOutput.b"
Decorate 23(@entryPointOutput.a) Location 0
Decorate 29(@entryPointOutput.b) Location 1
...
```
whereas the hull shader outputs
```hlsl
1: ExtInstImport "GLSL.std.450"
MemoryModel Logical GLSL450
EntryPoint TessellationControl 4 "HullShader" 27 31 34
ExecutionMode 4 OutputVertices 3
ExecutionMode 4 Triangles
ExecutionMode 4 SpacingFractionalOdd
ExecutionMode 4 VertexOrderCw
Source HLSL 500
Name 4 "HullShader"
Name 8 "VertexOutput"
MemberName 8(VertexOutput) 0 "a"
MemberName 8(VertexOutput) 1 "b"
Name 27 "patches"
Name 31 "pointID"
Name 34 "@entryPointOutput"
Name 36 "param"
Decorate 27(patches) Location 0
Decorate 31(pointID) BuiltIn InvocationId
Decorate 34(@entryPointOutput) Location 0
2: TypeVoid
3: TypeFunction 2
6: TypeFloat 32
7: TypeVector 6(float) 4
8(VertexOutput): TypeStruct 7(fvec4) 7(fvec4)
...
```
What I see there is that the first declares individual locations, whereas the second declares a struct with some offsets, and the validation layer will catch it. In our similar case (not for this simple example shader) this will throw a validation error saying
> Type mismatch on location 0.0: 'ptr to output vec4 of float32' vs 'ptr to input arr[3] of struct of (struct of (vec4 of float32, vec4 of float32, vec4 of float32, vec4 of float32, vec2 of float32), vec4 of uint32)'
We are required to have no validation layer errors to submit our game, it would be great if you could advise either on a workaround or whether you can see a solution in the near future coming to be able to fix this.
My worry now is that there have been similar issues reported some time ago such as https://github.com/KhronosGroup/glslang/issues/1737 and https://github.com/KhronosGroup/glslang/issues/1197 and they haven't been fixed before.
Thanks!
Contributor guide
Research direction
Start with the supplied HLSL VertexShader and HullShader entry points and compare their generated SPIR-V interface declarations, especially Location 0/1 versus the struct. Reproduce the validation-layer mismatch and define done as compatible interfaces for the repro with no interface type-mismatch error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100