microsoft / microsoft/DirectXShaderCompiler

Compiling hull shader with unused globals causes internal compiler error

Open
#4,958 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug crash dxil
Dominant language
C++
Stars
3.7k
Forks
900
Avg merge
2d 11h
Merged PRs (30d)
44

Description

ver: v1.7.2212

When compiling the hull shader below, you'll get a crash with an access violation, Commenting out either global eliminates the exception. It appears to be a result of combining the arithmetic when initializing g_ACESRRTDesaturationMatrix with a thread_local array of resource descriptors.

It gets more unusual when you start playing with the size of gProjTextureMaps. 0 and 2 appear to succeed? 1 is a read access violation at 0xFFFFFFFFFFFFFFFF and anything greater than 2 is a read access violation at 0x0000000000000000. Additionally, neither of these globals are even referenced in the shader.

// -T hs_6_6 -E mainHS -Fo output.dxil debug_hs.hlsl
//
//

#define g_ACESAP1ToY				float3(0.2722287, 0.6740818, 0.0536895)
#define g_ACESRRTDesaturationFactor	0.96

// Commenting out this stops the "Internal compiler error"
// Seems to be the arithmetic?
// This isn't used in the file!
static const float3x3 g_ACESRRTDesaturationMatrix = float3x3(
	g_ACESAP1ToY.x * (1.0 - g_ACESRRTDesaturationFactor) + g_ACESRRTDesaturationFactor,	g_ACESAP1ToY.y * (1.0 - g_ACESRRTDesaturationFactor),									g_ACESAP1ToY.z * (1.0 - g_ACESRRTDesaturationFactor),
	g_ACESAP1ToY.x * (1.0 - g_ACESRRTDesaturationFactor),								g_ACESAP1ToY.y * (1.0 - g_ACESRRTDesaturationFactor) + g_ACESRRTDesaturationFactor,	g_ACESAP1ToY.z * (1.0 - g_ACESRRTDesaturationFactor),
	g_ACESAP1ToY.x * (1.0 - g_ACESRRTDesaturationFactor),								g_ACESAP1ToY.y * (1.0 - g_ACESRRTDesaturationFactor),								g_ACESAP1ToY.z * (1.0 - g_ACESRRTDesaturationFactor) + g_ACESRRTDesaturationFactor
);

// 0 is ok?
// 1 is bad - Attempted to read from address 0xFFFFFFFFFFFFFFFF
// 2 is ok?
// greater than 2 is bad - Attempted to read from address 0x0000000000000000
#define ARRAY_SIZE 1

// OR Commenting out this stops the "Internal compiler error"
// This isn't used in the file!
static Texture2D<float4> gProjTextureMaps[ARRAY_SIZE];

struct VS_OUTPUT
{
	float4 Position : SV_POSITION;
};

struct HS_OUTPUT
{
	 float3 PositionWS : POSITION_WS;
};

struct HS_OUTPUT_CONST
{
	uint VertexID : VERTEX_ID;
	float EdgeTessFactors[3] : SV_TessFactor;
	float InsideTessFactors[1] : SV_InsideTessFactor;
};

HS_OUTPUT_CONST constHS(InputPatch<VS_OUTPUT, 12> hsInputPatch, OutputPatch<HS_OUTPUT, 3> hsOutputPatch)
{
	HS_OUTPUT_CONST hsOutputConst = (HS_OUTPUT_CONST)0;
	return hsOutputConst;
}

[domain("tri")]
[partitioning("integer")]
[outputtopology("triangle_cw")]
[outputcontrolpoints(3)]
[patchconstantfunc("constHS")]
[maxtessfactor(64.0)]
HS_OUTPUT mainHS(InputPatch<VS_OUTPUT, 12> hsInputPatch, uint primitiveID : SV_PrimitiveID, uint controlPointID : SV_OutputControlPointID)
{
	HS_OUTPUT hsVertexOutput = (HS_OUTPUT)0;
	return hsVertexOutput;
}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the crash with debug_hs.hlsl using -T hs_6_6 -E mainHS -Fo output.dxil, then vary ARRAY_SIZE and the two unused globals as described. Trace the compiler path reached by mainHS and constHS; done means the shader compiles without an internal compiler error for the reported cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.