microsoft / microsoft/DirectXShaderCompiler

Segfault when trying to use hull shader as lib.

Open
#6,622 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug crash incorrect-code
Dominant language
C++
Stars
3.7k
Forks
900
Avg merge
2d 11h
Merged PRs (30d)
44

Description

Description
This is generated code from a tool I built. Expectation is an error that this is only supported on hull shader stages. The issue seems to be because of the patchconstantfunc .

Steps to Reproduce
Run DXC_Debug_BUILD/bin/dxc -T lib_6_8 -enable-16bit-types -O0 test.hlsl

RWStructuredBuffer<float> buffer : register(u0);
void fn () {}

[domain("tri")]
[outputtopology("triangle_cw")]
[patchconstantfunc("fn")]
[outputcontrolpoints(3)]
[partitioning("pow2")]
void main( const uint index : SV_OutputControlPointID) {
	buffer[index] = 1;
}

Actual Behavior

Process 1116521 launched: '/mnt/DevDrive/projects/HLSLDocGenerator/DXC_Debug_BUILD/bin/dxc' (x86_64)
Process 1116521 stopped
* thread #1, name = 'dxc', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x20)
    frame #0: 0x00007ffff446a15c libdxcompiler.so`llvm::GlobalValue::getLinkage(this=0x0000000000000000) const at GlobalValue.h:289:44
   286        Visibility = DefaultVisibility;
   287      Linkage = LT;
   288    }
-> 289    LinkageTypes getLinkage() const { return Linkage; }
   290 
   291    bool isDiscardableIfUnused() const {
   292      return isDiscardableIfUnused(Linkage);
(lldb) bt
* thread #1, name = 'dxc', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x20)
  * frame #0: 0x00007ffff446a15c libdxcompiler.so`llvm::GlobalValue::getLinkage(this=0x0000000000000000) const at GlobalValue.h:289:44
    frame #1: 0x00007ffff50778c7 libdxcompiler.so`CGHLSLMSHelper::UpdateLinkage(HLM=0x0000555555a220b0, CGM=0x0000555555a1f1c0, exportMap=0x0000555555a21ba0, entryFunctionMap=0x0000555555a21d68, patchConstantFunctionMap=0x0000555555a21cd8) at CGHLSLMSFinishCodeGen.cpp:3474:17
    frame #2: 0x00007ffff50281a1 libdxcompiler.so`(anonymous namespace)::CGMSHLSLRuntime::FinishCodeGen(this=0x0000555555a21820) at CGHLSLMS.cpp:3980:3
    frame #3: 0x00007ffff4df5fed libdxcompiler.so`clang::CodeGen::CodeGenModule::FinishCodeGen(this=0x0000555555a1f1c0) at CodeGenModule.cpp:246:16
    frame #4: 0x00007ffff4df7855 libdxcompiler.so`clang::CodeGen::CodeGenModule::Release(this=0x0000555555a1f1c0) at CodeGenModule.cpp:447:5
    frame #5: 0x00007ffff4d75477 libdxcompiler.so`(anonymous namespace)::CodeGeneratorImpl::HandleTranslationUnit(this=0x0000555555a1a0f0, Ctx=0x0000555555a03ec0) at ModuleBuilder.cpp:232:18
    frame #6: 0x00007ffff4d56425 libdxcompiler.so`clang::BackendConsumer::HandleTranslationUnit(this=0x0000555555a19ff0, C=0x0000555555a03ec0) at CodeGenAction.cpp:147:14
    frame #7: 0x00007ffff62be9d4 libdxcompiler.so`clang::ParseAST(S=0x0000555555a1ad50, PrintStats=false, SkipFunctionBodies=false) at ParseAST.cpp:164:13
    frame #8: 0x00007ffff50ff656 libdxcompiler.so`clang::ASTFrontendAction::ExecuteAction(this=0x00007fffffff9ea0) at FrontendAction.cpp:554:3
    frame #9: 0x00007ffff4d54d41 libdxcompiler.so`clang::CodeGenAction::ExecuteAction(this=0x00007fffffff9ea0) at CodeGenAction.cpp:807:28
    frame #10: 0x00007ffff50ff14c libdxcompiler.so`clang::FrontendAction::Execute(this=0x00007fffffff9ea0) at FrontendAction.cpp:455:8
    frame #11: 0x00007ffff43e5459 libdxcompiler.so`DxcCompiler::Compile(this=0x00005555559d9af0, pSource=0x00007fffffffc540, pArguments=0x00005555559d9a10, argCount=7, pIncludeHandler=0x00005555559d9640, riid=0x00005555557fd6b0, ppResult=0x00007fffffffba38) at dxcompilerobj.cpp:984:18
    frame #12: 0x00007ffff43d9dbd libdxcompiler.so`hlsl::DxcCompilerAdapter::WrapCompile(this=0x00005555559d9f10, bPreprocess=NO, pSource=0x00005555559da1d0, pSourceName=L"scratch/GSInstanceID_test.hlsl", pEntryPoint=L"lib.no::entry", pTargetProfile=L"lib_6_8", pArguments=0x00005555559d99b0, argCount=4, pDefines=0x00007fffffffd188, defineCount=0, pIncludeHandler=0x00005555559d9640, ppResult=0x00007fffffffcd40, ppDebugBlobName=0x0000000000000000, ppDebugBlob=0x0000000000000000) at dxcompilerobj.cpp:1845:5
    frame #13: 0x00007ffff43db2ac libdxcompiler.so`hlsl::DxcCompilerAdapter::CompileWithDebug(this=0x00005555559d9f10, pSource=0x00005555559da1d0, pSourceName=L"scratch/GSInstanceID_test.hlsl", pEntryPoint=L"lib.no::entry", pTargetProfile=L"lib_6_8", pArguments=0x00005555559d99b0, argCount=4, pDefines=0x00007fffffffd188, defineCount=0, pIncludeHandler=0x00005555559d9640, ppResult=0x00007fffffffcd40, ppDebugBlobName=0x0000000000000000, ppDebugBlob=0x0000000000000000) at dxcompilerobj.cpp:1748:10
    frame #14: 0x00007ffff43df066 libdxcompiler.so`hlsl::DxcCompilerAdapter::Compile(this=0x00005555559d9f10, pSource=0x00005555559da1d0, pSourceName=L"scratch/GSInstanceID_test.hlsl", pEntryPoint=L"lib.no::entry", pTargetProfile=L"lib_6_8", pArguments=0x00005555559d99b0, argCount=4, pDefines=0x00007fffffffd188, defineCount=0, pIncludeHandler=0x00005555559d9640, ppResult=0x00007fffffffcd40) at dxcompileradapter.h:75:12
    frame #15: 0x00005555555c1c17 dxc`DxcContext::Compile(this=0x00007fffffffcfe8) at dxc.cpp:874:9
    frame #16: 0x00005555555c5db3 dxc`dxc::main(argc=6, argv_=0x00007fffffffdcc8) at dxc.cpp:1502:24
    frame #17: 0x00005555555ba732 dxc`main(argc=6, argv_=0x00007fffffffdcc8) at dxcmain.cpp:19:10
    frame #18: 0x00007ffff3a29d90 libc.so.6`__libc_start_call_main(main=(dxc`main at dxcmain.cpp:18), argc=6, argv=0x00007fffffffdcc8) at libc_start_call_main.h:58:16
    frame #19: 0x00007ffff3a29e40 libc.so.6`__libc_start_main_impl(main=(dxc`main at dxcmain.cpp:18), argc=6, argv=0x00007fffffffdcc8, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffdcb8) at libc-start.c:392:3
    frame #20: 0x00005555555ba645 dxc`_start + 37

Environment

  • DXC version e7b78ff9c99c19a6a0c98256db9794e0af4eb59d
  • Host Operating System Ubuntu 22.04

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with CGHLSLMSFinishCodeGen.cpp at CGHLSLMSHelper::UpdateLinkage, line 3474, and trace how patchConstantFunctionMap is handled during FinishCodeGen. Reproduce with the supplied dxc command and HLSL input. Done means the invalid hull-shader usage produces the expected diagnostic instead of a segmentation fault.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.