microsoft / microsoft/DirectXShaderCompiler

[Bug] segfault when using clip

Open
#6,590 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug crash incorrect-code
Dominant language
C++
Stars
3.7k
Forks
900
Avg merge
2d 11h
Merged PRs (30d)
44

Description

Description
This code is invalid. was generated via a code gen tool. The problem is we are segfaulting instead of failing with an error or asserting.

code:

[numthreads(1, 1, 1)]
[shader("pixel")]
void fn(float4 p1: SV_ViewID) : SV_Target {
    return clip(p1);
}

stack trace:

* thread #1, name = 'dxc', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x98)
  * frame #0: 0x00007ffff454686c libdxcompiler.so`llvm::Function::getFunctionType(this=0x0000000000000000) const at Function.cpp:227:58
    frame #1: 0x00007ffff46a61dd libdxcompiler.so`llvm::IRBuilder<true, llvm::ConstantFolder, llvm::IRBuilderDefaultInserter<true>>::CreateCall(this=0x00007fffffff73c0, Callee=0x0000000000000000, Args=ArrayRef<llvm::Value *> @ 0x00007fffffff6e10, Name=0x00007fffffff6f20) at IRBuilder.h:1530:31
    frame #2: 0x00007ffff49103b4 libdxcompiler.so`(anonymous namespace)::replaceInputOutputWithIntrinsic(semKind=ViewID, GV=0x0000555555a6e190, hlslOP=0x0000555555a21d60, Builder=0x00007fffffff73c0) at HLSignatureLower.cpp:189:22
    frame #3: 0x00007ffff490f67e libdxcompiler.so`hlsl::HLSignatureLower::ProcessArgument(this=0x00007fffffff7a48, func=0x0000555555a6d8f8, funcAnnotation=0x0000555555a6dbe0, arg=0x0000555555a6e190, props=0x0000555555a6dac0, pSM=0x00007ffff7dea268, isPatchConstantFunction=false, forceOut=false, hasClipPlane=0x00007fffffff77ff) at HLSignatureLower.cpp:367:7
    frame #4: 0x00007ffff4910e60 libdxcompiler.so`hlsl::HLSignatureLower::CreateDxilSignatures(this=0x00007fffffff7a48) at HLSignatureLower.cpp:478:5
    frame #5: 0x00007ffff4918f3b libdxcompiler.so`hlsl::HLSignatureLower::Run(this=0x00007fffffff7a48) at HLSignatureLower.cpp:1794:5
    frame #6: 0x00007ffff489f003 libdxcompiler.so`(anonymous namespace)::DxilGenerationPass::runOnModule(this=0x0000555555a959a0, M=0x0000555555a1a4d0) at DxilGenerationPass.cpp:240:22
    frame #7: 0x00007ffff679859d libdxcompiler.so`(anonymous namespace)::MPPassManager::runOnModule(this=0x0000555555a0e030, M=0x0000555555a1a4d0) at LegacyPassManager.cpp:1669:27
    frame #8: 0x00007ffff6798043 libdxcompiler.so`llvm::legacy::PassManagerImpl::run(this=0x0000555555a83120, M=0x0000555555a1a4d0) at LegacyPassManager.cpp:1771:44
    frame #9: 0x00007ffff6798c44 libdxcompiler.so`llvm::legacy::PassManager::run(this=0x0000555555a83090, M=0x0000555555a1a4d0) at LegacyPassManager.cpp:1814:14
    frame #10: 0x00007ffff4d7d540 libdxcompiler.so`(anonymous namespace)::EmitAssemblyHelper::EmitAssembly(this=0x00007fffffff8588, Action=Backend_EmitBC, OS=0x0000555555a19900) at BackendUtil.cpp:756:22
    frame #11: 0x00007ffff4d7c8be libdxcompiler.so`clang::EmitBackendOutput(Diags=0x00005555559dd940, CGOpts=0x00005555559dc400, TOpts=0x00005555559dd020, LOpts=0x00005555559dcea0, TDesc=(Data = "e-m:e-p:32:32-i1:32-i8:8-i16:16-i32:32-i64:64-f16:16-f32:32-f64:64-n8:16:32:64", Length = 78), M=0x0000555555a1a4d0, Action=Backend_EmitBC, OS=0x0000555555a19900) at BackendUtil.cpp:779:15
    frame #12: 0x00007ffff4d566b9 libdxcompiler.so`clang::BackendConsumer::HandleTranslationUnit(this=0x0000555555a19ab0, C=0x0000555555a03ac0) at CodeGenAction.cpp:191:7
    frame #13: 0x00007ffff62be484 libdxcompiler.so`clang::ParseAST(S=0x0000555555a1a790, PrintStats=false, SkipFunctionBodies=false) at ParseAST.cpp:164:13
    frame #14: 0x00007ffff50ff616 libdxcompiler.so`clang::ASTFrontendAction::ExecuteAction(this=0x00007fffffff9f00) at FrontendAction.cpp:554:3
    frame #15: 0x00007ffff4d54c31 libdxcompiler.so`clang::CodeGenAction::ExecuteAction(this=0x00007fffffff9f00) at CodeGenAction.cpp:807:28
    frame #16: 0x00007ffff50ff10c libdxcompiler.so`clang::FrontendAction::Execute(this=0x00007fffffff9f00) at FrontendAction.cpp:455:8
    frame #17: 0x00007ffff43e540a libdxcompiler.so`DxcCompiler::Compile(this=0x00005555559d99e0, pSource=0x00007fffffffc5a0, pArguments=0x00005555559d64f0, argCount=6, pIncludeHandler=0x00005555559d9590, riid=0x00005555557fd658, ppResult=0x00007fffffffba98) at dxcompilerobj.cpp:982:18
    frame #18: 0x00007ffff43d9dbd libdxcompiler.so`hlsl::DxcCompilerAdapter::WrapCompile(this=0x00005555559d9e00, bPreprocess=NO, pSource=0x00005555559d9fd0, pSourceName=L"scratch/clip_test.hlsl", pEntryPoint=L"lib.no::entry", pTargetProfile=L"lib_6_8", pArguments=0x00005555559d9570, argCount=3, pDefines=0x00007fffffffd1e8, defineCount=0, pIncludeHandler=0x00005555559d9590, ppResult=0x00007fffffffcda0, ppDebugBlobName=0x0000000000000000, ppDebugBlob=0x0000000000000000) at dxcompilerobj.cpp:1843:5
    frame #19: 0x00007ffff43db2ac libdxcompiler.so`hlsl::DxcCompilerAdapter::CompileWithDebug(this=0x00005555559d9e00, pSource=0x00005555559d9fd0, pSourceName=L"scratch/clip_test.hlsl", pEntryPoint=L"lib.no::entry", pTargetProfile=L"lib_6_8", pArguments=0x00005555559d9570, argCount=3, pDefines=0x00007fffffffd1e8, defineCount=0, pIncludeHandler=0x00005555559d9590, ppResult=0x00007fffffffcda0, ppDebugBlobName=0x0000000000000000, ppDebugBlob=0x0000000000000000) at dxcompilerobj.cpp:1746:10
    frame #20: 0x00007ffff43df066 libdxcompiler.so`hlsl::DxcCompilerAdapter::Compile(this=0x00005555559d9e00, pSource=0x00005555559d9fd0, pSourceName=L"scratch/clip_test.hlsl", pEntryPoint=L"lib.no::entry", pTargetProfile=L"lib_6_8", pArguments=0x00005555559d9570, argCount=3, pDefines=0x00007fffffffd1e8, defineCount=0, pIncludeHandler=0x00005555559d9590, ppResult=0x00007fffffffcda0) at dxcompileradapter.h:75:12
    frame #21: 0x00005555555c1c17 dxc`DxcContext::Compile(this=0x00007fffffffd048) at dxc.cpp:874:9
    frame #22: 0x00005555555c5db3 dxc`dxc::main(argc=5, argv_=0x00007fffffffdd28) at dxc.cpp:1502:24
    frame #23: 0x00005555555ba732 dxc`main(argc=5, argv_=0x00007fffffffdd28) at dxcmain.cpp:19:10
    frame #24: 0x00007ffff3a29d90 libc.so.6`__libc_start_call_main(main=(dxc`main at dxcmain.cpp:18), argc=5, argv=0x00007fffffffdd28) at libc_start_call_main.h:58:16
    frame #25: 0x00007ffff3a29e40 libc.so.6`__libc_start_main_impl(main=(dxc`main at dxcmain.cpp:18), argc=5, argv=0x00007fffffffdd28, init=<unavailable>, fini=<unavailable>, rtld_fini=<unavailable>, stack_end=0x00007fffffffdd18) at libc-start.c:392:3
    frame #26: 0x00005555555ba645 dxc`_start + 37

Steps to Reproduce
commands I used were
dxc scratch/clip_test.hlsl -T lib_6_8 -enable-16bit-types

Environment

  • DXC version: commit id: e7b78ff9c99c19a6a0c98256db9794e0af4eb59d
  • Host Operating System: Ubuntu 22.04

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the crash with dxc scratch/clip_test.hlsl -T lib_6_8 -enable-16bit-types, then inspect HLSignatureLower.cpp at replaceInputOutputWithIntrinsic and the surrounding signature-lowering flow named in the stack trace. Done means this invalid shader no longer segfaults and instead fails with an error or assertion.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.