Compiler does not use the custom include handler when compiling with `-Zi`

Hi!

I have been testing out ways to make debug information available in captures etc. and I have found an issue where enabling source-level debug information causes a crash in the SPIR-V compiler if any files included in the shader are not available relative to the current working directory.

This in itself is maybe not that surprising, however, in my application I have implemented a custom include handler that is designed to work similar to C++, in that you can import shader from a "global" shader directory (similar to adding-I <path-to-app>/Shaders to the command-line in C++), or by including files that are located relative to the compiled shader-file itself (similar to including relative files in C++)

Example

Say I have the following file/directory structure:

<path-to-app>/
  |-Shaders/
    |-Passthrough.hlsl
    |-Includes/
       |-Uniforms.hlsl

And Passthrough.hlsl includes Uniform.hlsl like this:

#include "Includes/Uniforms.hlsl" 
...

If the current directory when invoking pCompiler->Compile() is <path-to-app>/Shaders, then the SPIR-V compiler will succeed as the included file is able to be opened using the relative path.

However, if the current directory is <path-to-app> instead, then the shader compiler will fail, even though the custom include handler supports opening files relative to the current shader file. This seems to happen because the include handler is not used in some parts of the compilation process.

Note that this only seems to happen when debug information is requested (either via -Zi or via -fspv-debug=source). Otherwise, this particular code path in the SPIR-V compiler is never called.

Callstack

KernelBase.dll!00007ffd708d039c() (Unknown Source:0)
vcruntime140d.dll!00007ffd5a4ab760() (Unknown Source:0)
dxcompiler.dll!hlsl::ReadBinaryFile(IMalloc * pMalloc, const wchar_t * pFileName, void * * ppData, unsigned long * pDataSize) Line 103 (c:\Programming\Fractalite\third_party\dxc\dxc\lib\DxcSupport\FileIOHelper.cpp:103)
dxcompiler.dll!hlsl::DxcCreateBlobFromFile(IMalloc * pMalloc, const wchar_t * pFileName, unsigned int * pCodePage, IDxcBlobEncoding * * ppBlobEncoding) Line 829 (c:\Programming\Fractalite\third_party\dxc\dxc\lib\DxcSupport\FileIOHelper.cpp:829)
dxcompiler.dll!hlsl::DxcCreateBlobFromFile(const wchar_t * pFileName, unsigned int * pCodePage, IDxcBlobEncoding * * ppBlobEncoding) Line 846 (c:\Programming\Fractalite\third_party\dxc\dxc\lib\DxcSupport\FileIOHelper.cpp:846)
dxcompiler.dll!DxcUtils::LoadFile(const wchar_t * pFileName, unsigned int * pCodePage, IDxcBlobEncoding * * pBlobEncoding) Line 283 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\tools\dxcompiler\dxclibrary.cpp:283)
dxcompiler.dll!DxcLibrary::CreateBlobFromFile(const wchar_t * pFileName, unsigned int * pCodePage, IDxcBlobEncoding * * pBlobEncoding) Line 533 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\tools\dxcompiler\dxclibrary.cpp:533)
dxcompiler.dll!clang::spirv::`anonymous namespace'::ReadSourceCode(llvm::StringRef filePath) Line 144 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\SPIRV\EmitVisitor.cpp:144)
dxcompiler.dll!clang::spirv::`anonymous namespace'::getChoppedSourceCode(clang::spirv::SpirvSource * inst) Line 159 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\SPIRV\EmitVisitor.cpp:159)
dxcompiler.dll!clang::spirv::EmitVisitor::visit(clang::spirv::SpirvSource * inst) Line 666 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\SPIRV\EmitVisitor.cpp:666)
dxcompiler.dll!clang::spirv::SpirvSource::invokeVisitor(clang::spirv::Visitor * v) Line 34 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\SPIRV\SpirvInstruction.cpp:34)
dxcompiler.dll!clang::spirv::SpirvModule::invokeVisitor(clang::spirv::Visitor * visitor, bool reverseOrder) Line 190 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\SPIRV\SpirvModule.cpp:190)
dxcompiler.dll!clang::spirv::SpirvBuilder::takeModule() Line 1700 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\SPIRV\SpirvBuilder.cpp:1700)
dxcompiler.dll!clang::spirv::SpirvEmitter::HandleTranslationUnit(clang::ASTContext & context) Line 782 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\SPIRV\SpirvEmitter.cpp:782)
dxcompiler.dll!clang::ParseAST(clang::Sema & S, bool PrintStats, bool SkipFunctionBodies) Line 162 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\Parse\ParseAST.cpp:162)
dxcompiler.dll!clang::ASTFrontendAction::ExecuteAction() Line 556 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\Frontend\FrontendAction.cpp:556)
dxcompiler.dll!clang::FrontendAction::Execute() Line 468 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\lib\Frontend\FrontendAction.cpp:468)
dxcompiler.dll!DxcCompiler::Compile(const DxcBuffer * pSource, const wchar_t * * pArguments, unsigned int argCount, IDxcIncludeHandler * pIncludeHandler, const _GUID & riid, void * * ppResult) Line 1038 (c:\Programming\Fractalite\third_party\dxc\dxc\tools\clang\tools\dxcompiler\dxcompilerobj.cpp:1038)

Start with clang/lib/SPIRV/EmitVisitor.cpp, especially ReadSourceCode and getChoppedSourceCode, then trace how DxcCompiler::Compile passes the custom include handler. Reproduce with -Zi or -fspv-debug=source from a different working directory using the included shader layout. Done means compilation no longer bypasses the handler or crashes when included files are unavailable relative to the working directory.