microsoft / microsoft/DirectXShaderCompiler

Shader compilation fails when file path has utf8 characters

Open
#7,394 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C++
Stars
3.7k
Forks
900
Avg merge
2d 11h
Merged PRs (30d)
44

Description

Description

Shader compilation fails when file path has utf8 characters (Cyrillic, Korean, ... in our case username is part of the path).

We use dxc as a shared library to generate shaders in multi-threaded context.
That's how our CompileShader function is structured:

#include "dxcapi.h"

bool CompileShader( CStringView Source, ... )
{
    CComPtr<IDxcLibrary> Library;
    HRESULT hr = DxcCreateInstance( CLSID_DxcLibrary, IID_PPV_ARGS( &Library ) );
    ...
    CComPtr<IDxcCompiler3> Compiler;
    hr = DxcCreateInstance( CLSID_DxcCompiler, IID_PPV_ARGS( &Compiler ) );
    ...
    DxcBuffer SourceBuffer {};
    SourceBuffer.Ptr = Source.GetData();
    SourceBuffer.Size = Source.GetSize();
    SourceBuffer.Encoding = DXC_CP_ACP;

    CComPtr<IDxcResult> Result;
    hr = Compiler->Compile( &SourceBuffer, Args.AccessData(), Args.GetSize(), nullptr, IID_PPV_ARGS( &Result ) );
    ...
}

Source contains shader source code, I've checked that it has only ASCII characters.
Args is an array that contains <source file path>/<filename>.scache -spirv -fvk-b-shift 0 all -fvk-t-shift 0 all -fvk-s-shift 0 all -fvk-u-shift 0 all -fvk-auto-shift-bindings -fvk-use-dx-layout -HV 2021 -fspv-target-env=vulkan1.1 -T cs_6_0.

Steps to Reproduce
Compile any shader with the CompileShader above.

Actual Behavior

On Windows: hr is E_FAIL and Result contains "error: error reading '/000000005dcd549100000000.scache", even though the file exists in the folder.

On Linux: hr is E_FAIL and Result contains "Internal compiler error:" (See next post for more info)

Environment

  • DXC version 1.8.2407 and 1.8.2502
  • Host Operating System Windows 11 and Ubuntu 24.04.2

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The report centers on CompileShader in the issue body, including dxcapi.h, DxcCreateInstance, and IDxcCompiler3::Compile; start by reproducing it with a UTF-8 path on Windows and Linux. Trace how the source-file argument is passed and identify why the existing file cannot be read; done means shader compilation succeeds without a path-read or internal compiler error.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.