KhronosGroup / KhronosGroup/glslang

Bug: Include Preprocessing reformats include file's source

Open
#2,233 14 comments 0 reactions 0 assignees View on GitHub
PP
Dominant language
C++
Stars
3.6k
Forks
989
Avg merge
1d 2h
Merged PRs (30d)
31

Description

From examining `glslang/MachineIndependent/ShaderLang.cpp` it seems apparent that the entire include file is parsed (obviously so the preprocessor directives can be honoured), **but then the actual source produced from all the headers is de-parsed into a string of formatted tokens**

This is a problem for me as I outline in this shaderc issue:
https://github.com/google/shaderc/issues/1069

Long story short :
> I have a peculiar method of disabling preprocessor directives and checks for GL_ extension defines until after the #includes have been collected. I replace all preprocessor directives except for #version and #include with _this_is_a_hash and some programmatically generated guards against multiple inclusion.

This is working for me quite well, but the only place where it breaks are:

- `#defines` that have a () after them, such as `_this_is_a_hashdefine ONE_MORE_THAN_A (A+1)`
- preprocessor function macros such, ex. `_this_is_a_hashdefine FUNC(X) (X*X)`

The cause of all this is because your `PreprocessDeferred` re-glues together the headers from tokens and the `(` and `)` symbols are on some sort of a list of tokens that are explicitly stated not to need a space before them.

```cpp
// Output a space in between tokens, but not at the start of a line,
// and also not around special tokens. This helps with readability
// and consistency.
if (!isNewString && !isNewLine && lastToken != EndOfInput &&
(unNeededSpaceTokens.find((char)token) == std::string::npos) &&
(unNeededSpaceTokens.find((char)lastToken) == std::string::npos) &&
(noSpaceBeforeTokens.find((char)token) == std::string::npos)) {
outputBuffer += ' ';
}
```

Is there any way to fix this behaviour?
could I remove `(` from `noSpaceBeforeTokens` without affecting somethng else adversely?

Contributor guide

Open the contributing guide

Research direction

Start in glslang/MachineIndependent/ShaderLang.cpp, focusing on PreprocessDeferred and the token-spacing logic involving noSpaceBeforeTokens. Reproduce the include cases with function-like macros and parenthesized macro bodies, then verify that preprocessing preserves the required separation without changing other formatted output.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.