AcademySoftwareFoundation / AcademySoftwareFoundation/OpenShadingLanguage
[BUG] Assert in llvm_util.cpp validate_struct_data_layout using batched mode and LLVM 21.1.x
- Dominant language
- C++
- Stars
- 2.3k
- Forks
- 414
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 10
Description
Note: The prompts below are just to help you formulate a good and actionable
request. Feel free to deviate from this format or to delete this text and
write free-form if you prefer.
**Describe the bug**
When using OSL in batched mode eg. in GafferOSL's OSLObject, an assertion happens in `validate_struct_data_layout`:
https://github.com/AcademySoftwareFoundation/OpenShadingLanguage/blob/b3c601f291a9fedf4c43f311fe7224c3b2d073ea/src/liboslexec/llvm_util.cpp#L1797
I saw the `OSL_DEV` definition is one mechanism to print out the values and see where it differs which I tried, however I ran into an issue where TextureOpt's `envlayout` is now private so that codepath doesn't quite work anymore without modification (and risk not getting the right information to debug the original issue...)
**OSL version and dependencies**
* OSL branch/version: tagged 1.14.8.0 and 1.14.10.0 tested
* OS: Windows 11
* C++ compiler: MSVC 14.44 (v143 toolkit with `-D_DISABLE_CONSTEXPR_MUTEX_CONSTRUCTOR`)
* LLVM version: 21.1.3 and 21.1.8 tested (not an issue with 19.1.7)
* OIIO version: 2.5.18.0
* Architecture: x86_64 Ryzen 7 6800M CPU (Zen3) so most likely using AVX2 in batched mode.
**To Reproduce**
Steps to reproduce the behavior:
1. On Windows it is the combination of OIIO 2.5.18.0, LLVM 21.1.3, OSL 1.14.10.0
2. For OSL it is built with: `USE_BATCHED="b8_AVX,b8_AVX2,b8_AVX2_noFMA,b8_AVX512,b8_AVX512_noFMA,b16_AVX512,b16_AVX512_noFMA"` (default for Gaffer builds on Linux, not on Windows yet)
3. Create a basic OSLObject or something that uses batched mode.
4. Assert! In `validate_struct_data_layout` call of `liboslexec/llvm_util.cpp`
**Evidence**
```
C:\dev\rpks\osl\1.14.10.0\build\osl\src\liboslexec\llvm_util.cpp:1797: void __cdecl OSL::v1_14::pvt::LLVM_Util::validate_struct_data_layout(class llvm::Type *,const class std::vector > &): Assertion 'expected_offset_by_index[index] == actual_offset' failed.
```
```
int i;
getattribute( "shading:index", i );
output1 = 0.1 * hashnoise( i );
```
**IF YOU ALREADY HAVE A CODE FIX:** Not yet but this is a fairly obscure combination mix that I don't expect someone else to reproduce easily, so I'm willing to understand things better so I can help make a fix. I can also check Linux later to see if it happens there. The big suspect is an LLVM change that has been made between 19 and 21 that's not being taken into account here somewhere.
Contributor guide
Research direction
Start in src/liboslexec/llvm_util.cpp at validate_struct_data_layout and reproduce the assertion with batched mode, OIIO 2.5.18.0, LLVM 21.1.x, and the reported shader. Compare the struct layout behavior with LLVM 19.1.7 and inspect the expected versus actual offsets; done means the reported configuration no longer asserts while layout validation remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100