Different out/inout behaviour during fragment shader execution
- Dominant language
- C++
- Stars
- 2.2k
- Forks
- 445
- Avg merge
- 11h 6m
- Merged PRs (30d)
- 6
Description
By suggestion of @nvining-work (https://github.com/nvining-work) I am filing an issue here. He also wants to say hi to @dneto0.
I ran into the current situation, i was building a BVH raytracer using vulkan on Ubuntu, compiling with shaderc.
In this shader I declared the function:
```cpp
int best_box;
float triangle_distance;
int IntersectBVH(out Ray ray, int id)
{
best_box = 0;
triangle_distance = 1e30f;
uint stack[256];
int sp = 0;
stack[sp] = id;
sp++;
int safety = 0;
while(sp > 0)
{
sp--;
uint b_id = stack[sp];
bool collides = IntersectAABB(ray, nodes[b_id].aabbMin, nodes[b_id].aabbMax);
if(safety++ > 200) return collides ? int(b_id) : -1;
if(nodes[b_id].triCount > 0 && collides)
{
// return int(b_id);
for (uint i = 0; i < nodes[b_id].triCount; i++ )
{
const uint triangle_id = tri_idxs[nodes[b_id].leftFirst + i];
IntersectTri(
ray,
triangles[triangle_id + 0].position.xyz,
triangles[triangle_id + 1].position.xyz,
triangles[triangle_id + 2].position.xyz);
if(ray.t < triangle_distance)
{
triangle_distance = ray.t;
best_box = int(b_id);
}
}
}
else if (collides)
{
stack[sp++] = nodes[b_id].leftFirst;
stack[sp++] = nodes[b_id].leftFirst + 1;
}
}
return -1;
}
```
With this signature the program behaved strangely, depending on things like wheter or not I declared a variable or I called seemingly inconsequential functions, the program would sometimes work, sometimes it would output a blank screen.
Upon modifying the signature to `int IntersectBVH(inout Ray ray, int id)` all seemingly illogical behaviour stopped and my program followed my expectations.
The call to this function was:
```cpp
Ray ray = Ray(ray_pos, normalize(r), 1e30f);
int val = IntersectBVH(ray, 0);
```
I have attached the broken fragment shader as a reference. In this file changing the out parameter qualifiers as specified above should result in different behaviours. I can provide more information if required.
[broken.txt](https://github.com/google/shaderc/files/8588002/broken.txt)
Contributor guide
Assessment
This issue has not been assessed yet.