google / google/shaderc

Different out/inout behaviour during fragment shader execution

Open
#1,245 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
2.2k
Forks
445
Avg merge
11h 6m
Merged PRs (30d)
6

Description

By suggestion of @nvining-work (https://github.com/nvining-work) I am filing an issue here. He also wants to say hi to @dneto0.

I ran into the current situation, i was building a BVH raytracer using vulkan on Ubuntu, compiling with shaderc.

In this shader I declared the function:

```cpp

int best_box;
float triangle_distance;
int IntersectBVH(out Ray ray, int id)
{
best_box = 0;
triangle_distance = 1e30f;

uint stack[256];
int sp = 0;

stack[sp] = id;
sp++;

int safety = 0;
while(sp > 0)
{
sp--;
uint b_id = stack[sp];

bool collides = IntersectAABB(ray, nodes[b_id].aabbMin, nodes[b_id].aabbMax);
if(safety++ > 200) return collides ? int(b_id) : -1;
if(nodes[b_id].triCount > 0 && collides)
{
// return int(b_id);
for (uint i = 0; i < nodes[b_id].triCount; i++ )
{
const uint triangle_id = tri_idxs[nodes[b_id].leftFirst + i];
IntersectTri(
ray,
triangles[triangle_id + 0].position.xyz,
triangles[triangle_id + 1].position.xyz,
triangles[triangle_id + 2].position.xyz);

if(ray.t < triangle_distance)
{
triangle_distance = ray.t;
best_box = int(b_id);
}
}

}
else if (collides)
{
stack[sp++] = nodes[b_id].leftFirst;
stack[sp++] = nodes[b_id].leftFirst + 1;
}
}

return -1;
}
```

With this signature the program behaved strangely, depending on things like wheter or not I declared a variable or I called seemingly inconsequential functions, the program would sometimes work, sometimes it would output a blank screen.

Upon modifying the signature to `int IntersectBVH(inout Ray ray, int id)` all seemingly illogical behaviour stopped and my program followed my expectations.

The call to this function was:

```cpp
Ray ray = Ray(ray_pos, normalize(r), 1e30f);
int val = IntersectBVH(ray, 0);
```

I have attached the broken fragment shader as a reference. In this file changing the out parameter qualifiers as specified above should result in different behaviours. I can provide more information if required.

[broken.txt](https://github.com/google/shaderc/files/8588002/broken.txt)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.