gazebosim / gazebosim/gz-rendering

INTEGRATION_depth_camera fails with DEPTH_CLAMP disabled

Open
#395 9 comments 0 reactions 0 assignees View on GitHub
bug help wanted
Dominant language
C++
Stars
81
Forks
90
Avg merge
1d 13h
Merged PRs (30d)
10

Description

As @iche033 noticed, [this change in ogre-next](https://github.com/OGRECave/ogre-next/commit/d18a9c991fa955acbff0475bceea54a514b11e0b) caused INTEGRATION_depth_camera to fail.

However the "correct way" is to have that change, and it is currently being workarounded in ign-rendering by temporarily turning on DEPTH_CLAMP manually.

This ticket will track the causes of the bugs and fix them

## Explanation of DEPTH_CLAMP

We need to first explain what DEPTH_CLAMP is. Normally when rendering a triangle whose Z (depth) is outside [near; far] range it gets clipped. i.e. it doesn't appear at all. In normalized depth the value falls outside [0; 1] range.

However when DEPTH_CLAMP is enabled; any value outside is clamped instead of getting clipped. Thus a depth of 1.2 becomes 1.0 and a depth of -0.2 gets clamped to 0 and ends up appearing on screen.

The effect visually (if you'd imagine it) would be something like this:

![01](https://user-images.githubusercontent.com/3395130/132135895-2ff6d523-c6b0-49d5-b4b7-f66865dd16e4.jpg)

As the sphere gets closer to the camera and tries to go through it, its depth gets clamped and covers the screen:

![02](https://user-images.githubusercontent.com/3395130/132135909-097d7d1d-0c1b-4013-a75a-f1275b987933.jpg)

## INTEGRATION_depth_camera places the camera inside the cube

The following snippet:

```cpp
// Check that for a box really close it returns it is not seen
ignition::math::Vector3d boxPositionNear(
unitBoxSize * 0.5 + nearDist * 0.5, 0.0, 0.0);
box->SetLocalPosition(boxPositionNear);
```

Places the cube a bit too close:

![03](https://user-images.githubusercontent.com/3395130/132136031-c75b97f8-8d0a-4ffb-a38d-e53989e96da4.png)

Therefore the cube:

1. If DEPTH_CLAMP is disabled, it won't be rendered due to backface culling (correct behavior). The final colour is red because of the background.
2. If DEPTH_CLAMP is enabled, it will be rendered due to the face behind the camera getting clamped to be on screen. The final colour is blue, which is the colour of the cube.

Later when the postprocess shader is run:

1. With DEPTH_CLAMP disabled; it will see depth is at +INF, colour is already red and does nothing
2. With DEPTH_CLAMP enabled; it will see depth is behind the near plane, thus it sets the depth to -INF to signal this situation; and override the blue colour with the background colour red.

And this is why the test fails. As it expects red colour (due to the postprocess shader override) but it expects depth to be at -inf instead of +inf:

```cpp
// Verify Depth
// box not detected
EXPECT_FLOAT_EQ(minVal, scan[mid]);
EXPECT_FLOAT_EQ(minVal, scan[left]);
EXPECT_FLOAT_EQ(minVal, scan[right]);

// Verify Point Cloud XYZ
{
// all points should be min val
for (unsigned int i = 0; i < depthCamera->ImageHeight(); ++i)
{
unsigned int step = i*depthCamera->ImageWidth()*pointCloudChannelCount;
for (unsigned int j = 0; j < depthCamera->ImageWidth(); ++j)
{
float x = pointCloudData[step + j*pointCloudChannelCount];
float y = pointCloudData[step + j*pointCloudChannelCount + 1];
float z = pointCloudData[step + j*pointCloudChannelCount + 2];
EXPECT_FLOAT_EQ(minVal, x);
EXPECT_FLOAT_EQ(minVal, y);
EXPECT_FLOAT_EQ(minVal, z);
}
}
```

## Possible solutions

I'm not sure what is it we're testing:

- If we're testing the code correctly marks things too close to camera as -inf; then we should move the cube a bit further away from camera so the first face doesn't get clipped
- If we're testing the object is clipped, then we should look for +inf results

## Current workaround

Currently we're workarounding the issue with:

```cpp
void Ogre2DepthCamera::Render()
{
#ifndef _WIN32
glEnable(GL_DEPTH_CLAMP);
#endif
//...
#ifndef _WIN32
glDisable(GL_DEPTH_CLAMP);
#endif
}
```

With the following message:
```cpp
// GL_DEPTH_CLAMP was disabled in later version of ogre2.2
// however our shaders rely on clamped values so enable it for this sensor
```

Actually after a close inspection: no, ign-rendering's shaders do not rely on GL_DEPTH_CLAMP. It is the tests which rely on it because the tests are submitting wrong inputs.

If sensors behave differently when GL_DEPTH_CLAMP is turned off then one of the following is true:

1. The sensor was previously picking up stuff too close to camera that should never have been captured at all (e.g. https://github.com/osrf/subt/issues/888)
2. The scene is incorrect and bad data is supplied (model should turn off backface culling, model or sensor should back away a bit) e.g. the tests.

## Environment
* OS Version: Ubuntu 18.04 LTS
* Source or binary build?
Source, `main`, using an up to date ogre-next library

## Steps to reproduce

1. Run INTEGRATION_depth_camera with latest ogre-next

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.