Unexpected render item behavior for 'casts shadows'
- Dominant language
- Wolfram Language
- Stars
- 905
- Forks
- 223
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 16
Description
**Describe the bug**
Hi,
We received a feature request from production to see if we can toggle "casts shadows" for USD geometries in the same way as native Maya geometries, since proxy render delegate internally creates `MRenderItem`, we were experimenting such possibility by setting `MRenderItem::castsShadows()` , it seems straightforward if we make the changes in two places in [`lib/mayaUsd/render/vp2RenderDelegate/mayaPrimCommon.cpp`](https://github.com/Autodesk/maya-usd/blob/release/v0.30.0/lib/mayaUsd/render/vp2RenderDelegate/mayaPrimCommon.cpp) (see attached diff files for the full changes):
1. [`MayaUsdRPrim::_InitRenderItemCommon()`](https://github.com/Autodesk/maya-usd/blob/release/v0.30.0/lib/mayaUsd/render/vp2RenderDelegate/mayaPrimCommon.cpp#L428): set `MRenderItem::castsShadows()` when initialize the render item;
2. [`MayaUsdRPrim::_SyncSharedData()`](https://github.com/Autodesk/maya-usd/blob/release/v0.30.0/lib/mayaUsd/render/vp2RenderDelegate/mayaPrimCommon.cpp#L841): keep tracking of primvar dirtiness to toggle the "casts shadows" value for corresponding render item.
The "casts shadows" attribute is exposed as an additional primvar named "primvars:visibility:castsShadows" (see attached USD file **CORRECT_casts_shadows_off_two_groups.usda** for the full content), we created a directional light, turned on "all lights" and "shadow" on the viewport, everything seems working as expected:

However, when we started to test production scene, something looks fishy, for example the above example has two groups of planes, the left & right groups, if we have them separated in different USD files, toggling the primvar does not change anything, it remains as initial state (`"castsShadows" == false`):
The left group (see attached USD file **INCORRECT_casts_shadows_off_left_group_only.usda**) - looks incorrect:

The right group (see attached USD file **INCORRECT_casts_shadows_off_right_group_only.usda**) - looks incorrect:

By having further tests, it looks like if foreground plane is smaller than background plane (see attached USD file **CORRECT_casts_shadows_off_fg_smaller_than_bg.usda**) , it **works as expected**:

But if foreground plane is larger than background plane (see attached USD file **INCORRECT_casts_shadows_off_fg_larger_than_bg.usda**) , it **DOES NOT work as expected**:

When debugging, we accidently found something even confusing, if we leave `MayaUsdRPrim::_InitRenderItemCommon()` untouched (which expected to see inconsistent result at init) , keeps the changes in `MayaUsdRPrim::_SyncSharedData()` (which expected to see inconsistent result at the first time when toggling the primvar), and the further toggling works as expected. For example the previous **INCORRECT_casts_shadows_off_fg_larger_than_bg** USD file now behaves as expected:

We can't reproduce the issue if they were in native Maya geometries, so it looks like related to MayaUSD only.
What did we miss? I feel like there is a bug somewhere but not sure where it is.
I attached our changes but I could submit a draft PR if that's easier to review - let me know!
**Steps to reproduce**
Steps to reproduce the behavior:
1. Apply the attached file **casts_shadows_changes.diff.txt** to get the cast shadows reacts on toggling
2. Unzip attached example USD files
3. Load one of the example USD files in Maya
4. Turn on "Usa all lights" and "Shadows" on the viewport
5. Observe the expected / unexpected shadows behaviors
**Expected behavior**
MayaUSD could properly supports "casts shadows".
**Attachments**
- GIFs - see above inline images
- Example USD files:
[casts_shadows_off_example_files.zip](https://github.com/user-attachments/files/17109262/casts_shadows_off_example_files.zip)
- Code changes in diff format:
[casts_shadows_changes.diff.txt](https://github.com/user-attachments/files/17109263/casts_shadows_changes.diff.txt)
**Specs (if applicable):**
- OS & version : CentOS 7.8
- Compiler & version : gcc 9.3
- Maya version: tested against Maya-2023.2.x - 2023.3.3
- Maya USD commit SHA: `release/v0.30.0` branch b3d4a06f5 + custom changes
- Pixar USD commit SHA: official v24.08 / v22.05
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.