Autodesk / Autodesk/maya-usd

Hard crashes while running 'ogsRender' command with proxyShapes

Open
#3,415 5 comments 0 reactions 1 assignee Claimed by @santosg87 View on GitHub
bug
Dominant language
Wolfram Language
Stars
905
Forks
223
Avg merge
2d 9h
Merged PRs (30d)
17

Description

**Describe the bug**

We get hard crashes in some scenarios while running ogsRender command. We noticed it happens when the proxyShape's stage contains instances. It also manifests when working with display layers that have USD memberships. I am joining two reproductions even if the underlying cause is likely the same.

**Steps to reproduce**

##### ogsRender and instances
1. Create a mayaUsd proxyShape with some instances

```python
import maya.cmds
import mayaUsd.ufe
import mayaUsd_createStageWithNewLayer
maya.cmds.loadPlugin("mayaUsdPlugin", quiet=True)

proxyShape = mayaUsd_createStageWithNewLayer.createStageWithNewLayer()
mayaUsd.ufe.getStage(proxyShape).GetRootLayer().ImportFromString('''#sdf 1
def Scope "Proto"
{
def Sphere "Geo"
{
}
}
def "Inst_1" (
instanceable = True
references =
)
{
}
def "Inst_2" (
instanceable = True
references =
)
{
}
''')
```
2. Execute a non-viewport VP2 render, e.g. by calling ogsRender
```python
maya.cmds.ogsRender()
```

3. It should crash, we get this stack trace:
```
Autodesk::Maya::OpenMaya20230000::MHWRender::MFrameContext::applyViewTransform(Autodesk::Maya::OpenMaya20230000::MColor const&, Autodesk::Maya::OpenMaya20230000::MHWRender::MFrameContext::Direction) const
pxrInternal_v0_22__pxrReserved__::ProxyRenderDelegate::_GetDisplayColor(std::pair >&, char const*, bool, Autodesk::Maya::OpenMaya20230000::MColor const&)
pxrInternal_v0_22__pxrReserved__::ProxyRenderDelegate::GetTemplateColor(bool)
pxrInternal_v0_22__pxrReserved__::HdVP2Mesh::_UpdateDrawItem(pxrInternal_v0_22__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::HdVP2DrawItem*, pxrInternal_v0_22__pxrReserved__::HdVP2DrawItem::RenderItemData&, pxrInternal_v0_22__pxrReserved__::HdMeshReprDesc const&, pxrInternal_v0_22__pxrReserved__::TfToken const&)
pxrInternal_v0_22__pxrReserved__::HdVP2Mesh::_UpdateRepr(pxrInternal_v0_22__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::TfToken const&)
pxrInternal_v0_22__pxrReserved__::HdVP2Mesh::Sync(pxrInternal_v0_22__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::HdRenderParam*, unsigned int*, pxrInternal_v0_22__pxrReserved__::TfToken const&)
...
```

##### ogsRender and displayLayer

1. Create a mayaUsd proxyShape with a simple stage

```python
import maya.cmds
import pxr.UsdGeom
import mayaUsd.ufe
import mayaUsd_createStageWithNewLayer
maya.cmds.loadPlugin("mayaUsdPlugin", quiet=True)

proxyShape = mayaUsd_createStageWithNewLayer.createStageWithNewLayer()
pxr.UsdGeom.Sphere.Define(mayaUsd.ufe.getStage(proxyShape), "/Sphere")
``````

2. Create a displayLayer containing `/Sphere` with `template` display style
```python
maya.cmds.select(f"{proxyShape},/Sphere")
displayLayer = maya.cmds.createDisplayLayer()
maya.cmds.setAttr(f"{displayLayer}.displayType", 1)
```

3. Call ogsRender
```python
maya.cmds.ogsRender()
```

4. It should crash, we get a similar stack trace:
```
Autodesk::Maya::OpenMaya20230000::MHWRender::MFrameContext::applyViewTransform(Autodesk::Maya::OpenMaya20230000::MColor const&, Autodesk::Maya::OpenMaya20230000::MHWRender::MFrameContext::Direction) const
pxrInternal_v0_22__pxrReserved__::ProxyRenderDelegate::_GetDisplayColor(std::pair >&, char const*, bool, Autodesk::Maya::OpenMaya20230000::MColor const&)
pxrInternal_v0_22__pxrReserved__::ProxyRenderDelegate::GetTemplateColor(bool)
pxrInternal_v0_22__pxrReserved__::MayaUsdRPrim::_GetHighlightColor(pxrInternal_v0_22__pxrReserved__::TfToken const&, pxrInternal_v0_22__pxrReserved__::HdVP2SelectionStatus)
pxrInternal_v0_22__pxrReserved__::MayaUsdRPrim::_GetHighlightColor(pxrInternal_v0_22__pxrReserved__::TfToken const&)
pxrInternal_v0_22__pxrReserved__::HdVP2Mesh::_UpdateDrawItem(pxrInternal_v0_22__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::HdVP2DrawItem*, pxrInternal_v0_22__pxrReserved__::HdVP2DrawItem::RenderItemData&, pxrInternal_v0_22__pxrReserved__::HdMeshReprDesc const&, pxrInternal_v0_22__pxrReserved__::TfToken const&)
pxrInternal_v0_22__pxrReserved__::HdVP2Mesh::_UpdateRepr(pxrInternal_v0_22__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::TfToken const&)
pxrInternal_v0_22__pxrReserved__::HdVP2Mesh::Sync(pxrInternal_v0_22__pxrReserved__::HdSceneDelegate*, pxrInternal_v0_22__pxrReserved__::HdRenderParam*, unsigned int*, pxrInternal_v0_22__pxrReserved__::TfToken const&)
...
```

**Specs (if applicable):**

- CentOS 7.8
- Maya 2023.3
- maya-usd 0.25
- USD 22.11

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.