AcademySoftwareFoundation / AcademySoftwareFoundation/OpenCue

Frame dependencies are incorrect with an interleaved range and chunking

Closed
#1,448 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
957
Forks
259
Avg merge
3d 6h
Merged PRs (30d)
16

Description

**Describe the bug**
I've discovered a case where the frame dependencies aren't set up correctly and it leads to frames running before they should, normally resulting in file not found errors.
It can be reproduced using basic shell commands.
In our real world scenario, we have a render layer which produces images and a denoise layer which denoises the rendered image and produces a second image. The denoise layer has a chunk size of 3.

**To Reproduce**
```
cmd = 'echo "Frame: %{FRAME}, zframe: %{ZFRAME}, range: %{RANGE}"'
layer1 = Shell("mock_render_layer", command=cmd, env={"CUE_ENV": "show"}, range="1001-1200:10", chunk=1, shell=True)
layer2 = Shell("mock_denoise_layer", command=cmd, env={"CUE_ENV": "show"}, range="1001-1200:10", chunk=3, shell=True)
layer2.depend_on(layer1) # frame on frame
# dispatch an outline with those 2 layers added.
```

**Expected behavior**
After the first round of frames on 10s, the frame dependencies on the mock_denoise_layer are incorrect, as viewed in cuetopia.
The output for denoise frame 1046:
![image](https://github.com/user-attachments/assets/b659e054-2b38-4771-9b71-86629348a64d)
The dependencies as shown in cuetopia:
![image](https://github.com/user-attachments/assets/cbb930a9-62af-4628-b6ad-09a22679655e)

**Second example**
It can be shown better by changing the commands to write and read files:
You'll need to substitute a file path that is accessible by the render hosts.
```
render_cmd = `touch /jobs/pipeline/interleave_test/mock_render.%{FRAME}'
layer1 = Shell("mock_render_layer", command=render_cmd, env={"CUE_ENV": "show"}, range="1001-1200:10", chunk=1, shell=True)

denoise_cmd = 'ls /jobs/pipeline/interleave_test/mock_render.%{FRAME}'
layer2 = Shell("mock_denoise_layer", command=denoise_cmd, env={"CUE_ENV": "show"}, range="1001-1200:10", chunk=3, shell=True)
layer2.depend_on(layer1) # frame on frame
# dispatch an outline with those 2 layers added.
```

This will lead to a lot of denoise frames failing as they attempt to `ls` the file before the render layer has touched it.

**Version Number**
0.21

Please let me know if any more information is needed.

Many thanks,
Michael

Contributor guide

Open the contributing guide

Research direction

Start by running the provided Python reproducer with the two Shell layers, interleaved range 1001-1200:10, and downstream chunk size 3. Trace how frame-on-frame dependencies are generated and inspect the resulting dependencies in cuetopia. Done means denoise frames do not run before their corresponding render frames and the file-based example no longer fails because the input is missing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, shell
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.