lllyasviel / lllyasviel/ControlNet
OpenPose in batch mode only generates for the first combination when used with X/Y/Z plot
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
When using OpenPose in batch mode with XYZ plot, it only generates "n" images (where n is the number of poses in the openpose batch folder) based on the first combination in XYZ plot.
Using a single pose is fine with XYZ Prompt but it seems like batch mode knows it had X photos in the folder, and stops generation after X images.
Logs for my test generation:
- XYZ prompt had 3x3 (3 characters, and 3 seeds)
- 7 poses in the openpose folder
- 30 steps sampler +25 highresfix per image
- ControlNet v1.1.157
- control_v11p_sd15_openpose [cab727d4]
X/Y/Z plot will create 9 images on 1 3x3 grid. (Total steps to process: 495)
controlnet batch mode
Loading model from cache: control_v11p_sd15_openpose [cab727d4]
Loading preprocessor: none
preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00, 3.83it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:42<00:00, 1.69s/it]
Loading model from cache: control_v11p_sd15_openpose [cab727d4] | 55/3465 [00:51<1:36:03, 1.69s/it]
Loading preprocessor: none
preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00, 3.90it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:42<00:00, 1.69s/it]
Loading model from cache: control_v11p_sd15_openpose [cab727d4] | 110/3465 [01:43<1:34:35, 1.69s/it]
Loading preprocessor: none
preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00, 3.96it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:42<00:00, 1.70s/it]
Loading model from cache: control_v11p_sd15_openpose [cab727d4] | 165/3465 [02:35<1:33:16, 1.70s/it]
Loading preprocessor: none
preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00, 3.94it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:42<00:00, 1.70s/it]
Loading model from cache: control_v11p_sd15_openpose [cab727d4] | 220/3465 [03:28<1:31:43, 1.70s/it]
Loading preprocessor: none
preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00, 3.93it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:42<00:00, 1.70s/it]
Loading model from cache: control_v11p_sd15_openpose [cab727d4] | 275/3465 [04:20<1:30:18, 1.70s/it]
Loading preprocessor: none
preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00, 3.94it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:42<00:00, 1.70s/it]
Loading model from cache: control_v11p_sd15_openpose [cab727d4] | 330/3465 [05:12<1:28:47, 1.70s/it]
Loading preprocessor: none
preprocessor resolution = 512
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:07<00:00, 3.92it/s]
100%|██████████████████████████████████████████████████████████████████████████████████| 25/25 [00:42<00:00, 1.70s/it]
Total progress: 11%|███████ | 385/3465 [06:07<49:00, 1.05it/s]
Total progress: 11%|██████▊ | 385/3465 [06:07<1:27:15, 1.70s/it]
The logs then stops and the "generate" button is back active in A1111.
At the start of generation, SD knows XYZ will create 9 images.
SD also know it should create 63 (9 images x 7 poses) images altogether based on total steps of 3465 (63 x 55 steps per image).
However after generating 7 images, it stops and the "generate" button is active again on A1111.
It looks like controlnet / openpose is only picking the first combination in the XYZ plot and then generating the images for the number of poses in the folder, and then terminates. I have confirmed the first combination is used by setting 987654321, 888888888, 123456789 as the seeds in XYZ plot and 7 resulting images were all generated with 987654321 as the seed.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the reported setup with XYZ Plot using 3 characters, 3 seeds, and 7 OpenPose batch images, then compare the generated count and seed values with the expected 63 images. Trace the XYZ Plot and OpenPose batch interaction from the relevant ControlNet integration entry points; done means every XYZ combination is generated for each pose rather than only the first combination.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100