lllyasviel / lllyasviel/ControlNet
Support for 16-Bit Color Depth
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
The depth models for ControlNet can produce some really high quality images that have a phenomenal understanding of relative perspective, especially if used with a dedicated depth map as opposed to a preprocessor.
However, it has one very serious drawback:
From my testing, it seems as though ControlNet only supports 8-bit color depth. What this means is that there are only 255 levels of depth that ControlNet can interpret, and on images where there is a big difference in positioning between scene elements a lot of fine details can be lost. This can be an issue even in relatively small scenes where there is only a maximum depth of around 15 meters between scene elements, but it can especially become a massive issue on larger scenes like landscape shots where there can sometimes be multiple kilometers between elements.
This issue can be somewhat remedied by combining both a depth and normal map, but allowing for 16-bits of color depth, or 65,536 levels of depth, if a user chooses to toggle this option would allow for far more precision in generated images at the cost of performance, which I believe would be a trade-off a lot of people would be willing to take if they could.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not name any files, tests, or entry points, so begin by locating the depth-model input and preprocessing paths in the repository. The work is done when an optional 16-bit depth path is implemented, preserves the requested precision, and has tests or examples covering the new option and its trade-offs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100