lllyasviel / lllyasviel/ControlNet
[IDEA PROPOSAL] Using blender to output a 2D file + 3D metadata of each pixel
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.1k
- Forks
- 3k
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I'm not a developer, but I wonder if it could be possible to use blender, to generate using their render and camera system, not a .png file, but a 2D texture + metadata on each pixel, basically each pixel having information about the depth and stuff like what object in the 3D scene, belongs.
And then use said metadata + color information of each pixel as aditional input for a difussion model.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. First determine whether ControlNet has an existing Blender integration or rendering-data pipeline, then define the required per-pixel depth and object metadata format and how it would be consumed by the diffusion model; done means an agreed, implementable design.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- blender, machine-learning, python
- Domain
- computer-graphics, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100