lllyasviel / lllyasviel/ControlNet

normal-map-model: unexpected output

Open
#332 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
34.1k
Forks
3k
PR merge metrics
No merged PRs in 30d

Description

Awesome work! :)

One Problem, I tried the normals model and don't get any good results when I provide my own normals map.

Here is what I mean:

1. my own normals map and the output (output completely useless)
![Screenshot from 2023-04-05 22-10-37](https://user-images.githubusercontent.com/105629389/230198828-13ab2ea3-1da6-4ac5-80eb-e202556b7672.png)
![Screenshot from 2023-04-05 22-10-40](https://user-images.githubusercontent.com/105629389/230198840-f8a057cc-cc51-4600-b073-b2bdf8d61d7e.png)

1. my own normals map, additionally preprocessed with the normals preprocessor, and the output (output as expected)
![Screenshot from 2023-04-05 22-11-14](https://user-images.githubusercontent.com/105629389/230199030-b2510ed2-22d0-484d-9f99-cc7ff4eb2d9c.png)
![Screenshot from 2023-04-05 22-11-18](https://user-images.githubusercontent.com/105629389/230199049-47ed21b8-566d-4038-b9d0-de0e194f957c.png)

Why I get completely useless outputs with my own normals map? Is it the wrong color scheme?
Maybe you can share your code on how you convert depth maps to normal maps so that I can replicate the color scheme :)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the user-provided normals map with the same map after the normals preprocessor, using the attached input and output examples as the reproduction case. Inspect the normals-model entry point and preprocessing path to determine what input representation is expected. Done means documenting or correcting the accepted normals-map format so the direct input produces the expected output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.