facebookresearch / facebookresearch/pytorch3d
Incorporate Normal Maps into the renderer
- Dominant language
- Python
- Stars
- 10k
- Forks
- 1.5k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
Incorporate normal mapping into the rendering pipeline.
A normal map is an image using which the mesh normals are manipulated before rendering.
## Motivation
[Normal mapping](https://en.wikipedia.org/wiki/Normal_mapping) (or bump mapping) is a useful technique for enhancing the appearance and details of a low-polygon mesh.
Furthermore, normal maps are a part of many recent Computer Graphics research works.
Normal maps are created using a generative model to add details to a mesh [Link](https://arxiv.org/abs/1808.03417).
It is used in inverse rendering techniques which estimate the normal maps from images [Link](https://arxiv.org/pdf/1811.12328.pdf).
## Pitch
Incorporate normal maps into the rendering pipeline.
Since a normal map is essentially an image, its usage can be incorporated into the current Textures object, an example is as follows:
```python
normal_map_img = torch.from_numpy(cv2.imread("normal_map.png"))
texture = Textures(verts_uvs=verts_uvs, faces_uvs=faces_uvs, normal_map=normal_map_img)
mesh.textures = texture
```
The actual implementation can be done in the Textures class.
I can also implement this feature if there are no plans on incorporating it in the near future.
Contributor guide
Assessment
This issue has not been assessed yet.