facebookresearch / facebookresearch/pytorch3d

Incorporate Normal Maps into the renderer

Open
#642 4 comments 6 reactions 1 assignee Claimed by @nikhilaravi View on GitHub
enhancement how to
Dominant language
Python
Stars
10k
Forks
1.5k
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature
Incorporate normal mapping into the rendering pipeline.
A normal map is an image using which the mesh normals are manipulated before rendering.

## Motivation
[Normal mapping](https://en.wikipedia.org/wiki/Normal_mapping) (or bump mapping) is a useful technique for enhancing the appearance and details of a low-polygon mesh.

Furthermore, normal maps are a part of many recent Computer Graphics research works.
Normal maps are created using a generative model to add details to a mesh [Link](https://arxiv.org/abs/1808.03417).
It is used in inverse rendering techniques which estimate the normal maps from images [Link](https://arxiv.org/pdf/1811.12328.pdf).

## Pitch
Incorporate normal maps into the rendering pipeline.
Since a normal map is essentially an image, its usage can be incorporated into the current Textures object, an example is as follows:
```python
normal_map_img = torch.from_numpy(cv2.imread("normal_map.png"))
texture = Textures(verts_uvs=verts_uvs, faces_uvs=faces_uvs, normal_map=normal_map_img)
mesh.textures = texture
```
The actual implementation can be done in the Textures class.

I can also implement this feature if there are no plans on incorporating it in the near future.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.