facebookresearch / facebookresearch/pytorch3d

Seam when rendering triangle close up

Open
#1,209 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10k
Forks
1.5k
PR merge metrics
No merged PRs in 30d

Description

If you do not know the root cause of the problem / bug, and wish someone to help you, please
post according to this template:

## 🐛 Bugs / Unexpected behaviors
I am trying to render a simple quad created by two triangles, but there is a seam when rendered from a close distance.

## Instructions To Reproduce the Issue:

Please include the following (depending on what the issue is):

1. Any changes you made (`git diff`) or code you wrote
Minimal working example based on the tutorial
```
import os
import sys
import torch
import pytorch3d
import os
import torch
import matplotlib.pyplot as plt
import math

from pytorch3d.io import load_objs_as_meshes, load_obj

# Data structures and functions for rendering
from pytorch3d.structures import Meshes
from pytorch3d.vis.plotly_vis import AxisArgs, plot_batch_individually, plot_scene
from pytorch3d.vis.texture_vis import texturesuv_image_matplotlib
from pytorch3d.renderer import (
look_at_view_transform,
FoVPerspectiveCameras,
PointLights,
DirectionalLights,
Materials,
RasterizationSettings,
MeshRenderer,
MeshRasterizer,
SoftPhongShader,
TexturesUV,
TexturesVertex
)

# add path for demo utils functions
import sys
import os
sys.path.append(os.path.abspath(''))

# Setup
if torch.cuda.is_available():
device = torch.device("cuda:0")
torch.cuda.set_device(device)
else:
device = torch.device("cpu")

obj_filename = 'plane.obj'

mesh = load_objs_as_meshes([obj_filename], device=device)
fov = 45
half_fov_rad = (fov / 2) / 180.0 * math.pi
height = 0.5 / math.tan(half_fov_rad)
R, T = look_at_view_transform(height, 0, 0)
cameras = FoVPerspectiveCameras(device=device, R=R, T=T, fov=fov)

raster_settings = RasterizationSettings(
image_size=512,
blur_radius=0.0,
faces_per_pixel=1,
)

# Place a point light in front of the object. As mentioned above, the front of the cow is facing the
# -z direction.
lights = PointLights(device=device, location=[[0.0, 0.0, 1.0]])

# Create a Phong renderer by composing a rasterizer and a shader. The textured Phong shader will
# interpolate the texture uv coordinates for each vertex, sample from a texture image and
# apply the Phong lighting model
renderer = MeshRenderer(
rasterizer=MeshRasterizer(
cameras=cameras,
raster_settings=raster_settings
),
shader=SoftPhongShader(
device=device,
cameras=cameras,
lights=lights
)
)

images = renderer(mesh)
# plt.figure(figsize=(10, 10))
plt.imshow(images[0, ..., :3].cpu().numpy())
plt.show()
```

plane.obj and plane.mtl
```
mtllib plane.mtl
o Cube
v 0.5 0.5 0.0
v -0.5 0.5 0.0
v 0.5 -0.5 0.0
v -0.5 -0.5 0.0
vt 0.0 1.0
vt 1.0 1.0
vt 0.0 0.0
vt 1.0 0.0
usemtl material_1
s off
f 4/3/1 1/2/1 2/1/1
f 4/3/1 3/4/1 1/2/1
```
```
newmtl material_1
map_Kd banana.jpg

# Test colors

Ka 1.000 1.000 1.000 # white
Kd 1.000 1.000 1.000 # white
Ks 0.000 0.000 0.000 # black
Ns 10.0
```
banana.jpg
![image](https://upload.wikimedia.org/wikipedia/commons/thumb/8/8a/Banana-Single.jpg/2324px-Banana-Single.jpg)
3. The exact command(s) you ran:
python render.py

5. What you observed (including the full logs):
there is a seam between the two triangles.
![image](https://user-images.githubusercontent.com/12501995/170475209-3ce11a4b-2cf3-4100-9f53-c70d1e4c2b03.png)

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied `python render.py` example with `plane.obj`, `plane.mtl`, and the referenced texture to reproduce the seam. Read the PyTorch3D mesh loading and rasterization entry points used by `load_objs_as_meshes`, `MeshRasterizer`, and `SoftPhongShader`; done means identifying the rendering cause and confirming a fix removes the seam between the two triangles.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.