facebookresearch / facebookresearch/detectron2
IUV generation
- Dominant language
- Python
- Stars
- 34.7k
- Forks
- 7.9k
- PR merge metrics
- No merged PRs in 30d
Description
hi all,
I'm trying to use the detectrone2 - densepose to produce 3 channel IUV image.
My procedure description:
used build line (from Detectron2 install page):
```
python -m pip install detectron2 -f \
https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.8/index.html
```
step 1: Create .pkl file using the following cmd line:
`python apply_net.py dump configs/densepose_rcnn_R_50_FPN_s1x.yaml https://dl.fbaipublicfiles.com/densepose/densepose_rcnn_R_50_FPN_s1x/165712039/model_final_162be9.pkl "/home/haiho/Dropbox/vid2vid_folder/pose_datase/train/frame0.jpg" --output out.pkl -v`
step 2: trying to generate IUV image using the following code:
```
from PIL import Image
import pickle
import numpy as np
import sys
sys.path.append("/home/haiho/detectron2/projects/DensePose")
pklpath = '/home/haiho/Dropbox/vid2vid_folder/pose_datase/train/densepose-pkl/frame0.pkl'
with open(pklpath, "rb") as hFile:
dp_frame = pickle.load(hFile)[0]
if len(dp_frame['scores']) == 0:
print(f'{pklpath} does not contain persons')
exit(0)
instance_id = 0
bbox = np.array(dp_frame['pred_boxes_XYXY'][instance_id])
result = dp_frame['pred_densepose'][instance_id]
uv = np.array(result.uv.detach().cpu()*255, dtype=np.uint8)
labels = np.array(result.labels.detach().cpu(),dtype=np.uint8)
c,h,w = uv.shape
x1, y1, x2, y2 = round(bbox[0]), round(bbox[1]), round(bbox[0]+w), round(bbox[1]+h)
canvas_size = (1080, 1920, 3)
canvas = np.zeros(canvas_size, dtype=np.uint8)
iuv = np.stack((uv[1,:,:], uv[0,:,:], labels))
iuv = np.transpose(iuv, (1,2,0))
canvas[y1:y2, x1:x2, :] = iuv
Image.fromarray(canvas).save('output1.jpg')
```
expected results:

actual results:

Contributor guide
Research direction
Start by reproducing the command in apply_net.py with the listed DensePose configuration and model, then inspect the generated out.pkl fields used by the provided conversion script. Compare the saved output1.jpg with the expected IUV image; done means identifying why the documented procedure produces the differing result and recording a reproducible correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100