facebookresearch / facebookresearch/detectron2

DensePose_CSE KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>"

Open
#4,525 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## DensePose_CSE KeyError: "Could not find converter from to "
1. create a virtual environment and install the required libraries
```
conda create -n densepose python=3.7
conda activate densepose
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
pip install git+https://github.com/facebookresearch/detectron2@main#subdirectory=projects/DensePose
```
2. try to inference an image with a pre-trained human CSE model, following the tutorial of https://github.com/facebookresearch/detectron2/blob/main/projects/DensePose/doc/GETTING_STARTED.md
```
cd detectron2/projects/DensePose
python apply_net.py show configs/cse/densepose_rcnn_R_50_FPN_DL_soft_s1x.yaml model_final_9199f5.pkl image.png dp_contour,bbox --output image_densepose_contour.png
```
**3. KeyError was reported**
```
The checkpoint state_dict contains keys that are not used by the model:
pixel_mean
pixel_std
~/anaconda3/envs/densepose/lib/python3.7/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/TensorShape.cpp:2894.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
Traceback (most recent call last):
File "apply_net.py", line 354, in
main()
File "apply_net.py", line 350, in main
args.func(args)
File "apply_net.py", line 106, in execute
cls.execute_on_outputs(context, {"file_name": file_name, "image": img}, outputs)
File "apply_net.py", line 284, in execute_on_outputs
data = extractor(outputs)
File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 152, in __call__
data = extractor(instances, select)
File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 102, in __call__
results = [converter.convert(dpout[i], boxes_xyxy[[i]]) for i in range(len(dpout))]
File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 102, in
results = [converter.convert(dpout[i], boxes_xyxy[[i]]) for i in range(len(dpout))]
File "~/Projects/detectron2/projects/DensePose/densepose/converters/to_chart_result.py", line 65, in convert
predictor_outputs, boxes, *args, **kwargs
File "~/Projects/detectron2/projects/DensePose/densepose/converters/base.py", line 83, in convert
raise KeyError(f"Could not find converter from {instance_type} to {output_type_str}")
KeyError: "Could not find converter from to "
```
## Code Checking
I found that, in the **detectron2/projects/DensePose/densepose/converters/base.py**, **_lookup_converter** function returns None because there are only keys of **** in **cls.registry**, how can I modify this file to make the return result correct?
```
# the key-values in cls.registry using print function to output
{: }
{: }
```
## Environment:
```
---------------------- --------------------------------------------------------------------------------------------------------
sys.platform linux
Python 3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21) [GCC 9.4.0]
numpy 1.21.5
detectron2 0.6 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/detectron2
Compiler GCC 5.5
CUDA compiler CUDA 11.3
detectron2 arch flags 7.5
DETECTRON2_ENV_MODULE
PyTorch 1.12.1 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/torch
PyTorch debug build False
GPU available Yes
GPU 0 NVIDIA GeForce GTX 1660 (arch=7.5)
Driver version 470.86
CUDA_HOME /usr/local/cuda-11.3
Pillow 9.2.0
torchvision 0.13.1 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/torchvision
torchvision arch flags 3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
fvcore 0.1.5.post20220512
iopath 0.1.9
cv2 4.6.0
---------------------- --------------------------------------------------------------------------------------------------------
PyTorch built with:
- GCC 9.3
- C++ Version: 201402
- Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
- Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
- OpenMP 201511 (a.k.a. OpenMP 4.5)
- LAPACK is enabled (usually provided by MKL)
- NNPACK is enabled
- CPU capability usage: AVX2
- CUDA Runtime 11.3
- NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
- CuDNN 8.3.2 (built against CUDA 11.5)
- Magma 2.5.2
- Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,
```

Contributor guide

Open the contributing guide

Research direction

Start with projects/DensePose/densepose/converters/base.py and projects/DensePose/densepose/converters/to_chart_result.py, then trace the call from apply_net.py through densepose/vis/extractor.py. Reproduce the reported command and inspect the registered converter and predictor output types. Done means the pretrained CSE inference completes without the reported KeyError and produces the requested output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.