DensePose_CSE KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>"

[sword-shadow]

DensePose_CSE KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>"

1. create a virtual environment and install the required libraries

conda create -n densepose python=3.7
conda activate densepose
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
pip install git+https://github.com/facebookresearch/detectron2@main#subdirectory=projects/DensePose

2. try to inference an image with a pre-trained human CSE model, following the tutorial of https://github.com/facebookresearch/detectron2/blob/main/projects/DensePose/doc/GETTING_STARTED.md

cd detectron2/projects/DensePose
python apply_net.py show configs/cse/densepose_rcnn_R_50_FPN_DL_soft_s1x.yaml model_final_9199f5.pkl image.png dp_contour,bbox --output image_densepose_contour.png

3. KeyError was reported

The checkpoint state_dict contains keys that are not used by the model:
  pixel_mean
  pixel_std
~/anaconda3/envs/densepose/lib/python3.7/site-packages/torch/functional.py:478: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at  /opt/conda/conda-bld/pytorch_1659484809535/work/aten/src/ATen/native/TensorShape.cpp:2894.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
Traceback (most recent call last):
  File "apply_net.py", line 354, in <module>
    main()
  File "apply_net.py", line 350, in main
    args.func(args)
  File "apply_net.py", line 106, in execute
    cls.execute_on_outputs(context, {"file_name": file_name, "image": img}, outputs)
  File "apply_net.py", line 284, in execute_on_outputs
    data = extractor(outputs)
  File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 152, in __call__
    data = extractor(instances, select)
  File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 102, in __call__
    results = [converter.convert(dpout[i], boxes_xyxy[[i]]) for i in range(len(dpout))]
  File "~/Projects/detectron2/projects/DensePose/densepose/vis/extractor.py", line 102, in <listcomp>
    results = [converter.convert(dpout[i], boxes_xyxy[[i]]) for i in range(len(dpout))]
  File "~/Projects/detectron2/projects/DensePose/densepose/converters/to_chart_result.py", line 65, in convert
    predictor_outputs, boxes, *args, **kwargs
  File "~/Projects/detectron2/projects/DensePose/densepose/converters/base.py", line 83, in convert
    raise KeyError(f"Could not find converter from {instance_type} to {output_type_str}")
KeyError: "Could not find converter from <class 'densepose.structures.cse.DensePoseEmbeddingPredictorOutput'> to <class 'densepose.structures.chart_result.DensePoseChartResultWithConfidences'>"

Code Checking

I found that, in the detectron2/projects/DensePose/densepose/converters/base.py, _lookup_converter function returns None because there are only keys of <class 'densepose.structures.chart.DensePoseChartPredictorOutput'> in cls.registry, how can I modify this file to make the return result correct?

# the key-values in cls.registry using print function to output
{<class 'densepose.structures.chart.DensePoseChartPredictorOutput'>: <function densepose_chart_predictor_output_to_result_with_confidences at 0x7fb445a66dd0>}
{<class 'densepose.structures.chart.DensePoseChartPredictorOutput'>: <function densepose_chart_predictor_output_to_result_with_confidences at 0x7fb445a66dd0>}

Environment:

----------------------  --------------------------------------------------------------------------------------------------------
sys.platform            linux
Python                  3.7.12 | packaged by conda-forge | (default, Oct 26 2021, 06:08:21) [GCC 9.4.0]
numpy                   1.21.5
detectron2              0.6 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/detectron2
Compiler                GCC 5.5
CUDA compiler           CUDA 11.3
detectron2 arch flags   7.5
DETECTRON2_ENV_MODULE   <not set>
PyTorch                 1.12.1 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/torch
PyTorch debug build     False
GPU available           Yes
GPU 0                   NVIDIA GeForce GTX 1660 (arch=7.5)
Driver version          470.86
CUDA_HOME               /usr/local/cuda-11.3
Pillow                  9.2.0
torchvision             0.13.1 @~/anaconda3/envs/densepose/lib/python3.7/site-packages/torchvision
torchvision arch flags  3.5, 5.0, 6.0, 7.0, 7.5, 8.0, 8.6
fvcore                  0.1.5.post20220512
iopath                  0.1.9
cv2                     4.6.0
----------------------  --------------------------------------------------------------------------------------------------------
PyTorch built with:
  - GCC 9.3
  - C++ Version: 201402
  - Intel(R) oneAPI Math Kernel Library Version 2021.4-Product Build 20210904 for Intel(R) 64 architecture applications
  - Intel(R) MKL-DNN v2.6.0 (Git Hash 52b5f107dd9cf10910aaa19cb47f3abf9b349815)
  - OpenMP 201511 (a.k.a. OpenMP 4.5)
  - LAPACK is enabled (usually provided by MKL)
  - NNPACK is enabled
  - CPU capability usage: AVX2
  - CUDA Runtime 11.3
  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
  - CuDNN 8.3.2  (built against CUDA 11.5)
  - Magma 2.5.2
  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.3.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.12.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, 

[github-actions]

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template.
The following information is missing: "Instructions To Reproduce the Issue and Full Logs";

[vnghia]

Are you able to solve this bug ? I meet the same bug today :(

[nxrcqupt01]

Are you able to solve this bug ? I meet the same bug today

[kim-sehun]

anj dlEKsrjf zhemfkrh rhdrogoshKTdj qudtlsrkxdl