Awesome 3D Reconstruction Papers

A collection of 3D reconstruction papers in the deep learning era. Feel free to contribute :)

Table of Contents

• Object-level
  • Single-view
  • Multi-view
• Scene-level
  • Single-view
  • Multi-view
• Survey

Object-level

Single-view

Paper	Representation	Publisher	Project/Code
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision	Voxel	NIPS 2016	[Code]
A Point Set Generation Network for 3D Object Reconstruction from a Single Image	Point Cloud	CVPR 2017	[Code]
SurfNet: Generating 3D Shape Surfaces Using Deep Residual Networks	Mesh	CVPR 2017	[Code]
Multi-View Supervision for Single-View Reconstruction via Differentiable Ray Consistency	Voxel	CVPR 2017	[Project]
OctNet: Learning Deep 3D Representations at High Resolutions	Voxel	CVPR 2017	[Code]
Rethinking Reprojection: Closing the Loop for Pose-Aware Shape Reconstruction From a Single Image	Voxel	ICCV 2017	/
MarrNet: 3D Shape Reconstruction via 2.5D Sketches	Voxel	NIPS 2017	[Project]
Hierarchical Surface Prediction for 3D Object Reconstruction	Voxel	3DV 2017	/
Image2Mesh: A Learning Framework for Single Image 3D Reconstruction	Mesh	ACCV 2018	[Code]
Learning Efficient Point CloudGeneration for Dense 3D Object Reconstruction	Point Cloud	AAAI 2018	[Project]
A Papier-Mâché Approach to Learning 3D Surface Generation	Mesh	CVPR 2018	[Project]
Im2Struct: Recovering 3D Shape Structure From a Single RGB Image	Parts	CVPR 2018	[Code]
Matryoshka Networks: Predicting 3D Geometry via Nested Shape Layers	Voxel	CVPR 2018	[Code]
Multi-View Consistency as Supervisory Signal for Learning Shape and Pose Prediction	Voxel	CVPR 2018	[Project]
Efficient Dense Point Cloud Object Reconstruction using Deformation Vector Fields	Point Cloud	ECCV 2018	/
GAL: Geometric Adversarial Loss for Single-View 3D-Object Reconstruction	Point Cloud	ECCV 2018	/
Learning Category-Specific Mesh Reconstruction from Image Collections	Mesh	ECCV 2018	[Project]
Learning Shape Priors for Single-View 3D Completion and Reconstruction	Voxel	ECCV 2018	/
Learning Single-View 3D Reconstruction with Limited Pose Supervision	Voxel	ECCV 2018	[Code]
Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images	Mesh	ECCV 2018	[Code]
Residual MeshNet: Learning to Deform Meshes for Single-View 3D Reconstruction	Mesh	3DV 2018	/
Learning to Reconstruct Shapes from Unseen Classes	Generic	NIPS 2018	/
Multi-View Silhouette and Depth Decomposition for High Resolution 3D Object Representation	Voxel	NIPS 2018	[Code]
MVPNet: Multi-View Point Regression Networks for 3D Object Reconstruction from A Single Image	Point Cloud	AAAI 2019	/
Deep Single-View 3D Object Reconstruction with Visual Hull Embedding	Voxel	AAAI 2019	[Code]
Occupancy Networks: Learning 3D Reconstruction in Function Space	Implicit	CVPR 2019	[Code]
Learning Implicit Fields for Generative Shape Modeling	Implicit	CVPR 2019	[Project]
A Skeleton-Bridged Deep Learning Approach for Generating Meshes of Complex Topologies From Single RGB Images	Mesh	CVPR 2019	[Code]
What Do Single-view 3D Reconstruction Networks Learn?	Generic	CVPR 2019	[Code]
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation	Implicit	CVPR 2019	[Code]
Deep Level Sets: Implicit Surface Representations for 3D Shape Inference	Implicit	Arxiv 2019	/
Deep Mesh Reconstruction From Single RGB Images via Topology Modification Networks	Mesh	ICCV 2019	[Code]
GraphX-Convolution for Point Cloud Deformation in 2D-to-3D Conversion	Point Cloud	ICCV 2019	[Code]
Learning Shape Templates With Structured Implicit Functions	Implicit	ICCV 2019	[Code]
Pix2Vox: Context-Aware 3D Reconstruction From Single and Multi-View Images	Voxel	ICCV 2019	[Code]
Domain-Adaptive Single-View 3D Reconstruction	Voxel	ICCV 2019	[Code]
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction	Implicit	NIPS 2019	[Code]
Front2Back: Single View 3D Shape Reconstruction via Front to Back Prediction	Mesh	CVPR 2020	[Code]
Local Deep Implicit Functions for 3D Shape	Implicit	CVPR 2020	[Code]
Differentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision	Implicit	CVPR 2020	[Project]
BSP-Net: Generating Compact Meshes via Binary Space Partitioning	Mesh	CVPR 2020	[Project]
Height and Uprightness Invariance for 3D Prediction From a Single View	Point Cloud	CVPR 2020	[Code]
Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion	Implicit	CVPR 2020	[Project]
Unsupervised Learning of Probably Symmetric Deformable 3D Objects From Images in the Wild	Mesh	CVPR 2020	[Project]
Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction	Implicit	ECCV 2020	/
Few-Shot Single-View 3-D Object Reconstruction with Compositional Priors	Voxel	ECCV 2020	[Code]
GSIR: Generalizable 3D Shape Interpretation and Reconstruction	Voxel	ECCV 2020	/
DR-KFS: A Differentiable Visual Similarity Metric for 3D Shape Reconstruction	Mesh	ECCV 2020	/
Self-supervised Single-view 3D Reconstruction via Semantic Consistency	Mesh	ECCV 2020	[Project]
Shape and Viewpoint without Keypoints	Mesh	ECCV 2020	[Project]
Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry	Mesh	ECCV 2020	[Code]
Learning to Detect 3D Reflection Symmetry for Single-View Reconstruction	/	Arxiv 2020	[Code]
3D Reconstruction of Novel Object Shapes from Single Images	Implicit	Arxiv 2020	[Project]
3D Shape Reconstruction from Free-Hand Sketches	Point Cloud	Arxiv 2020	[Code]
Implicit Mesh Reconstruction from Unannotated Image Collections	Mesh	Arxiv 2020	[Project]
SkeletonNet: A Topology-Preserving Solution for Learning Mesh Reconstruction of Object Surfaces from RGB Images	Mesh	Arxiv 2020	[Code]
MeshSDF: Differentiable Iso-Surface Extraction	Implicit	NIPS 2020	[Code]
Learning Deformable Tetrahedral Meshes for 3D Reconstruction	Mesh	NIPS 2020	[Project]
MetaSDF: Meta-learning Signed Distance Functions	Implicit	NIPS 2020	[Project]
SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images	Implicit	NIPS 2020	[Project]
UCLID-Net: Single View Reconstruction in Object Space	Mesh	NIPS 2020	[Code]
Pix2Vox++: Multi-scale Context-aware 3D Object Reconstruction from Single and Multiple Images	Voxel	IJCV 2020	[Code]
Silhouette Guided Point Cloud Reconstruction beyond Occlusion	Point Cloud	WACV 2020	[Code]
Look, Cast and Mold: Learning 3D Shape Manifold from Single-view Synthetic Data	Voxel	Arxiv 2021	/

Multi-view

Paper	Representation	Publisher	Project/Code
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction	Voxel	ECCV 2016	[Code]
3D Shape Induction from 2D Views of Multiple Objects	Voxel	3DV 2017	[Code]
Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction	Point Cloud	AAAI 2018	[Project]
Conditional Single-view Shape Generation for Multi-view Stereo Reconstruction	Point Cloud	CVPR 2019	[Code]
Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformation	Mesh	ICCV 2019	[Project]
Multiview Aggregation for Learning Category-Specific Shape Reconstruction	Point Cloud	NIPS 2019	[Code]
Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance	Mesh	NIPS 2020	[Project]

Scene-level

Single-view

Paper	Representation	Publisher	Project/Code
IM2CAD	CAD	CVPR 2017	[Code]
3D-RCNN: Instance-level 3D Object Reconstruction via Render-and-Compare	Priors	CVPR 2018	[Project]
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scene	Voxel	CVPR 2018	[Project]
Holistic 3D Scene Parsing and Reconstruction from a Single RGB Image	CAD	ECCV 2018	[Project]
Mesh R-CNN	Mesh	ICCV 2019	[Code]
3D Scene Reconstruction With Multi-Layer Depth and Epipolar Transformers	Mesh	ICCV 2019	[Project]
3D-RelNet: Joint Object and Relational Network for 3D Prediction	Voxel	ICCV 2019	[Project]
Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image	Mesh	CVPR 2020	[Project]
3D Scene Reconstruction from a Single Viewport	Voxel	ECCV 2020	[Code]
CoReNet: Coherent 3D scene reconstruction from a single RGB image	Voxel+Implicit	ECCV 2020	[Code]
Image-to-Voxel Model Translation for 3D Scene Reconstruction and Segmentation	Voxel	ECCV 2020	[Code]
Holistic 3D Scene Understanding from a Single Image with Implicit Representation	Implicit	CVPR 2021	[Project]

Multi-view

Paper	Representation	Publisher	Project/Code
MARMVS: Matching Ambiguity Reduced Multiple View Stereo for Efficient Large Scale Scene Reconstruction	Point Cloud	CVPR 2020	/
Associative3D: Volumetric Reconstruction from Sparse Views	Voxel	ECCV 2020	[Project]
Atlas: End-to-End 3D Scene Reconstruction from Posed Images	Mesh	ECCV 2020	[Project]
NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video	Mesh	CVPR 2021	[Project]

Survey

Paper	Publisher
Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era	TPAMI 2019