NeRF Project

Reconstruct 3D scenes with NeRF and Nerfstudio.

Project for Fudan University Computer Graphics course, spring 2024. Instructed by Professor Yan Bo.

Demo

Nerfacto model render:

Splatfacto model render:

Folder Structure

NeRF
  |__ assets               
        |__ report            // project report
        |__ slides            // project presentation slides
        |__ demo              // demo GIFs
  |__ eval                    // .json evaluation outputs
  |__ LICENSE 		             
  |__ models                 // trained models
        |__ nerfacto
        |__ splatfacto
  |__ notebooks              // Jupyter notebooks for training
  |__ README.md

Dataset

We first converted and extracted the video frames into images.

Then, we pre-processed 313 images of the M60 tank with COLMAP (feature extraction, feature matching, sparse reconstruction).

• Original video of the M60 Tank from Tanks & Temple Dataset
• COLMAP pre-processed images of the M60 Tank (Kaggle)

Models

We trained and used Nerfstudio's Nerfacto and Splatfacto model. Training was done on Kaggle using the P100 GPU.

• Trained Nerfacto model
• Trained Splatfacto model
• Nerfacto training notebook (Kaggle)
• Splatfacto training notebook (Kaggle)

Model performance

PSNR (Peak Signal-to-Noise Ratio) is a measure of the peak error between the rendered image and the ground truth image, and it is expressed in decibels (dB). A higher PSNR value indicates better image quality and a closer match to the ground truth image.

Dataset	Model	PSNR
M60 Tank	Nerfacto	19.25
M60 Tank	Splatfacto	27.46

Extras

For the extra part of our project, we've decided to reconstruct a 3D model from a 2D toy tank image with DreamGaussian.

2D base image:

Generated 3D model:

Documentation

For more details about the project, please check out our report and presentation slides.

• Project report (PDF)
• Project presentation slides (PDF)

License

Copyright © 2024 JinnyWong, Ng Yu Yue, Zhou Tao, Goh Xin Yie.

Licensed under the MIT License.