Training options

[voldemortX]

Hi! I'm new to NeRF, could I ask what the Train envmap Train extrinsics Train exposure Train distortion mean exactly, and what is the practical advice for setting these in practice?

[sqartox]

Hi. Did you made any progress? I'm looking for the answer to your question but can't seem to find anything.

[voldemortX]

I think they are what they mean. For example train extrinsics will learn something to make up for incorrect extrinsic estimation from COLMAP. But I'm not sure how they are implemented or what do they do exactly.
@Tom94 Could you give us some help please?

[Tom94]

Yes! "Train extrinsics" propagates gradients back onto the camera parameters in order to minimize the loss (in this case by treating the cameras as rigid bodies with momentum and the loss gradients as impulses acting on them). This can fix slight inaccuracies in the poses from transform.json. If you're using COLMAP, this option only yields a small benefit as COLMAP is already quite accurate -- I'd leave it off for most scenes.

Same with "Train distortion", which learns a grid-based camera distortion model on top of the OpenCV camera model parameterized in transforms.json. Again, COLMAP usually does a good enough job that you don't need this.

"Train exposure" learns to compensate for varying exposure levels of the training images. If you generate your own data, you should not use this and instead set your camera to fixed exposure.

"Train envmap" is experimental and currently not practically useful. Don't use it for now. :)

[voldemortX]

Thanks a lot for the detailed explanations!

[3020]

Hi. Any updates on the envMap functionality?