Take depth map from a mesh

[pty0220]

Hi everyone,

I would like to generate a depth map image from a mesh using camera settings similar to the code snippet below.

From my research, I found a distancemap function in MeshLib that appears to create depth images. However, I’m having trouble figuring out how to configure the settings correctly to match the desired camera parameters.

Could anyone provide some guidance or share any suggestions?

defaultCameraParams = {
    'camPos': [0, 0, 25],
    'camLookAt': [0, 0, 0],
    'camUp': [0, 1, 0],
    'xFov': 0.5,
    'height': 1000,
    'width': 1000,
    'translation': [0, 0, -25],
    'near': 0.1,
    'far': 100
}

[Grantim]

Hello!

Looks like you need this function with correctly specified parameters:

MeshLib/source/MRMesh/MRDistanceMap.h

Lines 178 to 181 in b68be6c

	/// computes distance (height) map for given projection parameters
	/// using float-precision for finding ray-mesh intersections, which is faster but less reliable
	MRMESH_API DistanceMap computeDistanceMap( const MeshPart& mp, const MeshToDistanceMapParams& params,
	ProgressCallback cb = {}, std::vector<MeshTriPoint> * outSamples = nullptr );

You can setup to fit mesh

MeshLib/source/MRMesh/MRDistanceMapParams.h

Lines 21 to 28 in b68be6c

	/// direction vector shows the projections vector to the distance map for points on model
	/// yRange and xRange directions make orthonormal basis with direction
	/// see Vector3<T>::perpendicular() for more details
	/// All Output Distance map values will be positive
	/// usePreciseBoundingBox false (fast): use general (cached) bounding box with applied rotation
	/// usePreciseBoundingBox true (slow): compute bounding box from points with respect to rotation
	MRMESH_API MeshToDistanceMapParams( const Vector3f& direction, const Vector2i& resolution, const MeshPart& mp, bool usePreciseBoundingBox = false );
	MRMESH_API MeshToDistanceMapParams( const Vector3f& direction, const Vector2f& pixelSize, const MeshPart& mp, bool usePreciseBoundingBox = false );

Or more precisely:

MeshLib/source/MRMesh/MRDistanceMapParams.h

Lines 30 to 36 in b68be6c

	/// input matrix should be orthonormal!
	/// rotation.z - direction
	/// rotation.x * (box X length) - xRange
	/// rotation.y * (box Y length) - yRange
	/// All Output Distance map values will be positive
	MRMESH_API MeshToDistanceMapParams( const Matrix3f& rotation, const Vector3f& origin, const Vector2i& resolution, const Vector2f& size );
	MRMESH_API MeshToDistanceMapParams( const Matrix3f& rotation, const Vector3f& origin, const Vector2f& pixelSize, const Vector2i& resolution );

Please note that this algorithm will make depth map with parallel rays, so if you want to achieve perspective effect you will need to raycast each pixel by yourself.

As I can see from your parameters you need something like this:

Matrix3f rotation;
rotation.z = Vector3f::minusZ(); // direction `camLookAt - camPos`
float pixelSize = 0.1f * std::tan( 0.25f ) / 500.0f; // near * tg( xFov/2 ) / ( width / 2 )        : suggest that we have orthographic camera with screen placed on near plane
MeshToDistanceMapParams(  rotation, Vector3f() /* origin = camPos + translation */,  Vector2f(pixelSize,pixelSize), Vector2i(1000,1000) /* resolution */ );

[pty0220]

Thank you for your answer!
Could I get the Python example also ?

[Grantim]

In 3rd version of meshlib python modules has all c++ functions, so it should look like this:

from meshlib import mrmeshpy as mm
import math

defaultCameraParams = {
    'camPos': [0, 0, 25],
    'camLookAt': [0, 0, 0],
    'camUp': [0, 1, 0],
    'xFov': 0.5,
    'height': 1000,
    'width': 1000,
    'translation': [0, 0, -25],
    'near': 0.1,
    'far': 100
}

mesh = mm.loadMesh("path/to/mesh.stl")

rot = mm.Matrix3f()
rot.z = mm.Vector3f( defaultCameraParams['camLookAt'][0] , defaultCameraParams['camLookAt'][1], defaultCameraParams['camLookAt'][2] ) - mm.Vector3f( defaultCameraParams['camPos'][0]  , defaultCameraParams['camPos'][1],defaultCameraParams['camPos'][2])
rot.z = rot.z.normalized()
pixSize = defaultCameraParams['near'] * math.tan( defaultCameraParams['xFov'] * 0.5 ) / ( defaultCameraParams['width'] * 0.5 )
org = mm.Vector3f( defaultCameraParams['camPos'][0]  , defaultCameraParams['camPos'][1],defaultCameraParams['camPos'][2]) + mm.Vector3f( defaultCameraParams['translation'][0]  , defaultCameraParams['translation'][1],defaultCameraParams['translation'][2])
org = org - pixSize * mm.Vector3f( defaultCameraParams['width'], defaultCameraParams['height'], 0 ) * 0.5 # to be in the center of the screen
pm = mm.MeshToDistanceMapParams( rot, org, mm.Vector2f( pixSize, pixSize ), mm.Vector2i( defaultCameraParams['width'], defaultCameraParams['height'] ) )

dm = mm.computeDistanceMap( mesh, pm )
img = mm.convertDistanceMapToImage( dm )

mm.ImageSave.toPng( img, "path/to/img/dm.png" )

note: 3rd versions of meshlib are in beta test now (hopefully we will publish them on this week), to install it you need to specify version explicitly: pip install meshlib==3.0.1.205