Input point cloud has no data! #171
Comments
|
if you print the depth range here, how does it look like? |
|
Hi, @wenbowen123, a great and useful work. I am also getting the same error. Please help. I am sharing a few details to provide more info about the situation.
if depth.encoding == "32FC1":
depth_cv = self.cv_bridge.imgmsg_to_cv2(depth)
depth_cv = np.array(depth_cv)
depth_cv[np.isnan(depth_cv)] = 0
depth_cv = depth_cv * 1000
depth_cv = depth_cv.astype(np.uint16)
python run_custom.py --mode run_video --video_dir ../data/test_obj_pose --out_folder ../data/test_obj_pose/out/bundlesdf --use_segmenter 0 --use_gui 1 --debug_level 2
bsdf-vie.mp4
bsdf-runs-with-default-milk-data.mp4 |
|
@jishnujayakumar at which step did you print the depth? It'd be great to keep track of the depth from here , and see at which step it become all zero. Then I can advise more. |
|
Thanks for the reply, @wenbowen123. Following is the depth trace after line#531. Seems it's not zero. A simple difference between milk example and my data is that the depth values in the former is between 0-1 while mine goes beyond 1. Not sure if this should be an issue as I have raw depth values and as I understand BundleSDF should work with raw depth values. Please correct if I am wrong. > /home/jishnu/Projects/mm-demo/vie/BundleSDF/bundlesdf.py(539)run()
538
--> 539 thres = np.percentile(depth[valid], percentile)
540 depth[depth>=thres] = 0
ipdb> np.unique(depth)
array([0. , 1.425, 1.431, 1.437, 1.443, 1.449, 1.456, 1.462, 1.468,
1.474, 1.481, 1.487, 1.493, 1.5 , 1.507, 1.513, 1.52 , 1.527,
1.533, 1.54 , 1.547, 1.554, 1.561, 1.568, 1.575, 1.583, 1.59 ,
1.597, 1.605, 1.612, 1.62 , 1.628, 1.635, 1.643, 1.651, 1.659,
1.667, 1.675, 1.683, 1.692, 1.7 , 1.708, 1.717, 1.726, 1.734,
1.743, 1.752, 1.761, 1.77 , 1.779, 1.788, 1.798, 1.807, 1.817,
1.826, 1.836, 1.846, 1.856, 1.866, 1.876, 1.886, 1.897, 1.907,
1.918, 1.929, 1.939, 1.95 , 1.961, 1.973, 1.984, 1.996, 2.007,
2.019, 2.03 , 2.043, 2.055, 2.068, 2.08 , 2.093, 2.105, 2.118,
2.132, 2.145, 2.158, 2.172, 2.186, 2.2 , 2.214, 2.228, 2.243])
> /home/jishnu/Projects/mm-demo/vie/BundleSDF/bundlesdf.py(540)run()
539 thres = np.percentile(depth[valid], percentile)
--> 540 depth[depth>=thres] = 0
541 logging.info("percentile denoise done")
ipdb> np.unique(depth)
array([0. , 1.425, 1.431, 1.437, 1.443, 1.449, 1.456, 1.462, 1.468,
1.474, 1.481, 1.487, 1.493, 1.5 , 1.507, 1.513, 1.52 , 1.527,
1.533, 1.54 , 1.547, 1.554, 1.561, 1.568, 1.575, 1.583, 1.59 ,
1.597, 1.605, 1.612, 1.62 , 1.628, 1.635, 1.643, 1.651, 1.659,
1.667, 1.675, 1.683, 1.692, 1.7 , 1.708, 1.717, 1.726, 1.734,
1.743, 1.752, 1.761, 1.77 , 1.779, 1.788, 1.798, 1.807, 1.817,
1.826, 1.836, 1.846, 1.856, 1.866, 1.876, 1.886, 1.897, 1.907,
1.918, 1.929, 1.939, 1.95 , 1.961, 1.973, 1.984, 1.996, 2.007,
2.019, 2.03 , 2.043, 2.055, 2.068, 2.08 , 2.093, 2.105, 2.118,
2.132, 2.145, 2.158, 2.172, 2.186, 2.2 , 2.214, 2.228, 2.243])
> /home/jishnu/Projects/mm-demo/vie/BundleSDF/bundlesdf.py(541)run()
540 depth[depth>=thres] = 0
--> 541 logging.info("percentile denoise done")
542
ipdb> np.unique(depth)
array([0. , 1.425, 1.431, 1.437, 1.443, 1.449, 1.456, 1.462, 1.468,
1.474, 1.481, 1.487, 1.493, 1.5 , 1.507, 1.513, 1.52 , 1.527,
1.533, 1.54 , 1.547, 1.554, 1.561, 1.568, 1.575, 1.583, 1.59 ,
1.597, 1.605, 1.612, 1.62 , 1.628, 1.635, 1.643, 1.651, 1.659,
1.667, 1.675, 1.683, 1.692, 1.7 , 1.708, 1.717, 1.726, 1.734,
1.743, 1.752, 1.761, 1.77 , 1.779, 1.788, 1.798, 1.807, 1.817,
1.826])
[bundlesdf.py] percentile denoise done
> /home/jishnu/Projects/mm-demo/vie/BundleSDF/bundlesdf.py(543)run()
542
--> 543 frame = self.make_frame(color, depth, K, id_str, mask, occ_mask, pose_in_model)
544 os.makedirs(f"{self.debug_dir}/{frame._id_str}", exist_ok=True)
ipdb> np.unique(depth)
array([0. , 1.425, 1.431, 1.437, 1.443, 1.449, 1.456, 1.462, 1.468,
1.474, 1.481, 1.487, 1.493, 1.5 , 1.507, 1.513, 1.52 , 1.527,
1.533, 1.54 , 1.547, 1.554, 1.561, 1.568, 1.575, 1.583, 1.59 ,
1.597, 1.605, 1.612, 1.62 , 1.628, 1.635, 1.643, 1.651, 1.659,
1.667, 1.675, 1.683, 1.692, 1.7 , 1.708, 1.717, 1.726, 1.734,
1.743, 1.752, 1.761, 1.77 , 1.779, 1.788, 1.798, 1.807, 1.817,
1.826])
> /home/jishnu/Projects/mm-demo/vie/BundleSDF/bundlesdf.py(544)run()
543 frame = self.make_frame(color, depth, K, id_str, mask, occ_mask, pose_in_model)
--> 544 os.makedirs(f"{self.debug_dir}/{frame._id_str}", exist_ok=True)
545
ipdb> np.unique(depth)
array([0. , 1.425, 1.431, 1.437, 1.443, 1.449, 1.456, 1.462, 1.468,
1.474, 1.481, 1.487, 1.493, 1.5 , 1.507, 1.513, 1.52 , 1.527,
1.533, 1.54 , 1.547, 1.554, 1.561, 1.568, 1.575, 1.583, 1.59 ,
1.597, 1.605, 1.612, 1.62 , 1.628, 1.635, 1.643, 1.651, 1.659,
1.667, 1.675, 1.683, 1.692, 1.7 , 1.708, 1.717, 1.726, 1.734,
1.743, 1.752, 1.761, 1.77 , 1.779, 1.788, 1.798, 1.807, 1.817,
1.826])
ipdb> frame
<my_cpp.Frame object at 0x7fef343bbdf0>
ipdb> n
> /home/jishnu/Projects/mm-demo/vie/BundleSDF/bundlesdf.py(546)run()
545
--> 546 logging.info(f"processNewFrame start {frame._id_str}")
547 # self.bundler.processNewFrame(frame)
ipdb> np.unique(depth)
array([0. , 1.425, 1.431, 1.437, 1.443, 1.449, 1.456, 1.462, 1.468,
1.474, 1.481, 1.487, 1.493, 1.5 , 1.507, 1.513, 1.52 , 1.527,
1.533, 1.54 , 1.547, 1.554, 1.561, 1.568, 1.575, 1.583, 1.59 ,
1.597, 1.605, 1.612, 1.62 , 1.628, 1.635, 1.643, 1.651, 1.659,
1.667, 1.675, 1.683, 1.692, 1.7 , 1.708, 1.717, 1.726, 1.734,
1.743, 1.752, 1.761, 1.77 , 1.779, 1.788, 1.798, 1.807, 1.817,
1.826])
[bundlesdf.py] processNewFrame start 000000
> /home/jishnu/Projects/mm-demo/vie/BundleSDF/bundlesdf.py(548)run()
547 # self.bundler.processNewFrame(frame)
--> 548 self.process_new_frame(frame)
549 logging.info(f"processNewFrame done {frame._id_str}")
ipdb> np.unique(depth)
array([0. , 1.425, 1.431, 1.437, 1.443, 1.449, 1.456, 1.462, 1.468,
1.474, 1.481, 1.487, 1.493, 1.5 , 1.507, 1.513, 1.52 , 1.527,
1.533, 1.54 , 1.547, 1.554, 1.561, 1.568, 1.575, 1.583, 1.59 ,
1.597, 1.605, 1.612, 1.62 , 1.628, 1.635, 1.643, 1.651, 1.659,
1.667, 1.675, 1.683, 1.692, 1.7 , 1.708, 1.717, 1.726, 1.734,
1.743, 1.752, 1.761, 1.77 , 1.779, 1.788, 1.798, 1.807, 1.817,
1.826])
ipdb> n
[bundlesdf.py] process frame 000000
[pcl::PLYWriter::writeASCII] Input point cloud has no data!
[pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
[pcl::PLYWriter::writeASCII] Input point cloud has no data!
Segmentation fault (core dumped)
|
|
Seems the problem was not normalizing depth. After normalizing as follows the pipeline works. @wenbowen123, please let me know if this is the correct way. percentile = self.cfg_track['depth_processing']["percentile"]
if percentile<100: # Denoise
logging.info("percentile denoise start")
valid = (depth>=0.1) & (mask>0)
thres = np.percentile(depth[valid], percentile)
depth[depth>=thres] = 0
# normalize the depth values between 0 and 1
min_depth, max_depth = np.min(depth), np.max(depth)
depth = (depth - min_depth) / (max_depth - min_depth)
logging.info("percentile denoise done")ipdb> np.unique(depth)
array([0. , 0.7803943 , 0.78368018, 0.78696605, 0.79025192,
0.79353779, 0.7973713 , 0.80065717, 0.80394304, 0.80722892,
0.81106243, 0.8143483 , 0.81763417, 0.82146769, 0.8253012 ,
0.82858708, 0.83242059, 0.83625411, 0.83953998, 0.84337349,
0.84720701, 0.85104053, 0.85487404, 0.85870756, 0.86254107,
0.86692223, 0.87075575, 0.87458927, 0.87897043, 0.88280394,
0.8871851 , 0.89156627, 0.89539978, 0.89978094, 0.9041621 ,
0.90854326, 0.91292442, 0.91730559, 0.92168675, 0.92661555,
0.93099671, 0.93537788, 0.94030668, 0.94523549, 0.94961665,
0.95454545, 0.95947426, 0.96440307, 0.96933187, 0.97426068,
0.97918949, 0.98466594, 0.98959474, 0.99507119, 1. ])Here's a demo run: bsdf-runs-after-depth-norm.mp4 |
|
oh, the default bundlesdf sets a clip value for depth at 1.0. I just updated the code. Please pull again and set the zfar here depending on your max possible depth. Normalizing depth is not the right way. |
|
Thank you for the clarification. It works at my end now. W.r.t. the output, I have one question. The output poses seem to be visually similar using the two diff bsdf-diff-camK-output-after-depth-param-fix-in-cfg.mp4 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
尊敬的wen
我进行自建数据纹理生成出现点云数据加载错误，具体问题如下：
(BundleSDF) a@robot:~/project/BundleSDF$ python run_custom.py --mode run_video --video_dir /data/dataset/output --out_folder /data/dataset/bundlesdf_output --use_segmenter 1 --use_gui 1 --debug_level 2
Warp UserWarning: Python 3.9 or newer is recommended for running Warp, detected sys.version_info(major=3, minor=8, micro=19, releaselevel='final', serial=0)
Warp 1.3.3 initialized:
CUDA Toolkit 12.5, Driver 12.4
Devices:
"cpu" : "x86_64"
"cuda:0" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
"cuda:1" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
CUDA peer access:
Supported fully (all-directional)
Kernel cache:
/home/a/.cache/warp/1.3.3
Warp UserWarning: Python 3.9 or newer is recommended for running Warp, detected sys.version_info(major=3, minor=8, micro=19, releaselevel='final', serial=0)
Warp 1.3.3 initialized:
CUDA Toolkit 12.5, Driver 12.4
Devices:
"cpu" : "x86_64"
"cuda:0" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
"cuda:1" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
CUDA peer access:
Supported fully (all-directional)
Kernel cache:
/home/a/.cache/warp/1.3.3
[2024-09-26 14:45:26.256] [warning] [Bundler.cpp:49] Connected to nerf_port 9999
[2024-09-26 14:45:26.257] [warning] [FeatureManager.cpp:2084] Connected to port 5555
default_cfg {'backbone_type': 'ResNetFPN', 'resolution': (8, 2), 'fine_window_size': 5, 'fine_concat_coarse_feat': True, 'resnetfpn': {'initial_dim': 128, 'block_dims': [128, 196, 256]}, 'coarse': {'d_model': 256, 'd_ffn': 256, 'nhead': 8, 'layer_names': ['self', 'cross', 'self', 'cross', 'self', 'cross', 'self', 'cross'], 'attention': 'linear', 'temp_bug_fix': False}, 'match_coarse': {'thr': 0.2, 'border_rm': 2, 'match_type': 'dual_softmax', 'dsmax_temperature': 0.1, 'skh_iters': 3, 'skh_init_bin_score': 1.0, 'skh_prefilter': True, 'train_coarse_percent': 0.4, 'train_pad_num_gt_min': 200}, 'fine': {'d_model': 128, 'd_ffn': 128, 'nhead': 8, 'layer_names': ['self', 'cross'], 'attention': 'linear'}}
Warp UserWarning: Python 3.9 or newer is recommended for running Warp, detected sys.version_info(major=3, minor=8, micro=19, releaselevel='final', serial=0)
Warp UserWarning: Python 3.9 or newer is recommended for running Warp, detected sys.version_info(major=3, minor=8, micro=19, releaselevel='final', serial=0)
Warp 1.3.3 initialized:
CUDA Toolkit 12.5, Driver 12.4
Devices:
"cpu" : "x86_64"
"cuda:0" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
"cuda:1" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
CUDA peer access:
Supported fully (all-directional)
Kernel cache:
/home/a/.cache/warp/1.3.3
Warp 1.3.3 initialized:
CUDA Toolkit 12.5, Driver 12.4
Devices:
"cpu" : "x86_64"
"cuda:0" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
"cuda:1" : "NVIDIA RTX A6000" (48 GiB, sm_86, mempool enabled)
CUDA peer access:
Supported fully (all-directional)
Kernel cache:
/home/a/.cache/warp/1.3.3
GUI started
[bundlesdf.py] percentile denoise start
[bundlesdf.py] percentile denoise done
[bundlesdf.py] processNewFrame start 000000
[bundlesdf.py] process frame 000000
[pcl::PLYWriter::writeASCII] Input point cloud has no data!
[pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
[pcl::PLYWriter::writeASCII] Input point cloud has no data!
段错误 (核心已转储)
请问这是由什么原因造成的呢？
Best wishes!
The text was updated successfully, but these errors were encountered: