Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Depthai support #116

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Serafadam
Copy link

@Serafadam Serafadam commented Sep 8, 2023

This one adds:

  • Launch file for running VSLAM together with an OAK device
  • Readme for setup
  • Configuration file
    One issue that seems to appear is that VSLAM node outputs following warnings:
    [visual_slam_node]: Delta between current and previous frame [44.023430] is above threshold [22.000000]
    Although camera FPS is ~90 according to ros2 topic echo, not sure what could be the issue is. I'd be happy to improve the integration in the near future.
    Related PR: Add DepthAI support isaac_ros_common#92

@mstiehm-NF
Copy link

mstiehm-NF commented Nov 1, 2023

Hi. I was testing out your PR and I'm running into the issue where the calculated baseline of the images for the VSLAM algorithm is 0 and it's throwing an error. Have you seen this before?

admin@BB-8:/workspaces/isaac_ros-dev$ ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam_oak-d.launch.py 
[INFO] [launch]: All log files can be found below /home/admin/.ros/log/2023-10-31-15-56-25-236927-BB-8-13327
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [component_container-1]: process started with pid [13341]
[INFO] [component_container-2]: process started with pid [13343]
[component_container-1] [INFO] [1698785785.688523791] [visual_slam_launch_container]: Load Library: /workspaces/isaac_ros-dev/install/isaac_ros_visual_slam/lib/libvisual_slam_node.so
[component_container-2] [INFO] [1698785785.752385267] [oak_container]: Load Library: /opt/ros/humble/install/lib/librobot_state_publisher_node.so
[component_container-2] [INFO] [1698785785.756013469] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<robot_state_publisher::RobotStatePublisher>
[component_container-2] [INFO] [1698785785.756044347] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<robot_state_publisher::RobotStatePublisher>
[component_container-1] [INFO] [1698785785.757271811] [visual_slam_launch_container]: Found class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_slam::VisualSlamNode>
[component_container-1] [INFO] [1698785785.757302819] [visual_slam_launch_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<isaac_ros::visual_slam::VisualSlamNode>
[component_container-2] [INFO] [1698785785.763172940] [oak_state_publisher]: got segment oak
[component_container-2] [INFO] [1698785785.763196795] [oak_state_publisher]: got segment oak-d-base-frame
[component_container-2] [INFO] [1698785785.763201844] [oak_state_publisher]: got segment oak_imu_frame
[component_container-2] [INFO] [1698785785.763205631] [oak_state_publisher]: got segment oak_left_camera_frame
[component_container-2] [INFO] [1698785785.763209298] [oak_state_publisher]: got segment oak_left_camera_optical_frame
[component_container-2] [INFO] [1698785785.763212925] [oak_state_publisher]: got segment oak_model_origin
[component_container-2] [INFO] [1698785785.763216572] [oak_state_publisher]: got segment oak_rgb_camera_frame
[component_container-2] [INFO] [1698785785.763219908] [oak_state_publisher]: got segment oak_rgb_camera_optical_frame
[component_container-2] [INFO] [1698785785.763223084] [oak_state_publisher]: got segment oak_right_camera_frame
[component_container-2] [INFO] [1698785785.763226310] [oak_state_publisher]: got segment oak_right_camera_optical_frame
[component_container-2] [INFO] [1698785785.764516242] [oak_container]: Load Library: /workspaces/isaac_ros-dev/install/depthai_ros_driver/lib/libdepthai_ros_driver.so
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak_state_publisher' in container 'oak_container'
[component_container-1] [INFO] [1698785785.770041846] [visual_slam_node]: cuVSLAM version: 11.0
[component_container-2] [INFO] [1698785785.816698090] [oak_container]: Found class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1698785785.816761609] [oak_container]: Instantiate class: rclcpp_components::NodeFactoryTemplate<depthai_ros_driver::Camera>
[component_container-2] [INFO] [1698785785.822580033] [oak]: No ip/mxid specified, connecting to the next available device.
[component_container-1] [INFO] [1698785786.657038580] [visual_slam_node]: Time taken by CUVSLAM_WarmUpGPU(): 0.886957
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/visual_slam_node' in container '/visual_slam_launch_container'
[component_container-2] [INFO] [1698785788.514943290] [oak]: Camera with MXID: 18443010B176681200 and Name: 1.1 connected!
[component_container-2] [INFO] [1698785788.515844091] [oak]: USB SPEED: HIGH
[component_container-2] [INFO] [1698785788.562302813] [oak]: Device type: OAK-D-LITE
[component_container-2] [INFO] [1698785788.565119361] [oak]: Pipeline type: depth
[component_container-2] [ERROR] [1698785788.578589720] [oak]: rgb parameters not set, defaulting to 1280x720 unless specified otherwise.
[component_container-2] [WARN] [1698785789.616953828] [oak]: IMU enabled but not available!
[component_container-2] [INFO] [1698785789.617959877] [oak]: Finished setting up pipeline.
[component_container-2] [INFO] [1698785789.816932580] [oak]: Setting up stereo pair sync timer with period 11 ms based on left sensor FPS.
[component_container-2] [INFO] [1698785790.284650386] [oak]: Published URDF
[component_container-2] [WARN] [1698785790.285325614] [oak]: IMU extrinsics appear to be default. Check if the IMU is calibrated.
[component_container-2] [INFO] [1698785790.287026107] [oak]: Camera ready!
[INFO] [launch_ros.actions.load_composable_nodes]: Loaded node '/oak' in container '/oak_container'
[component_container-1] [INFO] [1698785790.288250766] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.288284008] [visual_slam_node]: Baseline is too small: -0.000000
[component_container-1] [INFO] [1698785790.290741352] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.290777780] [visual_slam_node]: Baseline is too small: -0.000000
[component_container-2] [INFO] [1698785790.294824547] [oak_state_publisher]: got segment oak
[component_container-2] [INFO] [1698785790.294850256] [oak_state_publisher]: got segment oak-d-base-frame
[component_container-2] [INFO] [1698785790.294855896] [oak_state_publisher]: got segment oak_model_origin
[component_container-1] [INFO] [1698785790.302124382] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.302184675] [visual_slam_node]: Baseline is too small: -0.000000
[component_container-1] [INFO] [1698785790.313034194] [visual_slam_node]: left_pose_right reading from CameraInfo
[component_container-1] [ERROR] [1698785790.313105829] [visual_slam_node]: Baseline is too small: -0.000000

@Serafadam
Copy link
Author

Hi @mstiehm-NF, thanks for the notification, I'll investigate it next week, I assume that some things changed in the latest release of VSLAM library

@fsarfraz
Copy link

HI @Serafadam, Did you by any chance ever figure out what was causing that warning with the frames?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants