Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 72 additions & 0 deletions docs/tutorial-depthai.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Tutorial for Visual SLAM using a Depthai OAK Camera


## Overview

This tutorial walks you through setting up [Isaac ROS Visual SLAM](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_slam) with a [Luxonis OAK camera](https://www.luxonis.com/).


## Tutorial Walkthrough - VSLAM execution

Most of the steps overlap with [Realsense setup](./tutorial-realsense.md). Main differences include setting up Docker image for use with the cameras.

1. Clone this repository and its dependencies under `${ISAAC_ROS_WS}/src`.

```bash
cd ${ISAAC_ROS_WS}/src
git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_visual_slam
git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common
git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nitros
```
2. Clone `depthai` and `xacro` repositories to the workspace

```bash
cd ${ISAAC_ROS_WS}/src
git clone --branch ros-release https://github.com/luxonis/depthai-core
git clone --branch humble https://github.com/luxonis/depthai-ros
git clone --branch ros2 https://github.com/ros/xacro
```
3. Setup udev rules on host

```bash
echo 'SUBSYSTEM=="usb", ATTRS{idVendor}=="03e7", MODE="0666"' | sudo tee /etc/udev/rules.d/80-movidius.rules
sudo udevadm control --reload-rules && sudo udevadm trigger
```

4. Setup configuration for docker creation

```bash
cd ${ISAAC_ROS_WS}/src/isaac_ros_common/scripts && \
touch .isaac_ros_common-config && \
echo CONFIG_IMAGE_KEY=ros2_humble.depthai > .isaac_ros_common-config && \
touch .isaac_ros_dev-dockerargs && \
echo "-v /dev/bus/usb:/dev/bus/usb" > .isaac_ros_dev-dockerargs
```
5. Run the docker image

```bash
isaac_ros_container
```

> Or if you did not add the command in [step 1-3 of the quickstart section](../README.md#quickstart):
>
> ```bash
> cd ${ISAAC_ROS_WS}/src/isaac_ros_common && \
> ./scripts/run_dev.sh ${ISAAC_ROS_WS}
> ```

5. When inside the container, build the workspace:

```bash
cd /workspaces/isaac_ros-dev && \
colcon build --symlink-install && \
source install/setup.bash
```

6. Run launch file for the camera

```bash
ros2 launch isaac_ros_visual_slam isaac_ros_visual_slam_depthai.launch.py
```

7. `depthai-ros-driver` allows for a large amount of customization using parameters. Example parameters can be found in `isaac_ros_visual_slam/params.depthai.yaml`
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
# Copyright (c) 2021-2023 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

import launch
from launch_ros.actions import ComposableNodeContainer
from launch_ros.descriptions import ComposableNode

import os

from ament_index_python.packages import get_package_share_directory
from launch.actions import IncludeLaunchDescription
from launch.launch_description_sources import PythonLaunchDescriptionSource
from launch_ros.descriptions import ComposableNode
def generate_launch_description():
"""Launch file which brings up visual slam node configured for OAK."""

depthai_prefix = get_package_share_directory('depthai_ros_driver')
nvidia_slam_prefix = get_package_share_directory('isaac_ros_visual_slam')
params_file = os.path.join(nvidia_slam_prefix, 'params', 'depthai.yaml')

depthai = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(depthai_prefix, 'launch', 'camera.launch.py')),
launch_arguments={"name": 'oak',
"params_file": params_file,
}.items())

visual_slam_node = ComposableNode(
name='visual_slam_node',
package='isaac_ros_visual_slam',
plugin='isaac_ros::visual_slam::VisualSlamNode',
parameters=[{
'denoise_input_images': False,
'rectified_images': True,
'enable_debug_mode': False,
'debug_dump_path': '/tmp/cuvslam',
'enable_slam_visualization': True,
'enable_landmarks_view': True,
'enable_observations_view': True,
'map_frame': 'map',
'odom_frame': 'odom',
'base_frame': 'oak-d-base-frame',
'input_imu_frame': 'oak_imu_frame',
'enable_imu_fusion': True,
'gyro_noise_density': 0.9084722604427307 ,
'gyro_random_walk': 0.3189056080587463 ,
'accel_noise_density': 4.072394996046438 ,
'accel_random_walk': 1.9007919574432275 ,
'calibration_frequency': 100.0,
'img_jitter_threshold_ms': 22.00
}],
remappings=[('stereo_camera/left/image', '/oak/left/image_rect'),
('stereo_camera/left/camera_info', '/oak/left/camera_info'),
('stereo_camera/right/image', '/oak/right/image_rect'),
('stereo_camera/right/camera_info', '/oak/right/camera_info'),
('visual_slam/imu', '/oak/imu/data')]
)

visual_slam_launch_container = ComposableNodeContainer(
name='visual_slam_launch_container',
namespace='',
package='rclcpp_components',
executable='component_container',
composable_node_descriptions=[
visual_slam_node
],
output='screen'
)

return launch.LaunchDescription([visual_slam_launch_container, depthai])
18 changes: 18 additions & 0 deletions isaac_ros_visual_slam/params/depthai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/oak:
ros__parameters:
camera:
i_enable_imu: true
i_enable_ir: false
i_pipeline_type: depth
i_publish_tf_from_calibration: true
stereo:
i_publish_topic: false
i_reverse_stereo_socket_order: true
i_publish_synced_rect_pair: true
i_update_ros_base_time_on_ros_msg: true
left:
i_resolution: '400'
i_fps: 90.0
right:
i_resolution: '400'
i_fps: 90.0