Linux SDK Examples

Document version: V1.0; revision date: 2024-10-12.

Overview

The MRDVS Camera SDK for Linux includes C/C++, Java (HTTP service), ROS, and ROS 2 examples. The examples show the basic SDK workflow; add application-specific logic and error handling for your project. Complete the environment setup, installation script, and sample-directory checks first. Then follow the compilation and execution section for your target language. The appendix lists installation methods for common dependencies and test packages.

System and environment

Compatible systems and dependency versions

The following Linux systems are supported. The default program versions are listed below:

System version	Cmake/Gcc	Opencv	Pcl	Ros	Ros2
Ubuntu16.04	3.5/Gcc-5	3.2.0	1.7.2	Kinetic	Ardent、Bouncy
Ubuntu18.04	3.18/Gcc-7	4.5.1	1.9.1	Melodic	Dashing、Eloquent、Crystal
Ubuntu20.04	3.22/9.3.0	4.5.1	1.12.1	Noetic	Foxy、Galactic
Ubuntu22.04	3.22/11.3.0	4.6.0	1.12.1	Ninjemys	Humble、lron

2.2 Check the system version and application version

2.2.1 Check the system version

uname -a

result:

Operating system release version number: 5.15.0

Main version number: 5
Version number: 15 [Odd numbers are development versions, even numbers are stable versions]
Revision number: 0 [Number of modifications]
Nth revision of this version: 97
Kernel version: #107~20.0.4.1, usually the system version is the kernel version

2.2.2 Check Cmake version

cmake --version

result:

Represents the version of Cmake 3.16.3

2.2.3 Check Gcc version

gcc -v

result:

The version number is in the blue box in the lower left corner. The version here is: 9.4.0

2.2.4 View Opencv version

pkg-config --modversion opencv

result:

The version here is 3.2.0

2.2.5 Check Pcl version

dpkg -l libpcl-dev

result: The version here is 1.8.1

2.2.6 View ROS version

rosversion -d

result:

The version here is eloquent (the ros2 version corresponding to Ubuntu18.04)

The version here is noetic (the ros1 version corresponding to Ubuntu20)

2.3 Build a basic development environment

You need to update the installation list before installation:

sudo apt update

The result is as follows:

Illustrations related to building a basic development environment

2.3.1 Install Cmake

sudo apt install cmake

2.3.2 Install Gcc

sudo apt install build-essential

2.4 Build optional development environment

2.4.1 Install Opencv [can display camera images]

sudo apt-get install libopencv-dev

2.4.2 Install Pcl [can display point cloud]

sudo apt-get install libpcl-dev

Prepare the sample code

3.1 Execute the installation script

Unzip the Lanxin-MRDVS file and enter the Lanxin-MRDVS folder

Illustrations related to executing the installation script

Right click to open the console
Execute install.sh script

Enter the /opt/ directory to check whether the installation is successful.

Illustration related to executing the installation script 3

If the folder can be found, the installation is successful.
If the installation fails, you need to manually add it to the opt directory.
Execute the set_socket_buffer_size.sh script

At this point, the installation script has been completely executed!

3.2 Sample code introduction

Enter the Sample directory. The directory contents are as follows:

Illustrations related to sample code introduction

The C folder contains C and C++ related code
The Java folder contains an http server for indirectly calling the sdk
The Ros folder contains an example of ros
The Ros2 folder contains an example of ros2

3.2.1 C and C++ sample code

Enter the C directory. The directory contents are as follows:

Illustrations related to C and C++ sample code

Contains 9 folders and 1 Cmakelist file, each folder contains a cpp file The roughly implemented functions of each cpp are as follows:

application_obstacle: Obtain camera stream data and enable obstacle avoidance algorithm, output algorithm results
application_pallet: Obtain camera stream data and enable the pallet algorithm, output algorithm results
arm_local_camera: Call the SDK in the camera [ARM] and obtain the stream data
frame_callback: Use callback method to obtain camera stream data
multi_cameras: open multiple cameras and get streaming data
single_camera: Open a single camera and get streaming data
single_camera2: Get frame data through structure

Different sample codes can be selected for testing and modification according to different needs.

3.2.2 Java sample code

Enter the java directory. The directory contents are as follows:

Illustrations related to Java sample code

The directory contains 2 folders and a postman.json file and README

Config: Server-related configurations are stored in the folder
Middleware: server theme
LxCamera.postman_collection.json: postman configuration file
README: java virtual server usage tutorial

3.2.3 ROS sample code

Enter the ros directory. The directory contents are as follows:

Illustrations related to ROS sample code

lx_camera_node_ws: The folder contains a standard ros driver
README.EN: explains in English how to compile a node and create a new node

3.2.4 ROS2 sample code

Enter the ros2 directory. The directory contents are as follows:

lx_camera_node_ws: The folder contains a standard ros2 driver
README.EN: explains in English how to compile a node and create a new node

3.2.5 ros-v1pro sample code

Enter the ros-v1pro directory. The directory contents are as follows:

Illustrations related to ros-v1pro sample code

src: The folder contains a standard ros driver
README: explains how to use this node
README.pdf: explains how to use this node

Compile and run the samples

4.1 Compiling and running C and C++ sample code

Go to the C folder in the example:

Illustrations related to compiling and running C and C++ sample code

Open a terminal in the C folder and run:

mkdir build
cd build
cmake ..
make

After successful execution, enter the build folder and it should be as follows:

Illustrations related to compiling and running C and C++ sample code 2

The executable file is in the bin directory. Entering the bin directory should be as follows:

Illustrations related to compiling and running C and C++ sample code 3

Except for the arm example, all other examples will be compiled. The executable file name is the same as the example folder name. You can run it directly.

4.2 java sample running process

Java is different from other sample codes. It does not directly provide source code. It provides an http server and a postman example. If you need to control the camera indirectly through http, you can select the java example and enter the java folder. The folder content is as follows:

Illustrations related to the java sample running process

The running process is as follows:

Enter the config folder and modify the http_sever_host field and http_sever_port field in the Configs.json file:
Enter the middleware folder and execute the start.sh script to enable the transfer server

Illustrations related to the java sample running process 3

Use postman to open the LxCamera.postman_collection.json file to send data to the sample port

4.3 Compiling and running ROS sample code

Enter the ROS folder, open a console, and run:

roscore

Illustrations related to compiling and running ROS sample code

Right-click to open a console, enter the lx_camera_node_ws folder, and run:

catkin_make

Illustrations related to compiling and running ROS sample code 3

After successful execution, the following will be displayed:

Illustrations related to compiling and running ROS sample code 4

At this time devel and build in the folder are compiled and generated:

Illustrations related to compiling and running ROS sample code 5

Run the following commands to source the environment and start a node with roslaunch:

source devel/setup.bash
roslaunch lx_camera_ros lx_camera_ros.launch
roslaunch lx_camera_ros obstacleV2.launch
roslaunch lx_camera_ros obstacle.launch
roslaunch lx_camera_ros pallet.launch

From top to bottom, they are: running ordinary nodes, running obstacle avoidance algorithm 2 nodes, running obstacle avoidance algorithm nodes, running pallet algorithm nodes. Each node cannot be enabled at the same time.

4.4 Compiling and running ROS2 sample code

Enter the ros2 folder. The contents of the folder are as follows:

Illustrations related to compiling and running ROS2 sample code

Example ros node is compiled using colcon First, install the compilation tools:

sudo apt install python3-colcon-common-extensions

Then install the ros-pcl dependency package. Replace foxy with the ROS 2 distribution installed on your system:

sudo apt-get install ros-foxy-pcl*

After installation, enter the lx_camera_node_ws directory and build the project:

colcon build

Depending on the computer performance, compilation usually takes 30 seconds to 120 seconds. After successful compilation, it will be displayed as follows:

Illustrations related to compiling and running ROS2 sample code 3

Some warnings are due to version compatibility issues and do not need to be processed. After compilation is completed, the contents of the folder are as follows:

Illustrations related to compiling and running ROS2 sample code 4

Run the following commands to source the environment and start a node with ros2 launch:

source install/setup.bash
ros2 launch lx_camera_ros lx_camera_ros_launch.py
ros2 launch lx_camera_ros obstacleV2.launch.py
ros2 launch lx_camera_ros obstacle.launch.py
ros2 launch lx_camera_ros pallet.launch.py

4.5 Compilation and operation of ros-v1pro sample code

ros-v1pro is a dedicated ros driver for the V1PRO camera. There is a README.pdf in the ros-v1pro folder that describes the driver in detail. This document only gives a brief introduction to it: Enter the ros-v1pro folder and compile it:

catkin_make

Load the environment:

source devel/setup.bash

Illustrations related to compiling and running the ros-v1pro sample code 3

Finally run the node:

roslaunch lx_camera_ros localization.launch
roslaunch lx_camera_ros mapping.launch
roslaunch lx_camera_ros sensor_sim.launch

From top to bottom: run the positioning node, run the mapping node, and run the virtual data node. Each node cannot be enabled at the same time.

Sample code reference

In this section, all codes and pseudocode will be divided into several parts. Generally speaking, everything in a section belongs to the same file. Except for special files with detailed explanations, the source code files only explain the general modules. Detailed explanations are commented in the code.

5.1 C and C++

5.1.1 CMakeLists

5.1.2 application_obstacle and application_pallet

5.1.3 arm_local_camera

5.1.4 frame_callback

5.1.5 multi_cameras

5.1.6 single_camera

5.1.7 single_camera2

5.2 java

The Java example provides only an HTTP server, so the description is only a description of the interface and message format. Use postman to open the sample file as shown below: The messages are all in post mode. Before sending, remember to change the IP to the server’s IP, but do not modify the URL. The URL is hard-coded internally in the server. The response message format and content are both in the body. The interface provided in the example is as follows:

1.DcGetApiVersion: Get the version number of the SDK, no parameter input

2.DcSetInfoOutput: Set the path for SDK log output. The parameters are as follows:

log_path: path of log output
print_level: log printing level
enable_screen_print: whether the log is printed on the console

3.DcLog: Output the log to the SDK log. Any input content will be printed to the SDK log. 4.DcGetDeviceList: Get the device list, no input, input is JSON 5.DcOpenDevice: Open the device and enter the following:

open_mode: open the camera mode
param: parameters for opening the camera

6.DcStartStream: Let a certain camera start pushing streaming data. The input parameters are as follows:

handle: The handle of the camera, which will be returned in DcOpenDevice

7.DcSetCmd: Issue commands to the camera, enter as follows:

handle: the handle of the camera to be set
cmd: command number

8.DcGetPtrValue: Get pointer data (image, parameters, etc.), enter as follows:

handle: the handle of the camera to be set
cmd: command number

9.DcSetIntValue: Set the camera int type parameters, input as follows:

handle: the handle of the camera to be set
cmd: command number
value: the value to be set

10.DcGetIntValue: Get the int type parameter of the device, input as follows:

handle: the handle of the camera to be set
cmd: command number

Please see the table below for other relevant instructions:

Send	`{"log_path" : "./", "print_level" : 2, "enable_screen_print" : 0 }`	Receive	`{"errCode" : 0 }`
Send	`{"open_mode" : 0, "param" : "0"<br />}`	Receive	`{"algor_ver" : "VER_1.0.2.230315", "dev_type" : "M4 camera", "errCode" : 0,<br />"firmware_ver" : "V1.0.0.3_230315",<br />"handle" : 174539057817095, "id" : "f13141144bb1",<br />"ip" : "192.168.100.16:3956",<br />"mac" : "9e:be:0a:8a:66:07",<br />"name" :"camera_M4_192.168.100.16_3956",<br />"sn" : "9a5ed62d22b34bb1"<br />}`
Send	`{"handle" :174539057817<br />095,<br />"cmd" : 6001<br />}`	Receive	`{"errCode" : 0,<br />"value" :"\u0011\u0001\u0014\u0001..." }`

5.3 ROS1 & ROS2

In the ros description, we will not talk about the driver source code, but about the content corresponding to the configuration in the launch file, how to adjust the configuration, and how to send and subscribe to messages. The provided ros node already contains most of the functions, so it is not recommended to directly modify the driver source code. If you have special needs, you can first understand the code flow in the C++ example and then modify the driver source code. When executing the ros driver, you must ensure that the versions of opencv and pcl in the system default path are unique, otherwise an error will occur when linking the ros library, which may cause compilation failure or runtime crash.

5.3.1 ROS1 launch file description

5.3.1.1 All launch files

There are the following files in the src/launch folder:

localization.launch: Run file for visual positioning nodes
lx_camera_ros.launch: running file of ordinary nodes
mapping.launch: running file for visual positioning mapping
obstacle.launch: running file of obstacle avoidance algorithm
obstacleV2.launch: running file of obstacle avoidance algorithm V2
pallet.launch: pallet algorithm running file
sensor_sim.launch: running file for simulating positioning data

You can execute different functions of the node by running different launch files. Except for localization, mapping and sensor, the parameter contents are the same in all nodes, but the configuration is different. This will be introduced in detail in the following sections. For a detailed introduction to the parameters of the three launch files of localization, mapping and sensor, please see the introduction in the ros-v1pro node.

5.3.1.2 Camera IP, stream configuration, working mode configuration, point cloud unit, etc.

ip: The camera IP that needs to be connected. If it is 0, the first one will be connected by default.
log_path: SDK log storage path, the absolute path needs to be filled in
is_xyz: whether to push point cloud data
is_depth: whether to push depth data
is_amp: whether to push intensity map data
is_rgb: whether to push RGB image data
lx_work_mode: work mode
lx_application: whether to enable the algorithm, 0: off, 1: obstacle avoidance algorithm, 2: pallet algorithm, 3: positioning algorithm, 4: obstacle avoidance algorithm V2
lx_tof_unit: point cloud unit, 1 is meter, 0 is millimeter, needs to be changed to 1 when viewing with rviz

5.3.1.3 Camera pose configuration

The rotation angle and translation vector of the point cloud can be configured.

X: X translation amount of the camera from the center of the point cloud
Y: Y translation amount of the camera from the center of the point cloud
Z: Z translation amount of the camera from the center of the point cloud
Yaw: the angle of rotation of the point cloud around the Z axis
Roll: The angle of rotation of the point cloud around the X-axis
Pitch: The angle of rotation of the point cloud around the Y axis

5.3.1.4 Whether to use launch configuration

You can choose whether to use the camera parameter configuration in the launch file.

raw_param: 0: use the configuration in the camera, 1: use the configuration in the launch file

Regardless of whether this parameter is set to 0 or 1, all parameters before this parameter will be set, 0 and 1 are only valid for parameters after this parameter.

5.3.1.5 2D parameter configuration

lx_2d_binning：0：1x1，1：2x2，2：4x4
lx_2d_undistort: 2d anti-distortion, 0: off, 1: on
lx_2d_undistort_scale: 2D anti-distortion coefficient
lx_2d_auto_exposure: 2D automatic exposure, 0: off, 1: on
lx_2d_auto_exposure_value: 2D automatic exposure target intensity
lx_2d_exposure: 2D exposure value
lx_2d_gain: 2D gain

5.3.1.6 3D parameter configuration

lx_rgb_to_tof: RGB alignment
lx_3d_binning：0：1x1，1：2x2，2：4x4
lx_mulit_mode: multi-machine mode, which can eliminate multi-machine interference, 0: off, 1: on
lx_3d_undistort: 3d anti-distortion, 0: off, 1: on
lx_3d_undistort_scale: 3D anti-distortion coefficient
lx_hdr: hdr, 0: off, 1: on
lx_3d_auto_exposure: 3D automatic exposure, 0: off, 1: on
lx_3d_auto_exposure_value: 3D automatic exposure target intensity
lx_3d_first_exposure: 3D high points
lx_3d_second_exposure: 3D low points
lx_3d_gain: 3D gain

5.3.1.7 Maximum and minimum depth

lx_min_depth: minimum depth
lx_max_depth: maximum depth

5.3.2 ROS2 launch file description

5.3.2.1 All launch files

There are the following files in the src/launch folder:

localization.launch.py: running file of visual positioning node
lx_camera_ros.launch.py: running file of ordinary node
mapping.launch.py: running file for visual positioning mapping
obstacle.launch.py: running file of obstacle avoidance algorithm
obstacleV2.launch.py: running file of obstacle avoidance algorithm V2
pallet.launch.py: pallet algorithm running file
sensor_sim.launch.py: running file for simulating positioning data

You can execute different functions of the node by running different launch files. Except for localization, mapping and sensor, the parameter content is the same in all nodes, but the configuration is different. It will be introduced in detail in the following sections. For a detailed introduction to the parameters of the three launch files of localization, mapping and sensor, please see the introduction in the ros-v1pro node.

5.3.2.2 Camera IP, stream configuration, working mode configuration, point cloud unit, etc.

ip: The camera IP that needs to be connected. If it is 0, the first one will be connected by default.
log_path: SDK log storage path, the absolute path needs to be filled in
is_xyz: whether to push point cloud data
is_depth: whether to push depth data
is_amp: whether to push intensity map data
is_rgb: whether to push RGB image data
lx_work_mode: work mode
lx_application: whether to enable the algorithm, 0: off, 1: obstacle avoidance algorithm, 2: pallet algorithm, 3: positioning algorithm, 4: obstacle avoidance algorithm V2
lx_tof_unit: point cloud unit, 1 is meter, 0 is millimeter, needs to be changed to 1 when viewing with rviz

5.3.2.3 Camera pose configuration

The rotation angle and translation vector of the point cloud can be configured.

X: X translation amount of the camera from the center of the point cloud
Y: Y translation amount of the camera from the center of the point cloud
Z: Z translation amount of the camera from the center of the point cloud
Yaw: the angle of rotation of the point cloud around the Z axis
Roll: The angle of rotation of the point cloud around the X-axis
Pitch: The angle of rotation of the point cloud around the Y axis

5.3.2.4 Whether to use launch configuration

You can choose whether to use the camera parameter configuration in the launch file.

raw_param: 0: use the configuration in the camera, 1: use the configuration in the launch file

Regardless of whether this parameter is set to 0 or 1, all parameters before this parameter will be set, 0 and 1 are only valid for parameters after this parameter.

5.3.2.5 2D parameter configuration

lx_2d_binning：0：1x1，1：2x2，2：4x4
lx_2d_undistort: 2d anti-distortion, 0: off, 1: on
lx_2d_undistort_scale: 2D anti-distortion coefficient
lx_2d_auto_exposure: 2D automatic exposure, 0: off, 1: on
lx_2d_auto_exposure_value: 2D automatic exposure target intensity
lx_2d_exposure: 2D exposure value
lx_2d_gain: 2D gain

5.3.2.6 3D parameter configuration

lx_rgb_to_tof: RGB alignment
lx_3d_binning：0：1x1，1：2x2，2：4x4
lx_mulit_mode: multi-machine mode, which can eliminate multi-machine interference, 0: off, 1: on
lx_3d_undistort: 3d anti-distortion, 0: off, 1: on
lx_3d_undistort_scale: 3D anti-distortion coefficient
lx_hdr: hdr, 0: off, 1: on
lx_3d_auto_exposure: 3D automatic exposure, 0: off, 1: on
lx_3d_auto_exposure_value: 3D automatic exposure target intensity
lx_3d_first_exposure: 3D high points
lx_3d_second_exposure: 3D low points
lx_3d_gain: 3D gain

5.3.2.7 Maximum and minimum depth

lx_min_depth: minimum depth
lx_max_depth: maximum depth

5.3.3 Unique theme description

Since the unique topic types of ROS1 and ROS2 are the same, ROS1 is used as an example here. There are the following files in the src/msg folder:

Illustrations related to unique theme descriptions

5.3.3.1 FrameRate.msg

This node outputs frame rate information. The message format is as follows:

header: frame information and timestamp of the current data
depth: When the depth map is turned on, output the depth map frame rate
amp: When the intensity map is turned on, output the intensity map frame rate
rgb: When rgb is on, output rgb frame rate
temperature: output camera temperature

After the node starts running (except for nodes related to visual positioning), you can subscribe to the LxCamera_FrameRate node to obtain relevant information. The output is as follows:

5.3.3.2 Obstacle.msg

obstacle outputs information related to the obstacle avoidance algorithm results. The obstacle message format is as follows:

header: frame information and timestamp of the current data
status: status information returned by obstacle avoidance

0: Normal obstacle avoidance -1: No ground set -2: Input not set -3: No point cloud within the range -4: The point cloud is empty after filtering the ground -5: The point cloud is empty after radius filtering -6: No segmentation result was obtained -99: undefined error

io_output: Obstacle avoidance information output by the io interface

0: normal 1: warning 2: Shutdown

box_number: number of obstacles
box: obstacle information, variable length array, the length is determined by box_number

After the node starts running (the obstacle avoidance algorithm needs to be turned on), you can subscribe to the LxCamera_Obstacel node to obtain relevant information. The output is as follows (the data in the picture is simulated data):

5.3.3.3 Obstacle_box.msg

obstacle_box outputs obstacle-related information, and the obstacle_box message format is as follows:

center: the center of mass of the obstacle in the point cloud
rotation: rotation matrix of obstacle point cloud
translation: translation vector of obstacle point cloud
width: width of the obstacle frame
height: height of obstacle frame
depth: the depth of the obstacle frame

After starting the operation (the obstacle avoidance algorithm needs to be turned on), you can subscribe to the LxCamera_Obstacel node to obtain relevant information. The output is as follows:

5.3.3.4 Result.msg

The return value obtained after sending the command message to the servic in the node is stored in Result:

ret: the return value obtained by executing the SDK
msg: Description of the return value obtained by executing the SDK

5.3.3.5 Pallet.msg

The pallet outputs information related to the pallet algorithm results. The pallet message format is as follows:

header: frame information and timestamp of the current data
status: status information returned by pallet recognition:

0: Detection successful -1: No ground detected -2: Pallet legs not detected -3: The distance between the pallets is too far -4: unknown error

x: X distance between the tray and the optical center of the camera
y: Y distance between the tray and the optical center of the camera
yaw: theta angle between the center of the tray and the optical center of the camera

After the node starts running (the pallet algorithm needs to be turned on), you can subscribe to the LxCamera_Pallet node to obtain relevant information. The output is as follows:

5.3.4 Unique Service Description

Since the unique service types of ROS1 and ROS2 are the same, ROS1 is used as an example here. There are the following files in the src/srv folder:

Illustrations related to unique service descriptions

These messages are used to set or obtain certain parameter values of the camera. Above [---] is the sent data, below [---] is the returned data

5.3.4.1 LxBool.srv

cmd: the parameter number that needs to be obtained or set
val: the value set when setting
is_set: Whether to set the parameter. When it is true, the value in val will be set to the camera. When it is false, the value of this parameter will be read from the camera and displayed.
val: The value of the camera’s current parameter returned when getting or setting it
result: the value and information returned by calling the SDK interface

5.3.4.2 LxCmd.srv

cmd: The parameter number that needs to be obtained or set. When it is 0, all parameter numbers will be returned.
result: the value and information returned by calling the SDK interface, cmd may have multiple return values

5.3.4.3 LxFloat.srv

cmd: the parameter number that needs to be obtained or set
val: the value set when setting
is_set: Whether to set the parameter. When it is true, the value in val will be set to the camera. When it is false, the value of this parameter will be read from the camera and displayed.
max_value: the maximum value that can be set for this parameter
min_value: the minimum value that can be set for this parameter
cur_value: the current value of this parameter in the camera
available: whether this parameter supports setting
result: the value and information returned by calling the SDK interface

5.3.4.4 LxInt.srv

cmd: the parameter number that needs to be obtained or set
val: the value set when setting
is_set: Whether to set the parameter. When it is true, the value in val will be set to the camera. When it is false, the value of this parameter will be read from the camera and displayed.
max_value: the maximum value that can be set for this parameter
min_value: the minimum value that can be set for this parameter
cur_value: the current value of this parameter in the camera
available: whether this parameter supports setting
result: the value and information returned by calling the SDK interface

5.3.4.5 LxString.srv

cmd: the parameter number that needs to be obtained or set
val: the value set when setting
is_set: Whether to set the parameter. When it is true, the value in val will be set to the camera. When it is false, the value of this parameter will be read from the camera and displayed.
val: The value of the camera’s current parameter returned when getting or setting it
result: the value and information returned by calling the SDK interface

5.3.5 Description of topic subscription methods

5.3.5.1 Subscription frame rate, tray algorithm, obstacle avoidance algorithm output data

Subscribe Frame Rate In the lx_camera_node_ws directory, enter: bash rate.sh

After the lx_camera_node node image runs normally and is refreshed, you can see the following information:

Subscribe to pallet algorithm output In the lx_camera_node_ws directory, enter: bash pallet.sh

After the lx_camera_node node image runs normally and is refreshed, you can see the following information:

Subscribe to obstacle avoidance algorithm output In the lx_camera_node_ws directory, enter: bash obstacle.sh

After the lx_camera_node node image runs normally and is refreshed, you can see the following information:

5.3.5.2 Subscribe to image data (ROS1)

You can view the image information output by the node through the image tool that comes with ros: Console input command: rqt_image_view After entering the command, the following window will pop up:

Illustrations related to subscribing image data (ROS1)

The red box can select the images in all nodes. The picture shows the MRDVS camera RGB image. You can also select the depth map or intensity map (provided that the stream has been turned on):

In the figure, the first red box is the RGB information output by the positioning algorithm node, and the remaining three are image information output by the ordinary nodes. amp is the intensity map, depth is the depth map, and rgb is the rgb map. Select different items to view different images or use rviz to view image data. Enter the command on the console: rviz will pop up the following window:

After clicking the Add button at the bottom left, a window will pop up:

Select the desired image topic and select Image and click OK in the lower right corner. You can see the image data on the left side of the main interface, as shown below:

5.3.5.3 Subscribe to image data (ROS2)

You can view the image information output by the node through the image tool that comes with ros2: Enter the command on the console: rqt After entering the command, the following window will pop up:

Illustrations related to subscribing image data (ROS2)

Select the first Image View of the second option at the bottom of the menu bar. After selection, the following interface will pop up:

The red box can select the images in all nodes. The picture shows the MRDVS camera RGB image. You can also select the depth map or intensity map (provided that the stream has been turned on):

In the figure, the first red box is the RGB information output by the positioning algorithm node, and the remaining three are image information output by the ordinary nodes. amp is the intensity map, depth is the depth map, and rgb is the rgb map. Select different items to view different images or use rviz2 to view image data. Enter the command on the console: rviz2 will pop up the following window:

After clicking the Add button at the bottom left, a window will pop up:

Select the desired image topic and select Image and click OK in the lower right corner. You can see the image data on the left side of the main interface, as shown below:

Some systems need to fill in [mrdvs] in the [map] in the above picture to see the image

5.3.5.4 Subscribe to point cloud data

The point cloud information output by the node can be viewed through the point cloud tool provided by ros or ros2: Command: rviz【ROS1】, rviz2【ROS2】 After entering the command, the following window will pop up:

Illustrations related to subscribing to point cloud data

Click the button in the red box below and the following box will pop up:

Illustration related to subscribing to point cloud data 2

Select the second tab, and there will be a node named lx_camera_node. There is a topic named LxCamera_Cloud in the node. There is a message named PointCloud2 in the topic. After selecting, click OK. The interface after completion is as follows:

Illustrations related to subscribing to point cloud data 3

(ROS1 interface)

Illustrations related to subscribing to point cloud data 4

(ROS2 interface) At this time, the point cloud cannot be displayed normally. Fill in mrdvs in the red and green box map above to see the point cloud. The effect is as follows:

Illustrations related to subscribing to point cloud data 5

The default point cloud unit in ROS is meters, so when viewing using rviz, you need to modify lx_tof_unit in the launch file to 1 [the default is 1]

5.3.6 Description of service communication methods

In ROS1, all commands can be completed using the Tab key, but in ROS2, only topics can be completed, and the message part needs to be entered manually. When inputting data in ROS1, you can input it in the completion format. When inputting data in ROS2, you need to input it in the format of the yaml file. After the node is running, open a terminal under the lx_camera_node_ws folder, configure the environment in the terminal, and enter the command: source devel/setup.bash

5.3.6.1 Get the list of all parameter numbers (Lx_Cmd)

ros1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxCmd “cmd: 0”

You can see the output:

Get all parameter number list (Lx\_Cmd) related icon 2

ret is the number of the parameter. The msg describes the parameter type and description, for example: 1001. This parameter is of type int. The parameter is used to obtain and modify the first exposure value. ros2 Enter in the terminal: ros2 service call /LxCamera_LxCmd lx_camera_ros/srv/LxCmd ‘cmd: 0’

You can see the following output:

Get all parameter number list (Lx\_Cmd) related icon 4

Since it is not convenient to view parameter details in ROS2, the numbers, types and descriptions of all parameters are also listed in the following table:

Number	Type	Description in ROS	Detailed description
1001	INT	FIRST_EXPOSURE	Default high integral exposure value
1002	INT	SECOND_EXPOSURE	Default low integral exposure value
1003	INT	THIRD_EXPOSURE	Default medium integral exposure value
1004	INT	FOURTH_EXPOSURE	The fourth point
1005	INT	GAIN	Gain
1011	INT	MIN_DEPTH	Minimum depth value
1012	INT	MAX_DEPTH	Maximum depth value
1013	INT	MIN_AMPLITUDE	Minimum effective signal strength value
1014	INT	MAX_AMPLITUDE	Maximum effective signal strength value
1016	INT	CODE_MODE	Coding mode
1018	INT	WORK_MODE	Working mode
1019	INT	LINK_SPEED	Network card network speed
1021	INT	3D_IMAGE_WIDTH	3D image resolution width
1022	INT	3D_IMAGE_HEIGHT	3D image resolution height
1023	INT	3D_IMAGE_OFFSET_X	ROI horizontal offset pixels
1024	INT	3D_IMAGE_OFFSET_Y	ROI vertical offset pixels
1025	INT	3D_BINNING_MODE	3D image binnin
1026	INT	3D_DEPTH_DATA_TYPE	Depth image data format
1031	INT	3D_AMPLITUDE_CHANNEL	3D intensity image channel number
1032	INT	3D_AMPLITUDE_GET_TYPE	How to get intensity image
1033	INT	3D_AMPLITUDE_EXPOSURE	3D manual exposure value
1034	INT	3D_AMPLITUDE_INTENSITY	3D automatic target lighting
1035	INT	3D_AMPLITUDE_DATA_TYPE	Intensity image data format
1036	INT	3D_AUTO_EXPOSURE_LEVEL	3D automatic exposure level
1041	INT	2D_IMAGE_WIDTH	2D image resolution width
1042	INT	2D_IMAGE_HEIGHT	2D image high resolution
1043	INT	2D_IMAGE_OFFSET_X	2D image ROI horizontal offset
1044	INT	2D_IMAGE_OFFSET_Y	2D image ROI vertical offset
1045	INT	2D_BINNING_MODE	2D image binnin
1046	INT	2D_IMAGE_CHANNEL	Number of 2D image channels
1047	INT	2D_IMAGE_DATA_TYPE	2D image data format
1051	INT	2D_MANUAL_EXPOSURE	2D exposure value during manual exposure
1052	INT	2D_MANUAL_GAIN	2D gain during manual exposure
1053	INT	2D_ENCODE_TYPE	2D image compression format
1054	INT	2D_AUTO_EXPOSURE_LEVEL	2D image automatic exposure, etc.
1061	INT	TOF_GLOBAL_OFFSET	ToF depth data offset
1062	INT	3D_UNDISTORT_SCALE	ToF image anti-distortion coefficient
1065	INT	ALGORITHM_MODE	Set built-in application algorithm
1066	INT	MODBUS_ADDR	modbus address
1067	INT	HEART_TIME	Heartbeat time with the device
1068	INT	GVSP_PACKET_SIZE	GVSP single packet data packet size
1069	INT	TRIGGER_MODE	Trigger mode
1070	INT	CALCULATE_UP	Algorithm shift up and down
1072	INT	CAN_BAUD_RATE	baud rate value of can
1073	INT	MAX_3D_AUTO_EXPOSURE	3d automatic exposure limit
1074	INT	CUSTOM_PARAM_GROUP	User-defined parameter group
2001	FLOAT	FILTER_LEVEL	Filter level
2002	FLOAT	EST_OUT_EXPOSURE	Whether to evaluate overexposure data
2003	FLOAT	LIGHT_INTENSITY	light intensity
2004	FLOAT	3D_DEPTH_FPS	Current frame rate of depth map
2005	FLOAT	3D_AMPLITUDE_FPS	Intensity map current frame rate
2006	FLOAT	2D_IMAGE_FPS	RGB image current frame rate
2007	FLOAT	DEVICE_TEMPERATURE	Current camera temperature
3001	BOOL	CONNECT_STATE	Current connection status
3002	BOOL	ENABLE_3D_DEPTH_STREAM	Turn on/off depth data stream
3003	BOOL	ENABLE_3D_AMP_STREAM	Turn on/off intensity data stream
3006	BOOL	ENABLE_3D_AUTO_EXPOSURE	3D automatic exposure enable
3007	BOOL	ENABLE_3D_UNDISTORT	3D anti-distortion enable
3008	BOOL	ENABLE_ANTI_FLICKER	Anti-flicker enable
3011	BOOL	ENABLE_2D_STREAM	Turn on/off 2D data stream
3012	BOOL	ENABLE_2D_AUTO_EXPOSURE	2D automatic exposure enable
3015	BOOL	ENABLE_2D_UNDISTORT	2D image anti-distortion enabler
3016	BOOL	ENABLE_2D_TO_DEPTH	2D3D image alignment enable
3017	BOOL	ENABLE_BACKGROUND_AMP	Intensity background light enable
3018	BOOL	ENABLE_MULTI_MACHINE	Multi-machine mode enable
3019	BOOL	ENABLE_MULTI_EXPOSURE_HDR	HDR
3020	BOOL	ENABLE_SYNC_FRAME	Whether to enable forced frame synchronization
4001	STRING	DEVICE_VERSION	Device version number
4002	STRING	DEVICE_LOG_NAME	Log file name
4003	STRING	FIRMWARE_NAME	Firmware file name
4004	STRING	FILTER_PARAMS	Filter algorithm parameters
4005	STRING	ALGORITHM_PARAMS	Built-in algorithm parameters
4006	STRING	ALGORITHM_VERSION	Built-in algorithm version number
4007	STRING	DEVICE_OS_VERSION	Device system image version number
0	CMD	GET_PARAM_LIST	Get all parameter numbers
1	CMD	START_STREAM	Device startup
2	CMD	STOP_STREAM	Equipment stops flowing
5001	CMD	GET_NEW_FRAME	Update current latest data
5002	CMD	RETURN_VERSION	Roll back to the previous version
5003	CMD	RESTART_DEVICE	Restart camera
5004	CMD	WHITE_BALANCE	Auto white balance
5007	CMD	RESET_PARAM	Restore default parameters

Some parameters need to stop the flow before they can be set, or there is a sequence of settings. Please contact the technical support staff for specific needs.

5.3.6.2 Get Int parameter (Lx_Int)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxInt “cmd: 1001 val: 0 is_set: false”

You can see the output:

Get the related icon of Int parameter (Lx\_Int) 2

What is in the red box is the return value. What is obtained in the figure is the high integration time. According to the return result, you can see that the maximum value is 3600, the minimum value is 1, the current value is 1010, whether the setting is supported is supported, the interface call return value is 0, and the return value information is success.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxInt lx_camera_ros/srv/LxInt '{cmd: 1001,val: 0,is_set: false}'

You can see the output:

Get the related icon of Int parameter (Lx\_Int) 4

What is obtained in the figure is the high integration time. According to the returned results, we can see that the maximum value is 3600, the minimum value is 1, the current value is 1010, whether the setting is supported is supported, the return value of the interface call is 0, and the return value information is success.

5.3.6.3 Set Int parameters (Lx_Int)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxInt “cmd: 1001 val: 2000 is_set: true”

Illustrations related to setting Int parameters (Lx\_Int)

You can see the output:

Illustration 2 related to setting Int parameters (Lx\_Int)

What is in the red box is the return value. The high integration time is set in the figure. According to the return result, you can see that the maximum value is 3600, the minimum value is 1, the current value has been modified to 2000, whether the support setting is supported, the interface call return value is 0, and the return value information is success.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxInt lx_camera_ros/srv/LxInt '{cmd: 1001,val: 3000,is_set: true}'

You can see the output:

Illustration 4 related to setting Int parameters (Lx\_Int)

What is obtained in the figure is the high integration time. According to the returned results, we can see that the maximum value is 3600, the minimum value is 1, the current value has been modified to 3000, whether the setting is supported is supported, the interface call return value is 0, and the return value information is success.

5.3.6.4 Get Float parameters (Lx_Float)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxFloat “cmd: 2007 val: 0.0 is_set: false”

You can see the output:

Get the related icon of Float parameter (Lx\_Float) 2

What is in the red box is the return value. What is obtained in the picture is the camera temperature. According to the return result, you can see that the maximum value is 120, the minimum value is -40, the current value is 49.6875, whether the setting is supported is not supported, the interface call return value is 0, and the return value information is success.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxFloat lx_camera_ros/srv/LxFloat '{cmd: 2007,val: 0,is_set: false}'

You can see the output:

Get the related icon of Float parameter (Lx\_Float) 4

What is obtained in the picture is the camera temperature. According to the returned results, you can see that the maximum value is 120, the minimum value is -40, the current value is 50.4375, whether the setting is supported is not supported, the return value of the interface call is 0, and the return value information is success.

5.3.6.5 Set Float parameters (Lx_Float)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxFloat “cmd: 2007 val: 5.0 is_set: true”

Illustrations related to setting Float parameters (Lx\_Float)

You can see the output:

Illustration 2 related to setting Float parameters (Lx\_Float)

The red box is the return value. The camera temperature is set in the picture. According to the return result, you can see that the maximum value is 120, the minimum value is -40, the current value is 50.9375, whether the setting is supported is not supported, the interface call return value is -2, the return value information is not supported, the value set here is 5, the current value and the set value are inconsistent because the parameter does not support setting.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxFloat lx_camera_ros/srv/LxFloat '{cmd: 2007,val: 5.0,is_set: true}'

You can see the output:

Illustrations related to setting Float parameters (Lx\_Float) 4

The camera temperature is set in the picture. According to the returned results, you can see that the maximum value is 120, the minimum value is -40, the current value is 51.125, whether the setting is supported is not supported, the return value of the interface call is -2, the return value information is not supported, the value set here is 5, the current value and the set value are inconsistent because the parameter does not support setting.

5.3.6.6 Get Bool parameters (Lx_Bool)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxBool “cmd: 3012 val: false is_set: false”

Get related icons of Bool parameters (Lx\_Bool)

You can see the output:

Get Bool parameter (Lx\_Bool) related icon 2

What is in the red box is the return value. What is obtained in the picture is 2D automatic exposure. According to the return result, we can see that the current value is false, the return value of the interface call is 0, and the return value information is success.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxBool lx_camera_ros/srv/LxBool '{cmd: 3012,val: false,is_set: false}'

You can see the output:

Get Bool parameter (Lx\_Bool) related icon 4

What is obtained in the figure is 2D automatic exposure. According to the return result, we can see that the current value is false, the return value of the interface call is 0, and the return value information is success.

5.3.6.7 Set Bool parameters (Lx_Bool)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxBool “cmd: 3012 val: true is_set: true”

Illustrations related to setting Bool parameters (Lx\_Bool)

You can see the output:

Illustration 2 related to setting Bool parameters (Lx\_Bool)

What is in the red box is the return value. The 2D automatic exposure is set in the picture. According to the return result, we can see that the current value is true, the return value of the interface call is 0, the return value information is success, and the current value is consistent with the set value.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxBool lx_camera_ros/srv/LxBool '{cmd: 3012,val: true,is_set: true}'

You can see the output:

Illustrations related to setting Bool parameters (Lx\_Bool) 4

The setting in the picture is 2D automatic exposure. According to the return result, we can see that the current value is true, the return value of the interface call is 0, the return value information is success, and the current value is consistent with the set value.

5.3.6.8 Get String parameter (Lx_String)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxString “cmd: 4001 val: ” is_set: false”

Get related icons of String parameters (Lx\_String)

You can see the output:

Obtain String parameter (Lx\_String) related icon 2

What is in the red box is the return value. What is obtained in the figure is the device version number. According to the return result, we can see that the current value is V1.1.33_231201, the return value of the interface call is 0, and the return value information is success.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxString lx_camera_ros/srv/LxString '{cmd: 4001,val: '',is_set: false}'

You can see the output:

Get the related icon of String parameter (Lx\_String) 4

What is obtained in the figure is the device version number. According to the return result, we can see that the current value is V1.1.33_231201, the return value of the interface call is 0, and the return value information is success.

5.3.6.9 Set String parameters (Lx_String)

ROS1 Enter in the terminal: rosservice call /lx_camera_node/LxCamera_LxString “cmd: 4001 val: ‘put:\“123\”’ is_set: true”

Illustrations related to setting String parameters (Lx\_String)

You can see the output:

Illustration 2 related to setting String parameters (Lx\_String)

The red box is the return value. The device version number is set in the figure. According to the return result, we can see that the current value is V1.1.33_231201, the return value of the interface call is -2, and the return value information is not supported. The current value and the set value are inconsistent because this parameter does not support setting.
ROS2 Enter in the terminal: ros2 service call /LxCamera_LxString lx_camera_ros/srv/LxString '{cmd: 4001,val: "put:\\"123\\"",is_set: true}'

You can see the output:

Illustrations related to setting String parameters (Lx\_String) 4

The device version number is set in the figure. According to the returned result, we can see that the current value is V1.1.33_231201, the return value of the interface call is -2, and the return value information is not supported. The current value is inconsistent with the set value because this parameter does not support setting. When setting a value of String type, escape characters must be added to single quotes and double quotes, except for the initial double quotes in ROS2 and the initial single quotes in ROS1. For example, if you want to input the string [PUT”123”], you must add escape characters to become: [PUT\”123\”].

ROS recommended hardware and resource usage

6.1 Recommended hardware configuration

In the recommended hardware configuration, the analysis is based on CPU core count, CPU usage, and memory size. These are recommended configurations, not minimum requirements. If your hardware is below the recommendation, stable operation may still be possible by reducing stream output, disabling point-cloud computation, or lowering the algorithm load. Running a workload above the available configuration may crash the ROS core or Ubuntu. The second and third sections of this chapter provide detailed usage data. Choose the operating environment and stream configuration that best fits your system. All data here was collected in a VM using an M4 Pro camera. Ubuntu16.04

ROS version	CPU	Memory	Bandwidth
ros	AMD64 [Quad-core]	4G	1000M

Ubuntu18.04

ROS version	CPU	Memory	Bandwidth
ros	AMD64 [Quad-core]	4G	1000M
ros2	AMD64 [Quad-core]	4G	1000M

Ubuntu20.04

ROS version	CPU	Memory	Bandwidth
ros	AMD64 [Quad-core]	4G	1000M
ros2	AMD64 [Quad-core]	4G	1000M

Ubuntu22.04

ROS version	CPU	Memory	Bandwidth
ros	AMD64 [Quad-core]	4G	1000M

6.2 CPU details

The CPU details will show the amount of CPU resources occupied by the node under different operating conditions. The test is based on AMD64 [8 cores], 8G memory hardware environment, the test camera is M4 Pro, the test algorithm is tray algorithm, 2D width and height are 1280*960, and 3D width and height are 640*480, 2D anti-distortion is on, 3D anti-distortion is off, and integration 110 - 3000, depth 0 - 8000, ToF frame rate 14, streaming mode, smoothing level 1, noise level 1, time domain level 0, statistics command is TOP. Because the test soft environment is VM virtual machine, it is normal for the data displayed by the physical machine to fluctuate compared with that in the document. For some cameras, such as S2, some algorithms are calculated on the host computer, which results in a large amount of resource usage, which is normal.
1. Some nouns are explained as follows: -Single stream: refers to turning on any stream of RGB, depth, and intensity

Dual stream: refers to turning on any two streams of RGB, depth and intensity

-Three streams: refers to RGB, depth, and intensity all three streams are turned on

Four streams: means RGB, depth, intensity, and point cloud are all turned on

2. The algorithm has a binding relationship with some streams. The binding relationship is as follows:

To enable the pallet algorithm, the depth must be enabled
To turn on the obstacle avoidance algorithm, you must turn on depth
To turn on the positioning algorithm, you must turn on RGB

The light blue background in the table is the running configuration with point cloud enabled, and the colorless background is the running configuration without point cloud enabled. All the test data below have the algorithm shifted down by default, so for S2 series cameras, all test values may be too small [some algorithms of S2 series cameras are forced to move up]. Ubuntu16.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	10.59	0.0~26.7
Dual stream	14.13	0.0~60.0
Third rate	16.56	0.0~37.5
Fourth rate	108.25	93.8~162.5
Point cloud + depth	105.13	93.8~153.3
Algorithm + Depth	5.44	0.0~43.0
Algorithm + Depth + Point Cloud	104.39	93.8~120.0
Algorithm + Depth + RGB	13.15	6.2~40.0
Algorithm + depth + RGB + point cloud	105.96	93.8~146.7
Algorithm + depth + intensity + RGB + point cloud	109.29	93.8~162.5

Ubuntu18.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	10.31	0.0~27.5
Dual stream	12.81	5.9~27.8
Third rate	16.23	5.9~37.5
Fourth rate	105.95	88.2~131.2
Point cloud + depth	103.55	88.2~156.2
Algorithm + Depth	56.32	0.0~43.8
Algorithm + Depth + Point Cloud	104.07	78.3~43.8
Algorithm + Depth + RGB	15.22	5.9~70.6
Algorithm + depth + RGB + point cloud	104.56	93.8~125.0
Algorithm + depth + intensity + RGB + point cloud	105.52	61.5~204.8

ROS2
Run Configuration	Mean %	Interval %
Single stream	19.19	5.9~88.2
Dual stream	23.17	6.2~109.1
Third rate	30.59	9.1~125.0
Fourth rate	108.44	44.4~158.6
Point cloud + depth	102.19	37.5~143.9
Algorithm + Depth	6.44	0.0~26.7
Algorithm + Depth + Point Cloud	104.45	79.2~176.5
Algorithm + Depth + RGB	14.82	5.9~50.0
Algorithm + depth + RGB + point cloud	106.89	94.1~137.5
Algorithm + depth + intensity + RGB + point cloud	108.41	94.1~162.5

Ubuntu20.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	11.46	0.0~50.0
Dual stream	14.62	6.2~66.7
Third rate	18.46	6.2~50.0
Fourth rate	43.64	25.0~143.8
Point cloud + depth	28.37	12.5~75.0
Algorithm + Depth	6.53	0.0~43.8
Algorithm + Depth + Point Cloud	29.28	13.3~100
Algorithm+Depth+RGB	13.79	0.0~50.0
Algorithm + depth + RGB + point cloud	40.64	25.0~81.2
Algorithm+Depth+Intensity+RGB+Point Cloud	44.44	25.0~75.0

ROS2
Run Configuration	Mean %	Interval %
Single stream	8.07	0.0~20.0
Dual stream	11.25	0.0~33.3
Third rate	16.18	6.2~62.5
Fourth rate	33.77	0.0~86.7
Point cloud + depth	27.45	12.5~81.2
Algorithm + Depth	6.13	0.0~68.8
Algorithm + depth + point cloud	31.11	12.5~56.2
Algorithm+Depth+RGB	11.75	0.0~68.8
Algorithm + Depth + RGB + Point Cloud	38.04	6.2~75.0
Algorithm + depth + intensity + RGB + point cloud	40.85	18.8~106.2

Ubuntu22.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	8.05	0.0~23.5
Dual stream	11.99	0.0~75.0
Third rate	18.08	0.0~100
Fourth rate	50.06	6.2~200
Point cloud + depth	33.92	0.0~106.2
Algorithm + Depth	7.70	0.0~43.8
Algorithm + Depth + Point Cloud	36.20	6.2~131.2
Algorithm + Depth + RGB	14.46	6.2~37.5
Algorithm + Depth + RGB + Point Cloud	39.07	18.8~94.1
Algorithm + depth + intensity + RGB + point cloud	45.61	12.5~93.8

6.3 Memory details

The memory details will show the amount of memory resources occupied by the node under different operating conditions. The test is based on AMD64 [8 cores], 8G memory hardware environment, the test camera is M4 Pro, the test algorithm is tray algorithm, 2D width and height are 1280*960, and 3D width and height are 640*480, 2D anti-distortion is on, 3D anti-distortion is off, points 110 - 3000, depth 0 - 8000, ToF frame rate 14, streaming mode, smoothing level 1, noise level 1, time domain level 0, statistics command is TOP. Because the test soft environment is VM virtual machine, it is normal for the data displayed by the physical machine to fluctuate compared with that in the document. For some cameras, such as S2, some algorithms are calculated on the host computer, which results in a large amount of resource usage, which is normal.
1. Some nouns are explained as follows: -Single stream: refers to turning on any stream of RGB, depth, and intensity

Dual stream: refers to turning on any two streams of RGB, depth and intensity

-Three streams: refers to RGB, depth, and intensity all three streams are turned on

Four streams: means RGB, depth, intensity, and point cloud are all turned on

2. The algorithm has a binding relationship with some streams. The binding relationship is as follows:

To enable the pallet algorithm, the depth must be enabled
To turn on the obstacle avoidance algorithm, you must turn on depth
To turn on the positioning algorithm, you must turn on RGB

The light blue background in the table is the running configuration with point cloud enabled, and the colorless background is the running configuration without point cloud enabled. All the test data below have the algorithm shifted down by default, so for S2 series cameras, all test values may be too small [some algorithms of S2 series cameras are forced to move up].

Ubuntu16.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	0.60	0.6~0.7
Dual stream	0.61	0.6~0.8
Third rate	0.66	0.6~0.8
Fourth rate	0.79	0.7~0.9
Point cloud + depth	0.60	0.5~0.7
Algorithm + Depth	0.50	0.5~0.5
Algorithm + depth + point cloud	0.60	0.5~0.7
Algorithm + Depth + RGB	0.62	0.6~0.8
Algorithm + depth + RGB + point cloud	0.71	0.6~0.9
Algorithm + depth + intensity + RGB + point cloud	0.79	0.7~0.9

Ubuntu18.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	1.01	1.1~1.2
Dual stream	1.10	1.1~1.2
Third rate	1.10	1.1~1.2
Fourth rate	1.19	1.1~1.3
Point cloud + depth	1.09	0.9~1.2
Algorithm + Depth	0.90	0.9~1.0
Algorithm + depth + point cloud	1.09	0.9~1.2
Algorithm + Depth + RGB	1.10	1.1~1.2
Algorithm + depth + RGB + point cloud	1.19	1.1~1.3
Algorithm + depth + intensity + RGB + point cloud	1.20	1.1~1.4

ROS2
Run Configuration	Mean %	Interval %
Single stream	1.19	1.1~1.2
Dual stream	1.27	1.1~1.3
Third rate	1.29	1.2~1.3
Fourth rate	1.57	1.3~1.6
Point cloud + depth	1.20	1.1~1.3
Algorithm + Depth	0.90	0.9~0.9
Algorithm + depth + point cloud	1.10	1.0~1.2
Algorithm + Depth + RGB	1.10	1.1~1.2
Algorithm + depth + RGB + point cloud	1.30	1.3~1.4
Algorithm + depth + intensity + RGB + point cloud	1.41	1.3~1.6

Ubuntu20.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	1.21	1.2~1.4
Dual stream	1.30	1.3~1.4
Third rate	1.30	1.3~1.4
Fourth rate	1.33	1.3~1.5
Point cloud + depth	1.21	1.2~1.4
Algorithm + Depth	1.19	1.1~1.2
Algorithm + depth + point cloud	1.22	1.2~1.4
Algorithm + Depth + RGB	1.30	1.3~1.4
Algorithm + depth + RGB + point cloud	1.32	1.3~1.5
Algorithm + depth + intensity + RGB + point cloud	1.33	1.3~1.5

ROS2
Run Configuration	Mean %	Interval %
Single stream	1.20	1.2~1.3
Dual stream	1.20	1.2~1.3
Third rate	1.30	1.3~1.4
Fourth rate	1.64	1.4~1.7
Point cloud + depth	1.36	1.1~1.4
Algorithm + Depth	1.10	1.1~1.1
Algorithm + Depth + Point Cloud	1.19	1.1~1.3
Algorithm + Depth + RGB	1.40	1.4~1.4
Algorithm + depth + RGB + point cloud	1.58	1.4~1.7
Algorithm + depth + intensity + RGB + point cloud	1.60	1.5~1.7

Ubuntu22.04

ROS1
Run Configuration	Mean %	Interval %
Single stream	1.28	1.1~1.3
Dual stream	1.29	1.2~1.3
Third rate	1.20	1.2~1.3
Fourth rate	1.49	1.4~1.6
Point cloud + depth	1.11	1.1~1.2
Algorithm + Depth	1.00	1.0~1.0
Algorithm + Depth + Point Cloud	1.11	1.1~1.2
Algorithm + Depth + RGB	1.20	1.2~1.3
Algorithm + depth + RGB + point cloud	1.55	1.3~1.6
Algorithm + depth + intensity + RGB + point cloud	1.51	1.3~1.7

Appendix

This appendix provides installation methods for some tools. When using these methods, you need to have at least some basic knowledge of Linux. There is no guarantee that every computer can be successfully installed through the methods shown in the appendix of this document. The environment of each computer is different, and the problems encountered may not be consistent. If you encounter problems not shown in this appendix, please search for solutions yourself.

1. Cmake specified version installation method

Go to https://cmake.org/download/ and download the required CMake version. Extract the downloaded package with tar -zxvf cmake-XXX.tar.gz. Enter the extracted directory, then compile and configure it: ./bootstrap make sudo make install Check the Cmake version to see if the installation is successful: cmake —version

2. Opencv specified version installation method

Download the required version of opencv: Releases-OpenCV

Illustrations related to the installation method of the specified version of Opencv

Environment configuration, execute the following commands in sequence:

sudo apt-get install build-essential sudo apt-getinstallcmake gitlibgtk2.0-devpkg-config libavcodec-dev libavformat-dev libswscale-dev sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev

Create a new opencv folder and extract the downloaded source code into the opencv folder.
Execute the following commands in sequence:
cd opencv
mkdir build && cd build
sudocmake-DCMAKE_BUILD_TYPE=Release-D
CMAKE_INSTALL_PREFIX=/usr/local ..
sudo make -j4
sudo make install and wait for the installation to complete

3. Pcl specified version installation method

Check the supported pcl version:

apt-cache madison libpcl-dev

Install the specified pcl version (where 1.10.0 needs to be replaced with the required version number):

sudo apt-get install libpcl-dev=1.10.0

4. Install ROS

Ubuntu16.04

Add the software source and set the secret key:

sudo sh -c ‘echo “deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main” > /etc/apt/sources.list.d/ros-latest.list’ sudoapt-keyadv—keyserver’hkp://keyserver.ubuntu.com:80’—recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

Add secret key:

curl -sSL ‘http://keyserver.ubuntu.com/pks/lookup?op=get&search=0xC1CF6E31E6BADE8868B172B4F42ED6FBAB17C654’ | sudo apt-key add -

Update source:

sudo apt-get update

Install complete ros:

sudo apt-get install ros-kinetic-desktop-full

Configure environment variables:

echo “source /opt/ros/kinetic/setup.bash” >> ~/.bashrc source ~/.bashrc

Complete ROS-related tools:

sudoaptinstallpython-rosinstallpython-rosinstall-generatorpython-wstool build-essential

Now that the installation of ROS1 is complete, run the example that comes with ROS to check whether the installation is correct:

Open a terminal and enter: roscore Open another terminal and enter: rosrun turtlesim turtlesim_node Open another terminal: rosrun turtlesim turtle_teleop_key In the third terminal, you can control the little turtle through the arrow keys If it can run normally as shown below, it means that ros has been installed correctly:

Ubuntu18.04

Set up the software source:

sudo sh -c ’. /etc/lsb-release && echo “deb http://mirrors.ustc.edu.cn/ros/ubuntu/ `lsb_release -cs` main” > /etc/apt/sources.list.d/ros-latest.list’ 2.Set the public key: sudoapt-keyadv—keyserver’hkp://keyserver.ubuntu.com:80’—recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

Update software list:

sudo apt update

Install Ubuntu18.04 version of ROS1:

sudo apt install ros-melodic-desktop-full

Set environment variables:

echo “source /opt/ros/melodic/setup.bash” >> ~/.bashrc source ~/.bashrc if it is zsh: echo “source /opt/ros/melodic/setup.bash” >> ~/.zshrc source ~/.zshrc

Complete ROS-related tools:

sudoaptinstallpython-rosdeppython-rosinstallpython-rosinstall-generator python-wstool build-essential

Now that the installation of ROS1 is complete, run the example that comes with ROS to check whether the installation is correct:

Ubuntu20.04

Configure the public key:

sudoapt-keyadv—keyserver’hkp://keyserver.ubuntu.com:80’—recv-key C1CF6E31E6BADE8868B172B4F42ED6FBAB17C654

If it shows that it cannot connect to the server or times out, please update the software source list first:

sudo apt-get update sudo apt-get upgrade

After configuring the public key, update the software list again:

sudo apt-get update install ros: sudo apt install ros-noetic-desktop-full

Set environment variables:

echo “source /opt/ros/noetic/setup.bash” >> ~/.bashrc source ~/.bashrc

Install ROS tools:

sudo apt install python3-rosinstall python3-rosinstall-generator python3-wstool build-essential

Now that the installation of ROS1 is complete, run the example that comes with ROS to check whether the installation is correct:

Ubuntu22.04 It is not recommended to install ros1 in Ubuntu22.04. It is not officially supported. If you want to install it, you will encounter a lot of dependency problems.

5. How to create a new test package in Ros

Create a new folder ws, create a new src folder in the ws folder, enter the src folder and execute:

catkin_create_pkg example std_msgs rospy roscpp

This creates a new package named example, which depends on std_msgs rospy roscpp
Then compile this new package: catkin_make This is an empty package, there will not be any executable program, it is just a package
Next, we will demonstrate how to create a package that can output ‘Hellow world’
Based on the package just now, we enter the src folder (src in the package) and create a new main.cpp, as shown below (pay attention to the path):

Illustration of how to create a new test package in Ros

In main.cpp, we enter some codes:

Ros How to Create a New Test Package Related Illustration 2

Then we change the outside CMakeLists.txt and add a line at the bottom: add_executable(example src/main.cpp)

Ros How to Create a New Test Package Related Illustration 3

Then return to the ws directory to compile and execute the command: catkin_make

Ros How to Create a New Test Package Related Illustration 4

Seeing these means the compilation is successful.
Next we configure the environment:

source devel/setup.bash

Ros How to Create a New Test Package Related Illustrations 5

Then execute the program and you can see the output:

rosrun example example

Ros How to Create a New Test Package Related Illustrations 6

6. Install ROS2

Ubuntu16.04

Add secret key:

sudo apt-get update && sudo apt-get install curl curl http://repo.ros2.org/repos.key | sudo apt-key add -

Add permissions:

sudo apt update && sudo apt install curl gnupg2 lsb-release curl-shttps://raw.githubusercontent.com/ros/rosdistro/master/ros.asc|sudo apt-key add -

If an error is reported in this step, use VPN to bypass the wall and then open the browser, enter the URL, and copy the content, then create a new ros.asc and copy the content to the file. After the copy is completed, the following is as follows:
Then re-execute step 3: sudo apt-key add ros.asc
Continue execution:

sudo sh -c ‘echo “deb [arch=amd64] http://packages.ros.org/ros2/ubuntu bionic main” > /etc/apt/sources.list.d/ros2-latest.list’

Install ROS2:

sudo apt update sudo apt install ros-ardent-desktop

Configure environment variables:

source /opt/ros/ardent/setup.bash

Install the function package:

sudo apt-get update sudo apt-get install ros-ardent-ros1-bridge sudo apt-get install ros-ardent-turtlebot2-*

Try running the example program, open a terminal and run:

source ~/.bashrc ros2 run demo_nodes_cpp talker

Open another interrupt to run:

source~/.bashrc ros2 run demo_nodes_cpp talker

The following terminal output proves that the installation is successful:

Ubuntu18.04：

Add software source

sudo apt update && sudo apt install curl gnupg2 lsb-release curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc sudo apt-key add ros.asc

If this step reports an error as follows:
Use VPN to bypass the wall, then open the browser, enter the URL, and copy the content, then create a new ros.asc and copy the content to the file. After the copy is completed, it is as follows:
Then re-execute step 3: sudo apt-key add ros.asc
Continue execution:

sudo sh -c ‘echo “deb [arch=amd64] http://packages.ros.org/ros2/ubuntu bionic main” > /etc/apt/sources.list.d/ros2-latest.list’

After completion, start downloading ros2, in which the red fonts need to be replaced with different versions (see the table above):

sudo apt update sudo apt install ros-eloquent-desktop

Wait for the installation to complete and continue to install the python library. Execute: sudo apt install -y libpython3-dev python3-pip pip3 install -U argcomplete

Then configure the environment variables: echo “source /opt/ros/eloquent/setup.bash” >> ~/.bashrc source ~/.bashrc

Try running the sample program, open a terminal and run:

source ~/.bashrc ros2 run demo_nodes_cpp talker

Open another interrupt to run:

source ~/.bashrc ros2 run demo_nodes_py listener

The following terminal output proves that the installation is successful:

Ubuntu20.04

Add software source

sudo apt update && sudo apt install curl gnupg2 lsb-release sudo curl -sSL [https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o](https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o) /usr/share/keyrings/ros-archive-keyring.gpg

If you encounter the error “ailed to connect to raw.githubusercontent.com port 443 after 13 ms: connection refused”

sudo vi /etc/hosts

Add the following analysis

185.199.108.133raw.githubusercontent.com

The added files are as follows:

echo”deb[arch=(source/etc/os-release&&echo$UBUNTU_CODENAME) main” | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

Continue execution:

sudo sh -c ‘echo “deb [arch=amd64] http://packages.ros.org/ros2/ubuntu bionicmain” > /etc/apt/sources.list.d/ros2-latest.list’

After completion, start downloading ros2:

sudo apt update sudo apt install ros-foxy-desktop

Wait for the installation to complete and continue to install the python library. Execute: sudo apt install -y libpython3-dev python3-pip pip3 install -U argcomplete
Then configure the environment variables:

echo “source /opt/ros/foxy/setup.bash” >> ~/.bashrc source ~/.bashrc

Try running the sample program, open a terminal and run:

source ~/.bashrc ros2 run demo_nodes_cpp talker

Open another interrupt to run:

source ~/.bashrc ros2 run demo_nodes_py listener

The following terminal output proves that the installation is successful:

Ubuntu22.04

Add software source

sudo apt update && sudo apt install curl gnupg lsb-release sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpgecho”deb[arch=(source/etc/os-release&&echo$UBUNTU_CODENAME) main” | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

If you encounter the error “ailed to connect to raw.githubusercontent.com port 443 after 13 ms: connection refused”

sudo vi /etc/hosts adds the following analysis 185.199.108.133raw.githubusercontent.com The added files are as follows:

After completion, start downloading ros2: sudo apt update

sudo apt install ros-humble-desktop

Wait for the installation to complete and continue to install the python library. Execute: sudo apt install -y libpython3-dev python3-pip pip3 install -U argcomplete
Then configure the environment variables:

echo “source /opt/ros/humble/setup.bash” >> ~/.bashrc source ~/.bashrc

Try running the sample program, open a terminal and run:

source ~/.bashrc ros2 run demo_nodes_cpp talker

Open another interrupt to run:

source ~/.bashrc ros2 run demo_nodes_py listener

The following terminal output proves that the installation is successful:

7. How to create a new test package in Ros2

Execute the command:

mkdir example_ws && cd example_ws ros2 pkg create —build-type ament_cmake —node-name example_n example_p colcon build source install/setup.bash ros2 run example_p example_n

You can create a new package named example_p and a node named example_n. After running, it will output: hello world example_p package

​Overview

​System and environment

​Compatible systems and dependency versions

​2.2 Check the system version and application version

​2.2.1 Check the system version

​2.2.2 Check Cmake version

​2.2.3 Check Gcc version

​2.2.4 View Opencv version

​2.2.5 Check Pcl version

​2.2.6 View ROS version

​2.3 Build a basic development environment

​2.3.1 Install Cmake

​2.3.2 Install Gcc

​2.4 Build optional development environment

​2.4.1 Install Opencv [can display camera images]

​2.4.2 Install Pcl [can display point cloud]

​Prepare the sample code

​3.1 Execute the installation script

​3.2 Sample code introduction

​3.2.1 C and C++ sample code

​3.2.2 Java sample code

​3.2.3 ROS sample code

​3.2.4 ROS2 sample code

​3.2.5 ros-v1pro sample code

​Compile and run the samples

​4.1 Compiling and running C and C++ sample code

​4.2 java sample running process

​4.3 Compiling and running ROS sample code

​4.4 Compiling and running ROS2 sample code

​4.5 Compilation and operation of ros-v1pro sample code

​Sample code reference

​5.1 C and C++

​5.1.1 CMakeLists

​5.1.2 application_obstacle and application_pallet

​5.1.3 arm_local_camera

​5.1.4 frame_callback

​5.1.5 multi_cameras

​5.1.6 single_camera

​5.1.7 single_camera2

​5.2 java

​5.3 ROS1 & ROS2

​5.3.1 ROS1 launch file description

5.3.1.1 All launch files

5.3.1.2 Camera IP, stream configuration, working mode configuration, point cloud unit, etc.

5.3.1.3 Camera pose configuration

5.3.1.4 Whether to use launch configuration

5.3.1.5 2D parameter configuration

5.3.1.6 3D parameter configuration

5.3.1.7 Maximum and minimum depth

​5.3.2 ROS2 launch file description

5.3.2.1 All launch files

5.3.2.2 Camera IP, stream configuration, working mode configuration, point cloud unit, etc.

5.3.2.3 Camera pose configuration

5.3.2.4 Whether to use launch configuration

5.3.2.5 2D parameter configuration

5.3.2.6 3D parameter configuration

5.3.2.7 Maximum and minimum depth

​5.3.3 Unique theme description

5.3.3.1 FrameRate.msg

5.3.3.2 Obstacle.msg

5.3.3.3 Obstacle_box.msg

5.3.3.4 Result.msg

5.3.3.5 Pallet.msg

​5.3.4 Unique Service Description

5.3.4.1 LxBool.srv

5.3.4.2 LxCmd.srv

5.3.4.3 LxFloat.srv

5.3.4.4 LxInt.srv

5.3.4.5 LxString.srv

​5.3.5 Description of topic subscription methods

5.3.5.1 Subscription frame rate, tray algorithm, obstacle avoidance algorithm output data

5.3.5.2 Subscribe to image data (ROS1)

5.3.5.3 Subscribe to image data (ROS2)

5.3.5.4 Subscribe to point cloud data

​5.3.6 Description of service communication methods

5.3.6.1 Get the list of all parameter numbers (Lx_Cmd)

5.3.6.2 Get Int parameter (Lx_Int)

5.3.6.3 Set Int parameters (Lx_Int)

5.3.6.4 Get Float parameters (Lx_Float)

5.3.6.5 Set Float parameters (Lx_Float)

Overview

System and environment

Compatible systems and dependency versions

2.2 Check the system version and application version

2.2.1 Check the system version

2.2.2 Check Cmake version

2.2.3 Check Gcc version

2.2.4 View Opencv version

2.2.5 Check Pcl version

2.2.6 View ROS version

2.3 Build a basic development environment

2.3.1 Install Cmake

2.3.2 Install Gcc

2.4 Build optional development environment

2.4.1 Install Opencv [can display camera images]

2.4.2 Install Pcl [can display point cloud]

Prepare the sample code

3.1 Execute the installation script

3.2 Sample code introduction

3.2.1 C and C++ sample code

3.2.2 Java sample code

3.2.3 ROS sample code

3.2.4 ROS2 sample code

3.2.5 ros-v1pro sample code

Compile and run the samples

4.1 Compiling and running C and C++ sample code

4.2 java sample running process

4.3 Compiling and running ROS sample code

4.4 Compiling and running ROS2 sample code

4.5 Compilation and operation of ros-v1pro sample code

Sample code reference

5.1 C and C++

5.1.1 CMakeLists

5.1.2 application_obstacle and application_pallet

5.1.3 arm_local_camera

5.1.4 frame_callback

5.1.5 multi_cameras

5.1.6 single_camera

5.1.7 single_camera2

5.2 java

5.3 ROS1 & ROS2

5.3.1 ROS1 launch file description

5.3.2 ROS2 launch file description

5.3.3 Unique theme description

5.3.4 Unique Service Description

5.3.5 Description of topic subscription methods

5.3.6 Description of service communication methods

ROS recommended hardware and resource usage

6.1 Recommended hardware configuration

6.2 CPU details

6.3 Memory details

Appendix

1. Cmake specified version installation method

2. Opencv specified version installation method

3. Pcl specified version installation method

4. Install ROS

5. How to create a new test package in Ros

6. Install ROS2

7. How to create a new test package in Ros2