Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

D435i can't be found, ubuntu22.04 jammy(core 5.10), ROS2 humble #13611

Open
Frank198002 opened this issue Dec 17, 2024 · 11 comments
Open

D435i can't be found, ubuntu22.04 jammy(core 5.10), ROS2 humble #13611

Frank198002 opened this issue Dec 17, 2024 · 11 comments

Comments

@Frank198002
Copy link

Frank198002 commented Dec 17, 2024

  • Before opening a new issue, we wanted to provide you with some useful suggestions (Click "Preview" above for a better view):

  • All users are welcomed to report bugs, ask questions, suggest or request enhancements and generally feel free to open new issue, even if they haven't followed any of the suggestions above :)


Required Info
Camera Model {D400 }
Firmware Version (5.16)
Operating System & Version {Linux (Ubuntu 22.04)
Kernel Version (Linux Only) (5.10.110-rockchip-rk3588)
Platform OrangePi 5B
SDK Version {2.56.3 }
Language {C/python }
Segment {Robot}

Issue Description

When install SDK 2.0, I select "Option 3: Build from source". Follow the instructions under [Linux Installation] and Install librealsense2, on step 3: Build and apply patched kernel modules for:

11111WechatIMG370

My system is Ubuntu 22.04 jammy ,but core is 5.10, so I can't run patch script on this kernel. Could you give me some suggestion and I have another kernel version is 6.1, but still not in list.

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 17, 2024

Hi @Frank198002 It is recommended that on Rockchip (RK) devices, the librealsense SDK is built from source code with the libuvc backend installation method. A libuvc build of the SDK bypasses the kernel and so can avoid problems that may occur on a kernel-patched installation of librealsense.

https://github.com/IntelRealSense/librealsense/blob/master/doc/libuvc_installation.md

The libuvc backend installation method provides a pre-made build script called 'libuvc_installation.sh'.

https://github.com/IntelRealSense/librealsense/blob/master/scripts/libuvc_installation.sh

If you need your librealsense installation to include the realsense-viewer and other tools and examples then please change line 46 of the script from this:

cmake ../ -DFORCE_LIBUVC=true -DCMAKE_BUILD_TYPE=release

to this:

cmake ../ -DFORCE_LIBUVC=true -DCMAKE_BUILD_TYPE=release -DBUILD_EXAMPLES=true -DBUILD_GRAPHICAL_EXAMPLES=true

After installing librealsense with the libuvc backend method, please build the RealSense ROS2 wrapper from source code again with colcon build.

@Frank198002
Copy link
Author

Frank198002 commented Dec 19, 2024

Hi @MartyG-RealSense, thank you for your guider on on Rockchip (RK) devices! I have installed SDK successfully, and can use RealSense -viewer to see RGB,Stereo and Motion.

But after I connect D435i to usb3.0, and run verity command:
orangepi@orangepi5b:~/ros2_ws$ rs-enumerate-devices
No device detected. Is it plugged in?

And build the RealSense ROS2 wrapper from source code again with colcon build(option 2). But when we run ros node, can't find the device, info as following:

orangepi@orangepi5b:~/ros2_ws$ ros2 launch realsense2_camera rs_launch.py
[INFO] [launch]: All log files can be found below /home/orangepi/.ros/log/2024-12-19-15-53-52-617801-orangepi5b-23589
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [23591]
[realsense2_camera_node-1] [INFO] [1734594833.252505860] [camera.camera]: RealSense ROS v4.55.1
[realsense2_camera_node-1] [INFO] [1734594833.252671525] [camera.camera]: Built with LibRealSense v2.55.1
[realsense2_camera_node-1] [INFO] [1734594833.252701566] [camera.camera]: Running with LibRealSense v2.55.1
[realsense2_camera_node-1] [WARN] [1734594833.272007029] [camera.camera]: No RealSense devices were found!
[realsense2_camera_node-1] [WARN] [1734594839.289625687] [camera.camera]: No RealSense devices were found!
[realsense2_camera_node-1] [WARN] [1734594845.308464164] [camera.camera]: No RealSense devices were found!
[realsense2_camera_node-1] [WARN] [1734594851.331298381] [camera.camera]: No RealSense devices were found!
[realsense2_camera_node-1] [WARN] [1734594857.350718183] [camera.camera]: No RealSense devices were found!

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 19, 2024

Does it make a differene if you launch rs-enumerate-devices in sudo admin permissions mode with sudo rs-enumerate-devices

Ideally, the ROS wrapper should be the only program installed in the /ros2_ws catkin workspace directory and librealsense should be installed in another directory. The quoted command /ros2_ws$ rs-enumerate-devices indicates that librealsense is installed in the ROS catkin workspace directory.

@Frank198002
Copy link
Author

Frank198002 commented Dec 20, 2024

Here is the new system, realsense is the first application.
librealsense dir: ~/librealsense_build/librealsense-master/
ROS wrapper dir: ~/ros2_ws/realsense-ros

Now realsense-ros is the unique dir in ros2_ws/src as following:

orangepi@orangepi5b:~/ros2_ws/src$ ls
realsense-ros

Thank you for your suggestion! We can see info with "sudo". And we can also find device.
But new problem is IMU info missing not only in RealSense-viewer but also in ros2 topics:
20241220Realsense-viewer

orangepi@orangepi5b:~/ros2_ws$ sudo rs-enumerate-devices
Device info: 
    Name                          : 	Intel RealSense D435I
    Serial Number                 : 	233522071798
    Firmware Version              : 	5.16.0.1
    Recommended Firmware Version  : 	5.16.0.1
    Physical Port                 : 	7-1-9
    Debug Op Code                 : 	15
    Advanced Mode                 : 	YES
    Product Id                    : 	0B3A
    Camera Locked                 : 	YES
    Usb Type Descriptor           : 	2.1
    Product Line                  : 	D400
    Asic Serial Number            : 	226523132791
    Firmware Update Id            : 	226523132791
    Dfu Device Path               : 


orangepi@orangepi5b:~/ros2_ws$ source install/setup.bash 
orangepi@orangepi5b:~/ros2_ws$ ros2 launch realsense2_camera rs_launch.py initial_reset:=true
[INFO] [launch]: All log files can be found below /home/orangepi/.ros/log/2024-12-20-10-08-27-774495-orangepi5b-17605
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [17615]
[realsense2_camera_node-1] [INFO] [1734671481.597811205] [camera.camera]: RealSense ROS v4.55.1
[realsense2_camera_node-1] [INFO] [1734671481.597943328] [camera.camera]: Built with LibRealSense v2.55.1
[realsense2_camera_node-1] [INFO] [1734671481.597963744] [camera.camera]: Running with LibRealSense v2.55.1
[realsense2_camera_node-1] [INFO] [1734671482.141965086] [camera.camera]: Device with serial number 233522071798 was found.
[realsense2_camera_node-1] 
[realsense2_camera_node-1] [INFO] [1734671482.142244208] [camera.camera]: Device with physical ID 7-1-5 was found.
[realsense2_camera_node-1] [INFO] [1734671482.142357373] [camera.camera]: Device with name Intel RealSense D435I was found.
[realsense2_camera_node-1] [INFO] [1734671482.145226756] [camera.camera]: Device with port number 7-1 was found.
[realsense2_camera_node-1] [INFO] [1734671482.145326963] [camera.camera]: Device USB type: 2.1
[realsense2_camera_node-1] [WARN] [1734671482.145410671] [camera.camera]: Device 233522071798 is connected using a 2.1 port. Reduced performance is expected.
[realsense2_camera_node-1] [INFO] [1734671482.145756000] [camera.camera]: getParameters...
[realsense2_camera_node-1] [INFO] [1734671482.147072568] [camera.camera]: JSON file is not provided
[realsense2_camera_node-1] [INFO] [1734671482.147172025] [camera.camera]: Device Name: Intel RealSense D435I
[realsense2_camera_node-1] [INFO] [1734671482.147254274] [camera.camera]: Device Serial No: 233522071798
[realsense2_camera_node-1] [INFO] [1734671482.147332732] [camera.camera]: Device physical port: 7-1-5
[realsense2_camera_node-1] [INFO] [1734671482.147396314] [camera.camera]: Device FW version: 5.16.0.1
[realsense2_camera_node-1] [INFO] [1734671482.147470105] [camera.camera]: Device Product ID: 0x0B3A
[realsense2_camera_node-1] [INFO] [1734671482.147528146] [camera.camera]: Sync Mode: Off
[realsense2_camera_node-1] [INFO] [1734671485.711045796] [camera.camera]: Set ROS param depth_module.depth_profile to default: 640x480x15
[realsense2_camera_node-1] [INFO] [1734671485.719664154] [camera.camera]: Set ROS param depth_module.infra_profile to default: 640x480x15
[realsense2_camera_node-1] [INFO] [1734671485.943373094] [camera.camera]: Set ROS param rgb_camera.color_profile to default: 640x480x15
[realsense2_camera_node-1] [INFO] [1734671485.945513903] [camera.camera]: Set ROS param gyro_fps to default: 200
[realsense2_camera_node-1] [INFO] [1734671485.945972981] [camera.camera]: Set ROS param accel_fps to default: 100
[realsense2_camera_node-1] [INFO] [1734671485.956181196] [camera.camera]: Stopping Sensor: Depth Module
[realsense2_camera_node-1] [INFO] [1734671485.956683148] [camera.camera]: Stopping Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1734671486.037581457] [camera.camera]: Starting Sensor: Depth Module
[realsense2_camera_node-1] [INFO] [1734671486.146406859] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 640, Height: 480, FPS: 15
[realsense2_camera_node-1] [INFO] [1734671486.148946663] [camera.camera]: Starting Sensor: RGB Camera
[realsense2_camera_node-1]  20/12 13:11:26,149 WARNING [547641616608] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
[realsense2_camera_node-1]  20/12 13:11:26,160 WARNING [547641616608] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
[realsense2_camera_node-1] [INFO] [1734671486.161167937] [camera.camera]: Open profile: stream_type: Color(0), Format: RGB8, Width: 640, Height: 480, FPS: 15
[realsense2_camera_node-1] [INFO] [1734671486.166360418] [camera.camera]: RealSense Node Is Up!

Another terminal is

orangepi@orangepi5b:~$ ros2 topic list
/camera/camera/color/camera_info
/camera/camera/color/image_raw
/camera/camera/color/metadata
/camera/camera/depth/camera_info
/camera/camera/depth/image_rect_raw
/camera/camera/depth/metadata
/camera/camera/extrinsics/depth_to_color
/camera/camera/extrinsics/depth_to_depth
/parameter_events
/rosout
/tf_static

@MartyG-RealSense
Copy link
Collaborator

MartyG-RealSense commented Dec 20, 2024

Please try enabling only Motion Module on its own in realsense-viewer to see whether the IMU data appears. There is a known issue with realsense-viewer for some RealSense users where if you enable three streams and one of them is the Motion Module (such as Depth, RGB and Motion Module), one of the streams will not provide data. For users of realsense-viewer who experience this problem, there is no fix for this except to only enable Motion Module on its own, or only enable two streams (Depth + Motion Module or RGB + Motion Module).

In the ROS wrapper, the IMU topics are disabled by default unless you add the parameters below to your ros2 launch instruction to publish them.

enable_accel:=true enable_gyro:=true unite_imu_method:=2

@Frank198002
Copy link
Author

I only open IMU stream in viewer as you said, but still can't see IMU image. I remmber when I first install camera,which firmware is 5.13, realsense-viewer can show RGB, depth and IMU at the same time.

a. Here is the error log when i only open IMU in realsense-viewer.
IMU_error
At first, there is no stream, and wait for a while, it will report "control-transfer returned error "

b. then I close realsense-viewer, resatrt system, and us ROS2 command to run package as following:

ros2 launch realsense2_camera rs_launch.py initial_reset:=true enable_accel:=true enable_gyro:=true unite_imu_method:=2

********
[realsense2_camera_node-1]  22/12 16:34:06,441 WARNING [547440290016] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
then I check topics as followings:
orangepi@orangepi5b:~/ros2_ws$ ros2 topic list
/camera/camera/accel/imu_info
/camera/camera/accel/metadata
/camera/camera/accel/sample
/camera/camera/color/camera_info
/camera/camera/color/image_raw
/camera/camera/color/metadata
/camera/camera/depth/camera_info
/camera/camera/depth/image_rect_raw
/camera/camera/depth/metadata
/camera/camera/extrinsics/depth_to_accel
/camera/camera/extrinsics/depth_to_color
/camera/camera/extrinsics/depth_to_gyro
/camera/camera/gyro/imu_info
/camera/camera/gyro/metadata
/camera/camera/gyro/sample
/camera/camera/imu
/parameter_events
/rosout
/tf_static
orangepi@orangepi5b:~/ros2_ws$ ros2 topic echo /camera/camera/color/image_raw

orangepi@orangepi5b:~/ros2_ws$ ^C
orangepi@orangepi5b:~/ros2_ws$ ros2 topic echo /camera/camera/color/metadata
The message type 'realsense2_camera_msgs/msg/Metadata' is invalid
orangepi@orangepi5b:~/ros2_ws$ ros2 topic echo /camera/camera/color/camerainfo
WARNING: topic [/camera/camera/color/camerainfo] does not appear to be published yet
Could not determine the type for the passed topic

I can't find the topic output. But it is surprise that I close ros2 node and open realsense-view, I can see RGB and depth video. My request is receiving RGB and IMU ros stream from this camera, if depth ok, it will be best!

@MartyG-RealSense
Copy link
Collaborator

At #13611 (comment) earlier in this discussion you said that you could use realSense-viewer to see RGB, Stereo and Motion Module. But when you plugged the camera into USB 3.0 on your Orange Pi, that's when your problems occurred.

Looking at information about Orange Pi 5b, it has 1 USB 3.0 port and 1 USB 3.1 Gen 1 port (plus two USB 2 ports). It is highly recommendable to ensure that the camera is in the USB 3.1 port. This is because RealSense 400 Series cameras only minimally supported USB 3.0 even when they were first launched in December 2017. They should be used with USB 3.1 Gen 1.

@Frank198002
Copy link
Author

Yes, before install D435i, i read the relative issue about USB3.0; OrangePi5B has 2 USB2.0 and 1 USB3.0. I purchase one USB3.0 wire to connect.
When I use release-viewer , I will connect mouse and keyboard to One usb2.0 ,and D435i to USB3.0(Viewer display "usb3.2"), it will shows "control-transfer returned error " about once in ten messages.
But when I use ros2 command, I remote ssh OP5B and USB3.0 for D435i, both of USB2.0 ports are empty. I will receive a lot of "control-transfer returned error ".

$ ros2 launch realsense2_camera rs_launch.py initial_reset:=true enable_accel:=true enable_gyro:=true unite_imu_method:=2
[INFO] [launch]: All log files can be found below /home/orangepi/.ros/log/2024-12-22-23-32-12-001443-orangepi5b-32320
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [realsense2_camera_node-1]: process started with pid [32321]
[realsense2_camera_node-1] [INFO] [1734881532.212810785] [camera.camera]: RealSense ROS v4.55.1
[realsense2_camera_node-1] [INFO] [1734881532.213191988] [camera.camera]: Built with LibRealSense v2.55.1
[realsense2_camera_node-1] [INFO] [1734881532.213285028] [camera.camera]: Running with LibRealSense v2.55.1
[realsense2_camera_node-1] [INFO] [1734881532.749153374] [camera.camera]: Device with serial number 233522071798 was found.
[realsense2_camera_node-1] 
[realsense2_camera_node-1] [INFO] [1734881532.749326622] [camera.camera]: Device with physical ID 6-1-11 was found.
[realsense2_camera_node-1] [INFO] [1734881532.749384954] [camera.camera]: Device with name Intel RealSense D435I was found.
[realsense2_camera_node-1] [INFO] [1734881532.751045389] [camera.camera]: Device with port number 6-1 was found.
[realsense2_camera_node-1] [INFO] [1734881532.751130846] [camera.camera]: Device USB type: 3.2
[realsense2_camera_node-1] [INFO] [1734881532.751182470] [camera.camera]: Resetting device...
[realsense2_camera_node-1] [INFO] [1734881539.474003869] [camera.camera]: Device with serial number 233522071798 was found.
[realsense2_camera_node-1] 
[realsense2_camera_node-1] [INFO] [1734881539.474169241] [camera.camera]: Device with physical ID 6-1-12 was found.
[realsense2_camera_node-1] [INFO] [1734881539.474213283] [camera.camera]: Device with name Intel RealSense D435I was found.
[realsense2_camera_node-1] [INFO] [1734881539.475553472] [camera.camera]: Device with port number 6-1 was found.
[realsense2_camera_node-1] [INFO] [1734881539.475606846] [camera.camera]: Device USB type: 3.2
[realsense2_camera_node-1] [INFO] [1734881539.475850676] [camera.camera]: getParameters...
[realsense2_camera_node-1] [WARN] [1734881539.477987104] [camera.camera]: For the 'unite_imu_method' param update to take effect, re-enable either gyro or accel stream.
[realsense2_camera_node-1] [INFO] [1734881539.478113977] [camera.camera]: JSON file is not provided
[realsense2_camera_node-1] [INFO] [1734881539.478205268] [camera.camera]: Device Name: Intel RealSense D435I
[realsense2_camera_node-1] [INFO] [1734881539.478293642] [camera.camera]: Device Serial No: 233522071798
[realsense2_camera_node-1] [INFO] [1734881539.478380557] [camera.camera]: Device physical port: 6-1-12
[realsense2_camera_node-1] [INFO] [1734881539.478467181] [camera.camera]: Device FW version: 5.16.0.1
[realsense2_camera_node-1] [INFO] [1734881539.478563429] [camera.camera]: Device Product ID: 0x0B3A
[realsense2_camera_node-1] [INFO] [1734881539.478834676] [camera.camera]: Sync Mode: Off
[realsense2_camera_node-1] [INFO] [1734881542.383779261] [camera.camera]: Set ROS param depth_module.depth_profile to default: 848x480x30
[realsense2_camera_node-1] [INFO] [1734881542.386540722] [camera.camera]: Set ROS param depth_module.infra_profile to default: 848x480x30
[realsense2_camera_node-1] [INFO] [1734881542.522259180] [camera.camera]: Set ROS param rgb_camera.color_profile to default: 1280x720x30
[realsense2_camera_node-1] [INFO] [1734881542.523162750] [camera.camera]: Set ROS param gyro_fps to default: 200
[realsense2_camera_node-1] [INFO] [1734881542.523242666] [camera.camera]: Set ROS param accel_fps to default: 100
[realsense2_camera_node-1] [INFO] [1734881542.526616910] [camera.camera]: Stopping Sensor: Depth Module
[realsense2_camera_node-1] [INFO] [1734881542.526762449] [camera.camera]: Stopping Sensor: RGB Camera
[realsense2_camera_node-1] [INFO] [1734881542.526797449] [camera.camera]: Stopping Sensor: Motion Module
[realsense2_camera_node-1] [INFO] [1734881542.604336562] [camera.camera]: Starting Sensor: Depth Module
[realsense2_camera_node-1] [INFO] [1734881542.703155082] [camera.camera]: Open profile: stream_type: Depth(0), Format: Z16, Width: 848, Height: 480, FPS: 30
[realsense2_camera_node-1]  22/12 23:32:22,703 WARNING [547004344544] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0

Could you kindly give me more suggestions?

@MartyG-RealSense
Copy link
Collaborator

Have you checked RViz to see whether the topics can be correctly displayed, please? Sometimes the topics publish normally in RViz even when control_transfer returned messages (which indicate a communication problem between camera and computer) are continuously generating in the log.

@Frank198002
Copy link
Author

Hi MartyG, I find the cause. D435i Firmware should be 5.13.0.55, it runs OK not only in realsense-view but also in ROS2 topic.
20241223_D435i_ok
In ROS2, I can see the content in topic ../../../sample
The next problem is what is mean for "Frame drop rate in one second". It shows 78%, but I still receive IMU info.
And could you tell me the proper SDK version for FW-5.13.0.55, I didn't know whether SDK 2.55.1 can match FW-5.13.0.55

@MartyG-RealSense
Copy link
Collaborator

I'm pleased to hear that firmware 5.13.0.55 worked for you!

The 'Frame Drops Per Second' graph in realsense-viewer can be disregarded as it does not provide useful information about analysing performance. Clicking the 'i' button on the top of the stream panels to overlay information over the panel provides more reliable information, such as the current frames per second (FPS) speed of a stream.

The librealsense versions that match to firmware 5.13.0.55 are 2.50.0 and 2.51.1. However, as you found yourself, 5.13.0.55 can also be successfully used with librealsense 2.55.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants