-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
In OpenCV, a live depth stream doesn't output the correct depth value, but a snapshot of the stream does. #13335
Comments
Hi @MingyuPan2 Instead of multiplying the 16-bit pixel depth value by the camera's depth scale of 0.001, do you get more stable results if you change the line to use the alternative method of the get_distance() instruction instead? This instruction outputs the depth value in real-world meters without needing to multiply by the depth scale. For example:
Also, your code suggests that the full script is using post-processing filters such as hole-filling and spatial. If your script is using these filters, the .process lines of these filters that apply them should ideally be placed before the align.process instruction, as it is recommended that alignment is applied after post-processing. |
Hi @MartyG-RealSense, thanks for the response! Yes, i do get stable results with get_distance(). The filter also don't seem to interfere with the results, since if i turn off all the filters, i still get random distance outputs when the depth stream is live. I'll try to implement the get_depth method in the onMouse function to see if i can get a stable result. |
Here's my entire code, if it helps at all :/
|
If you are using a USB mouse then you could test whether the EVENT_MOUSEMOVE code is making depth_x and depth_y fluctuate when the depth stream is live by unplugging the mouse. If the mouse is a wireless one then the infrared of RealSense cameras can cause interference with the wireless mouse signal. |
I only use a wired USB mouse. Unplugging the mouse does not fix the problem when the stream is live. I also tried getting the output by clicking the live stream before and the output is interesting as seen below: I clicked on different areas of the the stream and some of the output is pretty similar (the actual distance is around 240cm). These screenshots were take probably a quarter of a second apart, and the stream looks completely different even though i set the threshold filter distance to 0 - 4m. The stream actually constantly flash much faster (like every 10ms) and the flashing is worse when i disable all the filters. Again, I don't understand why this is happening since when i visualized the depth stream in RVIZ for ROS, the depth stream also had this exact same flashing problem. I'll try to investigate further and any help you give is greatly appreciated!! |
The lower multi-colored image looks as though it is the correct one of the two, because the depth colorization correctly shifts between blue in the foreground near the camera and red in the far background. In the upper all-blue image it is as though all coordinates have approximately the same depth no matter how far from the camera they are. Could you try disabling auto exposure by setting it to '0' to see whether rapidly changing exposure values could be affecting your depth? Code for doing so can be found at #10239
|
Disabling the auto exposure still does not solve the problem. I can still see visable flashing and it actually introduces an area with no depth value at the center of the depth stream as seen below for some reason.... I changed around the other EXPOSURE settings but it doesn't seem to affect the image. Interestingly, if i freeze the depth stream, the image it produces will give an accurate depth value, no matter if the depth colorization is correct or not. So currently, i can only get accurate depth value from a still image, but not the live stream. |
Is depth more stable if you change the value of the temporal filter's RS2_OPTION_FILTER_SMOOTH_ALPHA setting from 0.4 to 0.1 so that the depth image still updates live but updates less frequently? |
Yes, I tried changing all the filter parameters but it is still the same. Screencast.from.09-18-2024.10.35.42.AM.webmHere's what i see on my screen. Sorry for the laggy video. Screen recording was not smooth at all while the camera was in use. However , you can still see the depth stream flash clearly, either when i move the camera slightly or keep the camera still. |
Thank you very much for the video. Do you have access to the realsense-viewer tool to test whether the flashing also occurs there when depth is streaming in real-time? |
There's no flashing in the realsense-viewer tool. The depth stream output is very stable and the distance it gives is accurate too. I've also seen the same flashing problem when visualizing the depth stream in RVIZ with ROS2 iron. Are there any filters that was not shown in the control panel that might've made the depth stream more stable? Thank you!!! |
All post-processing filters that are enabled are shown in the Post-Processing section. There are not hidden ones. The realsense-viewer does also apply a range of default depth colorization settings in the Depth Visualization category of the side-panel. |
It may also be worth trying to load one of the realsense-viewer's built-in Visual Presets into your C++ script to apply a range of configuration settings automatically. |
I loaded the presets. There's still flashing when the camera points at a specific distance, but the flashing was reduced abit in my opinion. The output distance is still wrong, basically the same as before. |
How does the ROS2 Iron depth image perform if you enable filters in the ros2 launch instruction?
|
Hi @MingyuPan2 Do you have an update about this case that you can provide, please? Thanks! |
I used the above method in ROS2 but the problem is still there. I am also currently working on other projects, so I may put this problem on hold for awhile. I'll let you know when i try something new in a few days! Thank you!! |
You are very welcome. I look forward to your next report. Good luck! |
Hi @MingyuPan2 Do you wish to put this issue on hold as you are working on other projects, please? Thanks! |
Yes, can you please put this question oh hold? Thank you! I'll start working on this project again very soon. |
That's no problem at all, I have added an Enhancement label to this case as a reminder to keep it open. Thanks! |
Hi Mingyu Pan, Ubuntu 20/22 (focal/jammy) with LTS kernel 5.13, 5.15, 5.19, 6.2, 6.5 |
Hi @Frank198002 I have replied at the issue you created at #13611 - thanks! |
| Camera Model | D435i |
| Firmware Version | 5.16.01 |
| Operating System & Version | Ubuntu 22.04.4 LTS |
| Kernel Version (Linux Only) | 5.10.0-1012-rockchip |
| Platform | OrangePi 5 Plus |
| SDK Version | 2.55.1.0 |
| Language | C++ for all code & OpenCV |
| Segment | Others |
Issue Description
Currently, i am trying to write some code in C++ to realize some of the camera's functions in Opencv's output window, as seen below:
As you can see at the bottom of the window where i try to get the distance, the output is not correct, since it gives a seemingly random value between 0 to 6000cm. However, if i turn off the depth stream and clicked on the frozen image of the stream, the output distance will be correct.
I've attached the sections of code where i output the distance below:
I assume that somehow, when the stream is live, the distance value is not properly aligned with the stream (but is aligned when the stream is paused)? Or the way i output the distance value when the stream is live is not correct somehow. I'm not good at coding so i did let chatgpt do a few passes on my code.
Can you help me with this issue in my code? Thank you so much!!!
The text was updated successfully, but these errors were encountered: