Accessing RealSense in Java via OpenCV VideoCapture - Works on Mac, not on Pi 4?

[thatdanield]

Hi there, I developed an elementary (but functional) april tag detector in Java using opencv on an M1 Mac with the Realsense attached, and was able to access the color feed using the line "VideoCapture cap = new VideoCapture(0);"
Now, when I port this over to a raspberry pi with opencv set up alright, it gives me an error about the index when I try running it, so I put "/dev/video2" in place of the 0, since video2, the black and white (IR?) output with polka dots, is the only one that works.
/dev/video4 does technically give me an image, but no matter how long I wait before grabbing a frame is super dark and useless. I believe it's the RGB one, since I get the same result if I try to grab the frame immediately after starting the capture on the Mac.
Thus, I come here to ask, how can I access the camera feed the same way on the Raspberry Pi 4 (running Raspbian), or at least in any usable way to get the color feed?

[MartyG-RealSense]

Hi @thatdanield Which RealSense camera model are you using, please? Typically on RealSense 400 Series cameras (D415, D435 / D435i, D455) the /video channels will default to the following values:

/video0: depth
/video1: infrared
/video2: RGB

The numbering of the /video channels can change though depending on which devices are detected by a computer.

[thatdanield]

Hi @MartyG-RealSense ,
I'm currently using a normal D435. From what I've seen, video 0 and 1 both don't give anything (null image) and video2 appears to be FLIR, not RGB. I believe video4 to be the RGB camera.

[MartyG-RealSense]

In a previous case where the /video addresses were changing, I suggested defining an udev rule (a device handling rule) that sets fixed numbers for the channels. An example of this can be found at the link below.

https://askubuntu.com/questions/715333/assign-webcam-to-a-specific-dev-video

If an image is black and white and has dots on it then yes, it is the infrared image.

If you have auto-exposure enabled, please try disabling auto-exposure to see whether it makes a difference to the brightness of the grabbed frame.