Has anyone had succeed with rolling shutter camera and unsync low frequency IMU (MPU9250)

[omeredemen]

Hi all,
I am using see3cam cu27 kamera and mpu9250. I know that the lowest robust setup is rolling shutter camera and unsync low frequency IMU but when i tried with my setup vins-mono couldn't even initialize.

[omeredemen]

Well, after some calibrations I got some good calibration result.
seecam.zip

Initialization is possible now, but drifts occur after a while. I think this issue is because of imu-cam sync and bad imu timestamp. here is my output after calibration:

How can i correct the imu timestamp with ros or if there is any alternative could you please let me know?

[SerpentWindy]

i read your calibration parameters. Because of your (640,480) image size, you can change the "min_dist" in feature traker paprameters from 30 to 25. And your timeshift_cam_imu and gyroscope_noise_density are too large, you may try to reduce the error caused by these parameters.

[SerpentWindy]

Also, may I ask if you frequently experience "system reboot"(Trajectory returns to origin), and if your dynamic calibration results match those of offline calibration.

[omeredemen]

oh sorry for the late reply. i will try what you said about parameters.
i actually didn't understand what you asked so can you explain if you don't mind

[SerpentWindy]

oh sorry for the late reply. i will try what you said about parameters. i actually didn't understand what you asked so can you explain if you don't mind

I explained in advance that there is no reliable evidence for the first point, only as a suggestion.
After reviewing your calibration file, I found three possible issues:

1. Your image size is (640,480), and VINS uses two feature detection parameters when processing images of (640,480) size, "min_dist" is 30 and 25, respectively. Therefore, I suggest you try using "min_dist" parameters of 30 and 25 respectively.

2. Your timeshift_cam_imu is 241ms, and according to the VINS development team, the timeshift_cam_imu should be controlled within 30ms, otherwise it will cause motion mismatch between the image and IMU data. Therefore, it is necessary to reduce the time error to within 30ms. My suggestion is that the best approach is hard synchronization. If hard synchronization is not possible. Not necessarily effective, but my suggestion is that you can inquire and research the time from when the camera and IMU received the trigger signal to when you stamped it, and subtract the corresponding time from your camera timestamp and IMU timestamp. Reduce time synchronization error through the above methods.

3. Your gyroscope noise standard deviation is 0.1495249516640554, which is larger than the noise standard deviation of the accelerometer 0.05166104637481483. In my experience, this is not reasonable. I suggest that you first confirm whether there is any abnormal data in your IMU data, whether the format of the calibration data is correct (timestamp, wx, wy, wz, ax, ay, az), and check whether the unit of the IMU output angular velocity you use is deg/s or rad/s (the unit required by VINS is rad/s), and then re calibrate the IMU internal parameters. If the output angular velocity unit of your IMU gyroscope is deg/s, please multiply the IMU output angular velocity by PI/180 to convert it to rad/s. May I ask what is the IMU calibration tool you are using? If it is imu_utils, because the units of the imu_utils result are not actually rad/s and m/s ^ 2, but rad/s/sqrt (Hz) and m/s ^ 2/sqrt (Hz), you also need to multiply the noise standard deviation of the accelerometer and gyroscope by the sqrt your IMU output frequency. The random walk of the accelerometer and gyroscope is divided by the sqrt your IMU output frequency. For example, if your IMU frequency is 100Hz, then the sqrt your IMU output frequency is 10.

[omeredemen]

i am using kalibr as calibrarion tool and to calculate allan variance parameters i am using this tool. And this repo gives allan variance parameters as kalibr requested format.

well with hardware i have now, i cant make a hard sync but i succesfully reduce the timeshift_cam_imu. But still my gyroscope and accelerameter noise standart deviation too big. I checked my acc and gyro units again and i think there is no problem with them

I will use another mpu9250 module and calculate allan variance parameters again

[SerpentWindy]

i am using kalibr as calibrarion tool and to calculate allan variance parameters i am using this tool. And this repo gives allan variance parameters as kalibr requested format.

well with hardware i have now, i cant make a hard sync but i succesfully reduce the timeshift_cam_imu. But still my gyroscope and accelerameter noise standart deviation too big. I checked my acc and gyro units again and i think there is no problem with them

I will use another mpu9250 module and calculate allan variance parameters again

Sorry, I have used the imu_utils tool instead of the one you used, so I cannot provide any feedback on this.

I have to apologize for the third point I made in my previous comment. I discussed with friends today and noticed that the noise of the gyroscope does not necessarily have to be smaller than the noise of the accelerometer, which means your calibration data may be correct.

May I ask if VINS often experiences "system reboot" (Trajectory returns to origin) after reducing the timeshift_cam_imu?

[omeredemen]

May I ask if VINS often experiences "system reboot" (Trajectory returns to origin) after reducing the timeshift_cam_imu?

I changed my camera to raspberry pi v2 a long time ago and i dont remember I've seen this expression before sorry.
with rpi camera and mpu9250 it cannot be even initialized now i think i should change my hardware to a reliable one.
What does that mean and where does it occur from?

Sorry, I have used the imu_utils tool instead of the one you used, so I cannot provide any feedback on this.

Thanks for the clues I think this issue is because of imu-cam unsync, imu noises and camera. i tried to solve imu noises with avarege filter, but i think i should use better camera for this.

[SerpentWindy]

May I ask if VINS often experiences "system reboot" (Trajectory returns to origin) after reducing the timeshift_cam_imu?

I changed my camera to raspberry pi v2 a long time ago and i dont remember I've seen this expression before sorry. with rpi camera and mpu9250 it cannot be even initialized now i think i should change my hardware to a reliable one. What does that mean and where does it occur from?

The "system reboot” is like the following figure, after running VINS for a period of time, the trajectory will jump to the starting point and run again.

Sorry, I have used the imu_utils tool instead of the one you used, so I cannot provide any feedback on this.

Thanks for the clues I think this issue is because of imu-cam unsync, imu noises and camera. i tried to solve imu noises with avarege filter, but i think i should use better camera for this.

I think your idea is right

[SerpentWindy]

[omeredemen]

Ohh i see. Yes it was happened continuously as the following image.

[SerpentWindy]

Ohh i see. Yes it was happened continuously as the following image.

I can't see the picture, can you send a new one?

[omeredemen]