Trying to solve extreme drift with software synchronization

[ardaboluk]

I've been successfully using VINS-Mono on i7 notebook with unsynchronized 50FPS global shutter camera and 250FPS high quality IMU. However, when I run the algorithm with the same camera and IMU on Jetson TX2, odometry flies away most of the time and good initialization occurs very rarely.

I suspect that this may be caused by camera and IMU being unsynchronized. On notebook, time difference is less than 30ms and changes very little. On TX2 however, initial value of time difference changes with each run. Also, it changes dramatically during the run.

As a possible solution, I think of synchronizing them with an external ROS node which publishes two topics for IMU and camera at fixed frequencies. The topic messages will have system time (Jetson TX2) as timestamp.

Then I will add subscribers to IMU and camera SDKs listening to corresponding topics. In callbacks of the subscribers, I will then publish the latest IMU and camera data, dropping the previous ones. Also, timestamps of external topic messages will be assigned to IMU and camera messages in the callbacks.

Any one tried this solution ? Is it viable ?

Could you share if you have an insight ?

[mansisarawata]

Were you able to solve it? I am trying to do something similar.

[ardaboluk]

No, I couldn't. This didn't work. As I understand hardware synchronization is essential.

Increasing camera FPS and throwing a powerful CPU minimizes the time difference between IMU and camera, enabling the algorithm to tolerate unsynchronized setup.