Skip to content

Latest commit

 

History

History
130 lines (84 loc) · 10.3 KB

CalibrationDataCollection.md

File metadata and controls

130 lines (84 loc) · 10.3 KB
Raw

Collecting calibration data

Overview and rationale

There is a nice overview on the objective of a typical calibration procedure here.

There are three main requirements for good calibration data (see intro to this paper)

  • calibration target must be placed in the common FOV
  • calibration target should cover at least 1/3 of the common FOV (this may be too strict for us at large distances)
  • the features of the calibration target need to be in focus and sufficiently spatially resolved (!) to allow for centroid detection.

All of these become challenging with a wide FOV (> 1m width)

Recommendations

0. Arrangement of the cameras

There are very nice recommendations on the arrangement of the cameras in the StereoMorph docs (although they focus on taking pictures to reconstruct a 3D object)

  • Remember! We are calibrating the cameras for a specific relative position of the cameras, but also zoom (focal length) and focus levels!

  • Re camera settings

    • They recommend to turn off auto-focus and auto-zoom modes, and vibration reduction (VR) if the lens has it.
    • They recommend setting the cameras to the smallest aperture

      "The smaller the aperture, the greater the depth of field (i.e. the more things are in focus both close and far away from the camera). This is essential in a stereo camera setup because in order to digitize points accurately throughout the calibration volume they must be in focus."

  • Re cameras' positions:

    • "Because the camera is often positioned half a meter or more away from the object, even a small shift of the camera can translate into a large shift in the image frame, causing large inaccuracies"
    • Leg stoppers (this type or these with spikes for uneven surfaces) could be useful to keep the tripods as fixed as possible against the wind and on top of uneven terrain
    • "It's best to use a remote (wireless or cord) to release the shutter so you minimizing touching the shutter button on the cameras as much as possible."

  • In our case, we probably want as much FOV overlap as possible

1. Design of the checkerboard

  • The pattern should be on a flat and rigid surface

    • From StereoMorph: "If you want an exceptionally flat surface, I recommend plexiglass (at least 0.22 inches thick)"

  • It should have a white border around it, of roughly the same width as one of the cells (this makes the detection more robust in various environments see here)

  • Size of the checkerboard

    • From StereoMorph docs:
      • What volume is visible from the two (or more) cameras?
        • If we approximated it to a box, what is the shortest dimension (n)?
          • is it smaller than 0.5 m? Then the checkerboard side should be around half of the shortest dimension (0.5n)

            For example, for a calibrated volume with approximate dimensions of 0.3 m x 0.35 m x 0.4 m, a checkerboard pattern measuring 0.15 m along one dimension should be ideal.

          • is it larger than 0.5 m? Then the checkerboard side should be around 0.5m

  • Min number of inner corners

    • From DLC 3D: at least 8x6 squares
    • Why do we say inner corners?: "All chessboard related algorithm expects amount of inner corners as board width and height" 1 - openCV parametrises the width and height based on number of inner corners because this is what will actually be detected
    • The number of inner corners should be different along each dimension, to ensure they are returned in the same order when the chessboard changes orientation

  • Min size of the cells

  • Creating a calibration pattern

  • Consider printing it in cardstock paper (from here)

    • "Once taped or glued to a hard surface, cardstock is less likely to get bubbles from moisture over time."

2. Position and orientation of the checkerboard

For a calibration frame to be useful, the checkerboard must fulfill the following criteria:

  • Keep the orientation of the checkboard within 90deg (conservatively 30 deg 2) of the "original"/default position

    • Why? The algorithm takes the bottom (right?) corner of the board as the origin, and then traverses the rest of corners from there. If in some frames the board is rotated beyond 30 deg, the origin may be detected at a different physical corner, and all the corners may be traversed in a different order.

  • Cover several distances, and within each distance, cover all parts of the image view/of the FOV overlap 2

    • We want to cover as much as possible of the calibration space (or the space where we want to collect data in)
    • Why?:

      "If you only photograph the checkerboard in one area of the calibration volume, reconstruction errors could be relatively higher in other areas. Similarly, if you only photograph the checkerboard at a particular angle (e.g. 45 degrees), you won't have good sampling of points along each dimension of the space (since a checkerboard is a flat surface it can only sample two dimensions at any one time). This can cause reconstruction errors to be higher along particular dimensions than along others." (from here) So we'd like to collect most images in the areas where the animals will be, so that we have especially low error there.

    • Note however that if at some distance the calibration pattern is too small, we may not be able to detect the corners reliably and the calibration may be poor. To fix this we may need a larger board (or move cameras closer to the area of interest before calibrating and collecting data)

  • For a calibration frame to be useful, the checkerboard must be fully visible in both cameras

    • Why? Since the algorithm first tries to search for the outer corners of the board, it will fail if the board is only partially visible. This may be avoidable using an alternative algorithm, but the approach doesn't seem 100% robust, so it's probably a good idea to be conservative here and keep it fully visible at all times.

  • Number of frames captured

    • In DeepLabCut 3D they claim 30-70 pairs of candidate calibration images should be sufficient, as after corner detection, some of the images might need to be discarded due to either incorrect corner detection or incorrect order of detected corners.

  • Keep the pattern steady when aiming to collect a calibration frame (this is to avoid having a blurry frame)

Timecode drift

  • The timecode option is sensitive to drift, because it relies on the clocks of the two cameras running perfectly in sync (right?)
    • In reality, one of the clocks may lag behind the other if the recording time is too long
  • Some questions
    • what is "too long"? (I think several hours or days)
    • is the Free Run option more sensitive to drift?
    • can we resync the cameras' timecodes frequently?
  • We could validate how good/bad the timecode drift is with the approach they use in the Acinoset paper:
    • they flashed a rig of LEDs three times, just before and after data collection

Additional comments

  • Acinoset paper paper may be a good reference on people doing stereo in the field, in a large area
    • code here and notebook using opencv here
  • Texas instruments have nice sample images of a good calibration.
  • Should we try other patterns too (charuco boards, circles grid)? Would it be more robust?

Useful references

Footnotes

  1. https://docs.opencv.org/4.x/d4/d94/tutorial_camera_calibration.html ↩

  2. from DeepLabCut 3D, and OpenPose ↩ ↩2