Make writing a habit together! This is the sixth day of my participation in the “Gold Digging Day New Plan · April More text Challenge”. Click here for more details.

0 Death Angle of the photo

The “death Angle” usually refers to taking a big shot of your washbasin and looking at people with your nostrils, as in this photo. At the death Angle of the photo, lens distortion will create distortion, making your body more round and thick, so some beauty software will have a “remove distortion” feature in the portrait correction. This paper discusses the lens distortion principle from the perspective of computer vision.

1 perspective camera model

The perspective camera model describes the mapping relationship between points in 3d space and pixels in 2d image plane.

According to the keyhole imaging principle, the direction between the imaging point and the space point of the perspective camera is opposite, as shown in the figure. In order to obtain the imaging point with the same direction as the space point, the imaging plane is moved to the normalized image plane along the optical axis, and the normalized image plane is the criterion for the following discussion.

After determining the normalized image plane, from the space point W ⁣X~^W\! Tilde {X}WX~ to pixel u~ tilde{u}u~ can be divided into two steps:

  1. The W ⁣ X ~ ^ W \! Tilde {X}WX~ maps to the image point of the normalized image plane C ⁣ X ^C\! Tilde {x}Cx~, the corresponding mapping is called Extrinsic.
  2. Will C ⁣ x ~ ^ C \! Tilde {x}Cx~ The pixels u~ tilde{u}u~ that are mapped to the pixel plane are called Intrinsic. All the points above are in the projection space, using homogeneous coordinates.

2 Camera Matrix

Set the posture of the world coordinate system {W}\left\{\boldsymbol{W} \right\}{W} with respect to the perspective camera coordinate system {C}\left\{\boldsymbol{C} \right\}{C} as


W C ⁣ T = [ W C    ⁣ ⁣ ⁣ R C ⁣ p w 0 0 1 ] _{\boldsymbol{W}}^{\boldsymbol{C}}\! \boldsymbol{T}=\left[ \begin{matrix} _{\boldsymbol{W}}^{\boldsymbol{C}}\;\!\!\!\boldsymbol{R}& ^{\boldsymbol{C}}\! \boldsymbol{p}_{w_0}\\ 0& 1\\\end{matrix} \right]

Considering from the three dimensional space dimension reduction to two-dimensional plane, a world coordinate values [W ⁣ ⁣ XW ⁣ ⁣ ⁣ ⁣ YW ⁣ ⁣ Z1] T \ left [\ begin ^ {matrix} {\ boldsymbol {W} \!} \! X & ^ {\ boldsymbol {W} \!} \! \! : \ \! \ : Y & ^ {\ boldsymbol {W} \!} \! Z & 1 {matrix} \ \ \ \ end right] [WXWYWZ1] T ^ T, then


C ⁣ x ~ = [ 1 0 0 0 0 1 0 0 0 0 1 0 ] . W C ⁣ T     W ⁣ ⁣ X ~ = [ C ⁣ X C Y C ⁣ Z ] ^{\boldsymbol{C}}\! \tilde{x}=\left[ \begin{matrix} 1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\\end{matrix} \right] {\color{white} ,}_{\boldsymbol{W}}^{\boldsymbol{C}}\! \boldsymbol{T}^{\,\,\boldsymbol{W}}\! The \! \tilde{X}=\left[ \begin{array}{c} ^{\boldsymbol{C}}\!X\\ ^{\boldsymbol{C}}Y\\ ^{\boldsymbol{C}}\!Z\\\end{array} \right]

In the above derivation, said


M E = [ 1 0 0 0 0 1 0 0 0 0 1 0 ] . W C ⁣ T = [ W C    ⁣ ⁣ ⁣ R C ⁣ p w 0 ] \boldsymbol{M}_{\boldsymbol{E}}=\left[ \begin{matrix} 1& 0& 0& 0\\ 0& 1& 0& 0\\ 0& 0& 1& 0\\\end{matrix} \right] {\color{white} ,}_{\boldsymbol{W}}^{\boldsymbol{C}}\! \boldsymbol{T}=\left[ \begin{matrix} _{\boldsymbol{W}}^{\boldsymbol{C}}\;\!\!\!\boldsymbol{R}& ^{\boldsymbol{C}}\! \boldsymbol{p}_{w_0}\\\end{matrix} \right]

Is the camera external parameter matrix. Usually let the scale factor C ⁣Z=1^{\boldsymbol{C}}\! Z = 1 cz = 1, C ⁣ x ~ = [C ⁣ XC ⁣ ZCYC ⁣ Z1] T = [⁣ ⁣ x ^ C y C ^ 1] T ^ {\ boldsymbol {C}} \! \tilde{x}=\left[ \begin{matrix} \frac{^{\boldsymbol{C}}\!X}{^{\boldsymbol{C}}\! Z}& \frac{^{\boldsymbol{C}}Y}{^{\boldsymbol{C}}\! Z}& 1\\\end{matrix} \right] ^T=\left[ \begin{matrix} ^{\boldsymbol{C}}\!\hat{x}& ^{\boldsymbol{C}}\! \hat{y}& 1\\\end{matrix} \right] ^TCx~=[CZCXCZCY1]T=[Cx^Cy^1]T

From C ⁣ x ~ ^ C \! \tilde{x}Cx~ to pixel u~\tilde{u}u~ is represented by a three-dimensional internal parameter matrix KKK, i.e


u ~ = K C ⁣ x ~ = [ f u s c u 0 f v c v 0 0 1 ]     C ⁣ x ~ \tilde{u}=\boldsymbol{K}^C\! \tilde{x}=\left[ \begin{matrix} f_u& s& c_u\\ 0& f_v& c_v\\ 0& 0& 1\\\end{matrix} \right] \,\,^C\! \tilde{x}

Among them

  • As shown in Figure (a), parameters cuc_UCu and CVC_VCV are used for center mapping — the intersection of optical axis and normalized image plane, that is, the center point of image plane is mapped to the center of pixel plane, depending on how bayer array is aligned with the optical axis.
  • As shown in Figure (b), the parameters FUF_UFU and FVF_VFV are used for normalization — mapping a rectangular sensor array to a square
  • As shown in Figure (c), the parameter SSS is used for orthogonalization — correction distortions need to be introduced when the sensor array is not orthogonal or the sensor plane is not perpendicular to the optical axis.

3 Lens distortion

The perspective camera model above is based on the assumption that the pinhole is infinitely small, but in reality, light passing through the lens will complicate the camera’s interior, creating significant radial distortion — lines in the scene appear as curves in the image. There are two types of Radial Distortion:

  • Barrel Distortion
  • Pincushion Distortion

In addition, Tangential Distortion is introduced because the lens cannot be strictly parallel to the imaging plane during the assembly process.

For cameras with lens distortion, perspective camera model cannot be directly applied, but such geometric deviation should be corrected by polynomial model first, namely:


{ x ^ = x ( 1 + κ 1 r 2 + κ 2 r 4 ) + 2 p 1 x y + p 2 ( r 2 + 2 x 2 ) y ^ = y ( 1 + κ 1 r 2 + κ 2 r 4 ) + p 1 ( r 2 + 2 y 2 ) + 2 p 2 x y \begin{cases} \hat{x}=x\left( 1+\kappa _1r^2+\kappa _2r^4 \right) +2p_1xy+p_2\left( r^2+2x^2 \right)\\ \hat{y}=y\left( 1+\kappa _1r^2+\kappa _2r^4 \right) +p_1\left( r^2+2y^2 \right) +2p_2xy\\\end{cases}

R2 = X2 + y2R ^2= X ^2+ Y ^ 2R2 = X2 +y2, κ1\kappa _1κ1, κ2\ Kappa _2κ2 are called radial distortion parameters, p1p_1P1 and p2P_2P2 are called tangential distortion parameters. (x, y) (x, y) coordinates (x, y) as the distortion, (x ^, ^) y \ left (\ hat {x}, \ hat {} y \ right) (x ^ ^ y) as the undistorted coordinates.

Computer vision basics tutorial outline

The chapter content

Color space and digital imaging

Fundamentals of computer geometry

2. Image enhancement, filtering, pyramid

3. Image feature extraction

4. Image feature description

5. Image feature matching

6 Stereovision

7 Actual Project

Welcome to my AI channel “AI Technology Club”.