Camera Calibration Guide - Intrinsic Parameters and Lens Distortion Correction

What Is Camera Calibration - Why It Is Needed

Camera calibration is the process of estimating mathematical parameters that describe a camera's optical and geometric characteristics. It is essential for accurately predicting where 3D world points project onto 2D images, forming the foundation for virtually all computer vision tasks.

Where calibration is needed:

• 3D measurement: Accurate distance measurement in stereo vision and Structure from Motion
• AR (Augmented Reality): Precisely placing virtual objects in the real world
• Autonomous driving: Estimating distances to obstacles from camera images
• Robot vision: Mapping between robot arm coordinates and camera coordinates
• Distortion correction: Removing barrel distortion from wide-angle lenses for accurate imagery

Camera parameter classification:

• Intrinsic parameters: Focal length, principal point, distortion coefficients. Camera-specific optical properties
• Extrinsic parameters: Rotation matrix and translation vector. Camera position and orientation

Intrinsic parameters remain constant as long as the camera-lens combination is unchanged, though zoom lenses require calibration at each focal length. Extrinsic parameters change whenever the camera moves. Standard calibration captures a known pattern (chessboard) from multiple angles to simultaneously estimate all parameters.

The Pinhole Camera Model - Mathematical Description of Projection

The pinhole camera model mathematically describes an ideal camera where light passes through a single point (pinhole) to project onto the image plane, ignoring lens thickness. It serves as the widely-used approximation model for real cameras in computer vision.

Projection equation: The relationship between a 3D point (X, Y, Z) and its image projection (u, v) is expressed as:

s[u, v, 1]^T = K[R|t][X, Y, Z, 1]^T

where K is the intrinsic parameter matrix (camera matrix), [R|t] is the extrinsic parameters (3x4 matrix), and s is a scale factor.

Intrinsic parameter matrix K:

K = [[fx, 0, cx], [0, fy, cy], [0, 0, 1]]

• fx, fy: Focal length in pixels. fx = f × sx (f: physical focal length mm, sx: pixel density pixel/mm)
• cx, cy: Principal point pixel coordinates (intersection of optical axis and image plane). Ideally image center but typically offset by several pixels

Typical intrinsic parameter examples: iPhone 14 Pro (main camera): fx ≈ 3000px, cx ≈ 2016px, cy ≈ 1512px (4032x3024 resolution). GoPro Hero 11: fx ≈ 1500px (shorter focal length due to wide angle). Industrial camera (5MP, 8mm lens): fx ≈ 2800px. Larger focal length means telephoto (narrow FOV), smaller means wide-angle (wide FOV).

Lens Distortion Models - Radial and Tangential Distortion

Real lenses produce deviations (distortion) from the ideal pinhole model. Accurately modeling and correcting distortion dramatically improves measurement precision. OpenCV standardly uses 5 distortion coefficients (k1, k2, p1, p2, k3).

Radial Distortion: Distortion that increases with distance from the lens center, affecting pixels more at image periphery:

• Barrel distortion: k1 < 0. Image periphery bulges outward. Prominent in wide-angle lenses
• Pincushion distortion: k1 > 0. Image periphery curves inward. Occurs in telephoto lenses

x_distorted = x(1 + k1*r² + k2*r⁴ + k3*r⁶)

y_distorted = y(1 + k1*r² + k2*r⁴ + k3*r⁶)

where r² = x² + y² (distance from center in normalized coordinates). Typical values: wide-angle k1 ≈ -0.3, standard lens k1 ≈ -0.05, telephoto k1 ≈ 0.01.

Tangential Distortion: Occurs when lens and sensor are not perfectly parallel, depending on manufacturing precision and typically smaller than radial distortion:

x_distorted = x + 2*p1*x*y + p2*(r² + 2*x²)

y_distorted = y + p1*(r² + 2*y²) + 2*p2*x*y

Typical values: p1, p2 ≈ ±0.001. Negligible for high-quality lenses but correction needed for inexpensive lenses and webcams. Ultra-wide lenses like GoPro may require higher-order coefficients (k4, k5, k6) or thin prism models.

Zhang's Method Calibration Procedure

Zhang's method (2000) estimates all camera parameters by simply photographing a planar pattern (chessboard) from multiple angles. Requiring no special equipment and fully implemented in OpenCV, it is the most widely used calibration method in practice.

Required equipment:

• Chessboard pattern (recommended: 9x6 or 10x7 inner corners)
• Pattern planarity is critical. Mount on cardboard or acrylic plate
• Accurately measure square size (e.g., 25mm)

Capture guidelines:

• Minimum 10 images, recommended 20-30
• Tilt pattern at various angles (approximately ±45°)
• Vary position so pattern covers entire image area
• Include pattern at image edges (important for distortion estimation)
• Use sharp, well-focused images

OpenCV implementation steps:

• cv2.findChessboardCorners() to detect corners
• cv2.cornerSubPix() for sub-pixel refinement
• Build correspondences between 3D points (known chessboard coordinates) and 2D points (detected corners)
• cv2.calibrateCamera() to simultaneously estimate all parameters

Reprojection error evaluation: Calibration quality is assessed by reprojection error - reprojecting 3D points using estimated parameters and computing distance to detected points. Good calibration achieves 0.1-0.5 pixels; acceptable is below 1.0 pixel. Above 1.0 requires reviewing capture conditions or corner detection.

Distortion Correction Implementation and Applications

This section covers how to remove image distortion using calibration-derived distortion coefficients. Distortion correction is a mandatory preprocessing step for 3D measurement, with correction accuracy determining downstream processing precision.

OpenCV distortion correction:

• Method 1: cv2.undistort(img, K, dist_coeffs) - Simplest approach. Recomputes each frame, inefficient for video
• Method 2: cv2.initUndistortRectifyMap() + cv2.remap() - Pre-compute maps for reuse. Optimal for video. Map computation once only, remap is fast (approximately 3ms for 1080p)

Corrected image size: Distortion correction causes edge cropping, so cv2.getOptimalNewCameraMatrix() computes a new camera matrix. alpha=0 completely removes black regions (smaller image), alpha=1 retains all pixels (black regions remain). Adjust alpha between 0-1 based on application requirements.

Fisheye lens correction: Standard distortion models cannot handle fisheye lenses with 180°+ field of view. OpenCV's cv2.fisheye module uses equidistant projection model for ultra-wide lens calibration and correction via cv2.fisheye.calibrate() and cv2.fisheye.undistortImage().

Saving calibration results: Save estimated parameters in YAML or JSON format for application use. OpenCV's cv2.FileStorage class provides easy save/load functionality. Results remain valid indefinitely as long as the camera-lens combination is unchanged.

Computer vision textbooks are available on Amazon

High-Precision Calibration Techniques and Troubleshooting

This section covers techniques for achieving high-precision calibration (reprojection error below 0.1 pixels) required in industrial and research applications, along with solutions to common problems.

High-precision techniques:

• Pattern quality: Laser-printed patterns may have distortion. For high precision, use patterns etched on glass substrates
• Lighting conditions: Capture under uniform diffuse illumination. Direct light and shadows degrade corner detection accuracy
• Number of captures: 50+ images provide statistically stable estimation
• Pattern placement: Ensure pattern appears in all four image corners. Center-only placement degrades distortion coefficient estimation
• Fixed parameters: When fx = fy (square pixels) can be assumed, add constraint with cv2.CALIB_FIX_ASPECT_RATIO flag

Troubleshooting:

• Large reprojection error (> 1.0px): Caused by blurry images, non-planar pattern, or corner detection errors. Review images individually and exclude high-error ones
• Abnormal focal length: Pattern too small relative to image, or insufficient angle variation. Ensure pattern occupies 20%+ of image area
• Unstable distortion coefficients: Insufficient images or pattern concentrated at image center. Place pattern at edges

Auto-calibration: SLAM systems use self-calibration that automatically estimates camera parameters from feature point tracking. ORB-SLAM3 and COLMAP can estimate intrinsic parameters from natural scene features. Precision is lower than pattern-based methods but requires no preparation, making it practical for many applications.