top of page
Add.png
Introduction

As a Design Technologist with experience at industry-leading companies like Samsung, Go-MMT Group, I have always been passionate about creating user-centered solutions that blend functionality with seamless user experiences. This portfolio project showcases my design journey in developing a lens calibration software that assists professionals in calibrating spherical lenses using three methods: Charuco, Chessboard, and Dot Grid.

Virtual Production

Camera Calibration & Lens Distortion Correction

Context

Camera lenses experience distortion due to imperfections in construction, variations in glass thickness, curvature, and light refraction. Resulting in barrel distortion (outward curving lines), pincushion distortion (inward curving lines), and chromatic aberrations from uneven bending of different wavelengths.

Virtual lenses in 3D Engines are mathematical models simulating ideal optically accurate behaviour. This discrepancy creates challenges in VFX and virtual production, as uncalibrated real lenses can lead to mismatched perspectives, incorrect alignments, and visual artifacts when integrating live footage with CG elements. Therefore, calibrating real lenses is crucial to replicate their characteristics digitally and achieve realistic and seamless results.

Problem Setting / Research Question

  1. Why do cine lenses have distortions, and how does this affect visuals?

  2. What makes lens calibration important for VFX and Virtual Production?

  3. What types of lenses and cameras are users typically working with?

  4. What are the challenges users face when calibrating lenses?

  5. How to do lens calibration?

  6. How familiar are users with the existing calibration tools and processes?

  7. What specific tasks in the calibration process are most time-consuming or error-prone?

  8. How do existing calibration tools work, and what limitations do they have?

  9. How do users currently gather data for calibration, and what tools do they use?

  10. What data outputs do users typically get from current calibration software?

  11. What to improve onboarding for users new to this software?

  12. What specific features or automations would users prefer in the software?

Secondary Research
Understanding Lens Distortion and Camera Calibration is the goal of SR
pin3.gif

Pincushion Distortion

barrel.gif

Barrel Distortion

Mustach 1.gif

Mustache Distortion

Group 9.png
Understanding Calibration Methods

  1. Checkerboard / Chessboard Method

  2. Dot Grid Method

  3. AprilTag Board Method

  4. ChArUco (Chessboard + ArUco markers) Board Method

1.Calibration using Chessboard / Checkerboard

The chessboard pattern is a traditional and widely used choice for lens calibration. It is highly effective for achieving sub-pixel accuracy, as the intersections of its squares can be detected with precision. Its geometric regularity and clearly defined points enhance calibration accuracy, resulting in lower reprojection errors.

Quick Calibration of Phantom Camera on Bolt-X with 25mm Cooke Lens 

Chessboard Calibration Steps (Unreal Engine Method) 

  • Prepare Chessboard: Print a chessboard pattern or you can even use your laptop screen with known square dimensions e.g. 25mm square. Use Figma or any tool of your choice to create grid. 

  • Capture Images: Now capture multiple images of the chessboard pattern from different angles and positions. Ensure that the entire chessboard is visible in the frame for accurate calibration.

  • Detect Corners: Using computer vision techniques, detect the interior corners of the chessboard pattern in the captured images. Unreal has libraries like OpenCV that provide functions like cv2.findChessboardCorners to find these corner points.

  • Solve Calibration: Once you have captured sufficient images, click on add to distortion calibration. This will solve for the lens intrinsic and extrinsic parameters. This can be done using functions like cv2.calibrateCamera in OpenCV.

  • Refinement: Optionally, refine the camera parameters by repeating the same method.

2.Calibration using Dot Grid 

As the name suggest Dot grid is another patern type that can be used to calibrate lens. It consist of Black circles of specific radius spaced at fixed distance to create a grid of Dots. Usually good for wide angle lenses or fish eye lens. Steps to calibrate lenses with these method is exactly similar to above Unreal Engine Method. UE5/4 may not have inbuilt functionality to calibrate lens using Dot Grid, but as said earlier engine gives you ability to use python and OpenCV libs.

Calibration of Wide Angle Camera using DotGrid Methid using python and opencv lib

3.Calibration using April Tag Board

AprilTag Board is another widely used pattern type for camera calibration. It consists of square fiducial markers with a black border and a unique binary pattern inside each square. These markers have an ID that can be decoded, making them ideal for real-time tracking and camera pose estimation. We tend to use them in scenarios requiring real-time performance, such as tracking multiple markers in motion or environments with wide-angle lenses or fisheye lenses. It is an excellent choice for Object , Talent, Plane tracking

https://april.eecs.umich.edu/software/apriltag

Calibration of Wide Angle Camera using AprilTag Board with python and opencv lib

4.Calibration using ChArUco (Chessboard + ArUco markers) Board

ChArUco boards are our go-to when we need solid lens calibration without the hassle. It’s basically a smart mix of a checkerboard and ArUco markers — so you get accurate corner detection like a chessboard, but with the flexibility of markers that can be identified individually, even if the board isn’t fully visible.

We like using ChArUco in VFX and virtual production because it works well in the real world — where lighting isn't always perfect, the board might be at an angle, or only part of it is in frame. It still gives us reliable, high-precision data to match real cameras with virtual ones. Whether we’re lining up CG extensions or prepping for LED stage shoots, it’s a calibration method that just fits the way we work.

https://docs.opencv.org/3.4/df/d4a/tutorial_charuco_detection.html

frame_0042.jpg

Calibration of Wide Angle Camera using ChArUco Board with python and opencv lib

Integrated Lens Data Systems (LDS)

The "Plug & Play" Workflow

High-end cinema lenses from leading manufacturers such as Cooke Optics (/i Technology), Zeiss (eXtended Data - XD), and ARRI (Lens Data System - LDS) come equipped with intelligent metadata systems. These systems provide comprehensive, frame-accurate lens calibration data, including:

  • Focal Length: Precise measurement of the lens's focal length at any given moment.

  • Focus Distance: Real-time data on the exact plane of focus.

  • T-stop (Aperture): Accurate transmission stop values.

  • Depth of Field (DoF): Calculated DoF to aid focus pullers and cinematographers.

  • Lens Distortion Characteristics: Detailed mapping of lens-specific geometric distortions (crucial for VFX). (Notably emphasized by Zeiss XD and Cooke /i3).

  • Lens Shading (Vignetting) Data: Information on light fall-off towards the edges of the frame. (Also a key feature of Zeiss XD and Cooke /i3).

  • Entrance Pupil Position: Essential data for complex 3D tracking and parallax effects.

This data is seamlessly integrated into the production workflow. It is typically:

  • Recorded directly into camera negative files (e.g., ARRIRAW, Redcode RAW) or alongside them.

  • Accessible on-set via specialized lens control systems, on-camera displays, and external monitors, providing real-time feedback to camera assistants and cinematographers.

  • Readily ingested by leading VFX and post-production software (e.g., Nuke, After Effects, DaVinci Resolve, 3DEqualizer) through dedicated plugins or native support. This allows for automated camera tracking, accurate 3D scene reconstruction, and streamlined compositing.

Primary Research

PR goal is to understand the following points from the Users:

  • The pain points faced in lens calibration.

  • The context in which they use calibration tools.

  • What do they want improved in the current tools (UX, features, results)?

  • Opportunities for innovation (automation, UI, portability, etc.).

​

Users: DOPs, VFX supervisors, VP supervisors, and Photogrammetry experts

APR_0179.JPG
Qualitative Analysis

Method: Semi-Structured Interviews
Type: Mixed (Video Call, In-Person)

Structure

1. Get to Know the User

  • Role in production (e.g. VFX Supervisor, DOP, Unreal Tech Artist)

  • Context of use (on set, post-production, real-time tracking, etc.)

  • Type of lenses and cameras used (spherical, anamorphic, zoom/fixed)

  • Experience level with lens calibration (Beginner, Intermediate, Expert)

2. Calibration Workflow

  • When and why do you perform lens calibration?

  • What tools or software do you currently use for this task?

    • (e.g., OpenCV, Unreal Engine, custom scripts)

  • What kind of calibration patterns do you prefer?

    • (Charuco, Chessboard, Dot grid, AprilTag)

  • Do you calibrate in controlled environments or on set?

  • What is your step-by-step process for a typical calibration session?

3. Experience with Tools

  • What works well in your current toolset?

  • What are the biggest pain points you face while calibrating?

    • (e.g., inconsistent detection, UI confusion, long solve times)

  • How do you validate whether the calibration worked?

  • Are you satisfied with the feedback and visualisation the tool provides?

​

4. Data Handling and Export

  • What kind of output do you expect (CSV, JSON, FBX)?

  • How do you use this data in your production pipeline?

  • Is exporting or sharing calibration data easy?

5. User Preferences and Ideas

  • Would you benefit from:

    • Real-time feedback?

    • Frame scrubbing and manual frame selection?

    • Mobile app-based calibration?

    • Automated best-frame suggestion?

  • What does an ideal calibration interface look like to you?

  • What would make your life easier as a user of such tools?

6. Wrap-up & Reflection

  • Have you ever skipped calibration due to tool frustration?

  • Any “hacks” or workarounds you currently use?

  • If we were building a new tool, what must-have feature should we include?

Summary of Primary Research

1

Current Calibration Setup

  • "I need a PC, cables, and an SDI capture card — it’s too much gear to carry."

  • "It’s frustrating being tied to a workstation when I’m on location."

3

Real-Time Feedback Needs

  • "I hate running an entire calibration only to find the markers weren’t detected properly."

  • "Why can’t I see if the system is picking up the pattern live?"

5

Displaying Calibration Boards

  • "Sometimes I don’t have a printed checkerboard with me."

  • "It's very difficult to source board types when I need them."

2

Calibration Methods and Flexibility

  • "Most tools only support one pattern, but sometimes I need different ones for specific lenses."

  • "I like having options — sometimes checkerboard works, sometimes ChArUco or AprilTag is better."

  • "Switching between software or patterns is a hassle."

4

Result Storage and Access

  • "I always forget where I saved my calibration files on my PC."

  • "I want my calibration data accessible from anywhere, not stuck on one machine."

6

End-to-End Workflow Efficiency

  • "I just want one clean, streamlined tool that does it all in one go."

Problem Statement

Professionals in VFX and virtual production face significant barriers when calibrating real camera lenses: current tools are bulky, PC-dependent, and fragmented, requiring multiple software and hardware components. These setups limit mobility, restrict platform flexibility, and offer limited real-time feedback, making the process time-consuming, error-prone, and inaccessible on the go.

Mask group (1).png
Solution

Identified Needs

  • On-the-Go Independence

  • Platform Agnostic Tools

  • Instant Visual Confirmation

  • Effortless Data Access

  • Flexible Control and Choice

  • Integrated, Seamless Workflow

Overview

I developed a portable Android-based lens calibration tool that solves the major pain points professionals face in VFX and virtual production. Instead of relying on bulky PCs, capture cards, and long cables, I designed a system where users can plug an OTG USB 3.0 HDMI capture card directly into their phone, making the entire calibration process mobile, lightweight, and location-independent.

​

To provide flexibility, I built the app to support multiple calibration methods — Checkerboard, ChArUco, and AprilTag — allowing users to pick what works best for their lens and setup. I ensured the app delivers real-time detection feedback so users can instantly verify marker tracking before running full calibrations, saving time and reducing errors.

​

Once calibration is complete, my tool automatically generates CSV summaries with distortion coefficients and uploads them directly to Google Drive, making data access seamless across devices. Additionally, to eliminate the need for physical calibration boards, I included a built-in local web server that displays rotating digital calibration patterns.

​

Through this solution, I deliver an end-to-end, integrated workflow that empowers users with freedom, control, and efficiency — transforming lens calibration from a complex, studio-bound task into a seamless, portable experience.

bottom of page