ICCV 2019 Tutorial on

Understanding Color and the In-Camera Image Processing Pipeline for Computer Vision

Oct 27, 2019 (Sunday, Half Day Tutorial - PM)
Instructor
Michael S. Brown
Professor, York University, Canada
Senior Director, Samsung AI Center - Toronto, Canada

Tutorial Description

Color is not a well-understood topic in computer vision. This tutorial aims to address this issue by providing a thorough background on color theory and its relationship to the in-camera processing pipeline and computer vision applications. The tutorial is organized into two parts. The first part provides a background on color theory and color representations, namely the CIE 1931 XYZ color space and its derivatives commonly found in computer vision (sRGB, L*ab, Yuv, etc.). The first part of the tutorial will also discuss routines applied onboard cameras to convert the low-level sensor raw-RGB responses to their final standard RGB (sRGB) colors. These routines include computational color constancy (auto white balance), colorimetric conversion, image demosaicing, image denoising, tone-mapping, super-resolution, and general color manipulation. The second part of this tutorial discusses recent research in the computer vision community on many of these camera pipeline components. The second part of the tutorial will also discuss various misconceptions about color and camera images made in many areas of computer vision.

Tentative Tutorial Schedule

Room 401

Part 1 (1.30PM-3.30PM) : Overview of Basic Color Manipulation and the Camera Imaging Pipeline (2 hours)
Level (Novice)

• Background on Color
• Human sensitivity
• CIE XYZ color matching functions and relationship to colorimetry
• Review of Color Spaces (CIE XYZ and derivatives, e.g., CIE L*ab, sRGB, NTSC, YUV, YIQ)
• Color constancy and its relationship to illuminations and color temperatures (CCT)
• Camera Pipeline Overview
• Flat field correction
• White balance
• Demosiacing
• Sensor characterization/Colorimetric conversion
• Tone Mapping
• Gamut Mapping/Color Preference
• ISO recommendations and Color Profiles

Coffee Break (3.30PM - 4.30PM)

Part 2 (4.30PM - 6.00PM) : Recent Computer Vision Research on Camera Pipeline Components (1.5 hours)
Level (Intermediate to Advanced)

• Common misconceptions in the computer vision literature regarding color
• Recent research on camera piplelines (including deep learning)
• Conclusion

Tutorial Slides

Link to PDF (link)

Additional Materials

Books
R.W.G. Hunt, The Reproduction of Colour, Wiley , 2004
G. Sharma, Digital Color Imaging Handbook, CRC Press , 2003
M. Fairchild, Color Appearance Models, Wiley , 2005
D. Forsyth and J. Ponce, Computer Vision: A modern approach, Prentice Hall, 2011
R. Lukac, Single-Sensor Imaging: Methods and Applications for Digital Cameras, CRC Press , 2008

Articles/Conference Papers
Abdelhamed A. et al. "A High-Quality Denoising Dataset for Smartphone Cameras", CVPR'18
Chakrabarti A. et al. "Modeling Radiometric Uncertainty for Vision with Tone-mapped Color Images", TPAMI 2014
Cheng D. et al. "Beyond White: Ground Truth Colors for Color Constancy Correction", ICCV'15
Garcia E. and Gupta M. "Building accurate and smooth ICC profiles by lattice regression", Color and Imaging Conference, 2009
Karaimer H. and Brown M.S. "Improving Color Reproduction Accuracy on Cameras", CVPR'18
Karaimer H. and Brown M.S. "A Software Platform for Manipulating the Camera Imaging Pipeline", ECCV'16
Kim S.J. et al. "A New In-Camera Imaging Model for Color Computer Vision and its Application", TPAMI'12
Lin H.T. et al. "Nonuniform Lattice Regression for Modeling the Camera Imaging Pipeline", ECCV'12
Ramantha R. et al. "Color Image Processing Pipeline: a general survey of digital still cameras", IEEE Signal Processing Magazine, 2005
Tai Y.-W. et al. "Nonlinear Camera Response Functions and Image Deblurring: Theoretical Analysis and Practice", TPAMI 2013

Preparation of this tutorial was supported by grants and gifts from NSERC, the NSERC Canada Reserach Chair program, the CFREF-VISTA program, Adobe Research, Google Research, Microsoft Research-Asia, and Samsung Research.