Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library

Copyright
Table of Contents
Preface
Purpose of This Book
Who This Book Is For
What This Book Is Not
About the Programs in This Book
Prerequisites
How This Book Is Best Used
Conventions Used in This Book
Using Code Examples
O’Reilly Safari
We’d Like to Hear from You
Acknowledgments
Thanks for Help on OpenCV
Thanks for Help on This Book
Adrian Adds...
Gary Adds...
Chapter 1. Overview
What Is OpenCV?
Who Uses OpenCV?
What Is Computer Vision?
The Origin of OpenCV
OpenCV Block Diagram
Speeding Up OpenCV with IPP
Who Owns OpenCV?
Downloading and Installing OpenCV
Installation
Getting the Latest OpenCV via Git
More OpenCV Documentation
Supplied Documentation
Online Documentation and the Wiki
OpenCV Contribution Repository
Downloading and Building Contributed Modules
Portability
Summary
Exercises
Chapter 2. Introduction to OpenCV
Include Files
Resources
First Program—Display a Picture
Second Program—Video
Moving Around
A Simple Transformation
A Not-So-Simple Transformation
Input from a Camera
Writing to an AVI File
Summary
Exercises
Chapter 3. Getting to Know OpenCV Data Types
The Basics
OpenCV Data Types
Overview of the Basic Types
Basic Types: Getting Down to Details
Helper Objects
Utility Functions
The Template Structures
Summary
Exercises
Chapter 4. Images and Large Array Types
Dynamic and Variable Storage
The cv::Mat Class: N-Dimensional Dense Arrays
Creating an Array
Accessing Array Elements Individually
The N-ary Array Iterator: NAryMatIterator
Accessing Array Elements by Block
Matrix Expressions: Algebra and cv::Mat
Saturation Casting
More Things an Array Can Do
The cv::SparseMat Class: Sparse Arrays
Accessing Sparse Array Elements
Functions Unique to Sparse Arrays
Template Structures for Large Array Types
Summary
Exercises
Chapter 5. Array Operations
More Things You Can Do with Arrays
cv::abs()
cv::absdiff()
cv::add()
cv::addWeighted()
cv::bitwise_and()
cv::bitwise_not()
cv::bitwise_or()
cv::bitwise_xor()
cv::calcCovarMatrix()
cv::cartToPolar()
cv::checkRange()
cv::compare()
cv::completeSymm()
cv::convertScaleAbs()
cv::countNonZero()
cv::cvarrToMat()
cv::dct()
cv::dft()
cv::cvtColor()
cv::determinant()
cv::divide()
cv::eigen()
cv::exp()
cv::extractImageCOI()
cv::flip()
cv::gemm()
cv::getConvertElem() and cv::getConvertScaleElem()
cv::idct()
cv::idft()
cv::inRange()
cv::insertImageCOI()
cv::invert()
cv::log()
cv::LUT()
cv::magnitude()
cv::Mahalanobis()
cv::max()
cv::mean()
cv::meanStdDev()
cv::merge()
cv::min()
cv::minMaxIdx()
cv::minMaxLoc()
cv::mixChannels()
cv::mulSpectrums()
cv::multiply()
cv::mulTransposed()
cv::norm()
cv::normalize()
cv::perspectiveTransform()
cv::phase()
cv::polarToCart()
cv::pow()
cv::randu()
cv::randn()
cv::randShuffle()
cv::reduce()
cv::repeat()
cv::scaleAdd()
cv::setIdentity()
cv::solve()
cv::solveCubic()
cv::solvePoly()
cv::sort()
cv::sortIdx()
cv::split()
cv::sqrt()
cv::subtract()
cv::sum()
cv::trace()
cv::transform()
cv::transpose()
Summary
Exercises
Chapter 6. Drawing and Annotating
Drawing Things
Line Art and Filled Polygons
Fonts and Text
Summary
Exercises
Chapter 7. Functors in OpenCV
Objects That “Do Stuff”
Principal Component Analysis (cv::PCA)
Singular Value Decomposition (cv::SVD)
Random Number Generator (cv::RNG)
Summary
Exercises
Chapter 8. Image, Video, and Data Files
HighGUI: Portable Graphics Toolkit
Working with Image Files
Loading and Saving Images
A Note About Codecs
Compression and Decompression
Working with Video
Reading Video with the cv::VideoCapture Object
Writing Video with the cv::VideoWriter Object
Data Persistence
Writing to a cv::FileStorage
Reading from a cv::FileStorage
cv::FileNode
Summary
Exercises
Chapter 9. Cross-Platform and Native Windows
Working with Windows
HighGUI Native Graphical User Interface
Working with the Qt Backend
Integrating OpenCV with Full GUI Toolkits
Summary
Exercises
Chapter 10. Filters and Convolution
Overview
Before We Begin
Filters, Kernels, and Convolution
Border Extrapolation and Boundary Conditions
Threshold Operations
Otsu’s Algorithm
Adaptive Threshold
Smoothing
Simple Blur and the Box Filter
Median Filter
Gaussian Filter
Bilateral Filter
Derivatives and Gradients
The Sobel Derivative
Scharr Filter
The Laplacian
Image Morphology
Dilation and Erosion
The General Morphology Function
Opening and Closing
Morphological Gradient
Top Hat and Black Hat
Making Your Own Kernel
Convolution with an Arbitrary Linear Filter
Applying a General Filter with cv::filter2D()
Applying a General Separable Filter with cv::sepFilter2D
Kernel Builders
Summary
Exercises
Chapter 11. General Image Transforms
Overview
Stretch, Shrink, Warp, and Rotate
Uniform Resize
Image Pyramids
Nonuniform Mappings
Affine Transformation
Perspective Transformation
General Remappings
Polar Mappings
LogPolar
Arbitrary Mappings
Image Repair
Inpainting
Denoising
Histogram Equalization
cv::equalizeHist(): Contrast equalization
Summary
Exercises
Chapter 12. Image Analysis
Overview
Discrete Fourier Transform
cv::dft(): The Discrete Fourier Transform
cv::idft(): The Inverse Discrete Fourier Transform
cv::mulSpectrums(): Spectrum Multiplication
Convolution Using Discrete Fourier Transforms
cv::dct(): The Discrete Cosine Transform
cv::idct(): The Inverse Discrete Cosine Transform
Integral Images
cv::integral() for Standard Summation Integral
cv::integral() for Squared Summation Integral
cv::integral() for Tilted Summation Integral
The Canny Edge Detector
cv::Canny()
Hough Transforms
Hough Line Transform
Hough Circle Transform
Distance Transformation
cv::distanceTransform() for Unlabeled Distance Transform
cv::distanceTransform() for Labeled Distance Transform
Segmentation
Flood Fill
Watershed Algorithm
Grabcuts
Mean-Shift Segmentation
Summary
Exercises
Chapter 13. Histograms and Templates
Histogram Representation in OpenCV
cv::calcHist(): Creating a Histogram from Data
Basic Manipulations with Histograms
Histogram Normalization
Histogram Threshold
Finding the Most Populated Bin
Comparing Two Histograms
Histogram Usage Examples
Some More Sophisticated Histograms Methods
Earth Mover’s Distance
Back Projection
Template Matching
Square Difference Matching Method (cv::TM_SQDIFF)
Normalized Square Difference Matching Method (cv::TM_SQDIFF_NORMED)
Correlation Matching Methods (cv::TM_CCORR)
Normalized Cross-Correlation Matching Method (cv::TM_CCORR_NORMED)
Correlation Coefficient Matching Methods (cv::TM_CCOEFF)
Normalized Correlation Coefficient Matching Method (cv::TM_CCOEFF_NORMED)
Summary
Exercises
Chapter 14. Contours
Contour Finding
Contour Hierarchies
Drawing Contours
A Contour Example
Another Contour Example
Fast Connected Component Analysis
More to Do with Contours
Polygon Approximations
Geometry and Summary Characteristics
Geometrical Tests
Matching Contours and Images
Moments
More About Moments
Matching and Hu Moments
Using Shape Context to Compare Shapes
Summary
Exercises
Chapter 15. Background Subtraction
Overview of Background Subtraction
Weaknesses of Background Subtraction
Scene Modeling
A Slice of Pixels
Frame Differencing
Averaging Background Method
Accumulating Means, Variances, and Covariances
A More Advanced Background Subtraction Method
Structures
Learning the Background
Learning with Moving Foreground Objects
Background Differencing: Finding Foreground Objects
Using the Codebook Background Model
A Few More Thoughts on Codebook Models
Connected Components for Foreground Cleanup
A Quick Test
Comparing Two Background Methods
OpenCV Background Subtraction Encapsulation
The cv::BackgroundSubtractor Base Class
KaewTraKuPong and Bowden Method
Zivkovic Method
Summary
Exercises
Chapter 16. Keypoints and Descriptors
Keypoints and the Basics of Tracking
Corner Finding
Introduction to Optical Flow
Lucas-Kanade Method for Sparse Optical Flow
Generalized Keypoints and Descriptors
Optical Flow, Tracking, and Recognition
How OpenCV Handles Keypoints and Descriptors, the General Case
Core Keypoint Detection Methods
Keypoint Filtering
Matching Methods
Displaying Results
Summary
Exercises
Chapter 17. Tracking
Concepts in Tracking
Dense Optical Flow
The Farnebäck Polynomial Expansion Algorithm
The Dual TV-L1 Algorithm
The Simple Flow Algorithm
Mean-Shift and Camshift Tracking
Mean-Shift
Camshift
Motion Templates
Estimators
The Kalman Filter
A Brief Note on the Extended Kalman Filter
Summary
Exercises
Chapter 18. Camera Models and Calibration
Camera Model
The Basics of Projective Geometry
Rodrigues Transform
Lens Distortions
Calibration
Rotation Matrix and Translation Vector
Calibration Boards
Homography
Camera Calibration
Undistortion
Undistortion Maps
Converting Undistortion Maps Between Representations with cv::convertMaps()
Computing Undistortion Maps with cv::initUndistortRectifyMap()
Undistorting an Image with cv::remap()
Undistortion with cv::undistort()
Sparse Undistortion with cv::undistortPoints()
Putting Calibration All Together
Summary
Exercises
Chapter 19. Projection and Three-Dimensional Vision
Projections
Affine and Perspective Transformations
Bird’s-Eye-View Transform Example
Three-Dimensional Pose Estimation
Pose Estimation from a Single Camera
Stereo Imaging
Triangulation
Epipolar Geometry
The Essential and Fundamental Matrices
Computing Epipolar Lines
Stereo Calibration
Stereo Rectification
Stereo Correspondence
Stereo Calibration, Rectification, and Correspondence Code Example
Depth Maps from Three-Dimensional Reprojection
Structure from Motion
Fitting Lines in Two and Three Dimensions
Summary
Exercises
Chapter 20. The Basics of Machine Learning in OpenCV
What Is Machine Learning?
Training and Test Sets
Supervised and Unsupervised Learning
Generative and Discriminative Models
OpenCV ML Algorithms
Using Machine Learning in Vision
Variable Importance
Diagnosing Machine Learning Problems
Legacy Routines in the ML Library
K-Means
Mahalanobis Distance
Summary
Exercises
Chapter 21. StatModel: The Standard Model for Learning in OpenCV
Common Routines in the ML Library
Training and the cv::ml::TrainData Structure
Prediction
Machine Learning Algorithms Using cv::StatModel
Naïve/Normal Bayes Classifier
Binary Decision Trees
Boosting
Random Trees
Expectation Maximization
K-Nearest Neighbors
Multilayer Perceptron
Support Vector Machine
Summary
Exercises
Chapter 22. Object Detection
Tree-Based Object Detection Techniques
Cascade Classifiers
Supervised Learning and Boosting Theory
Learning New Objects
Object Detection Using Support Vector Machines
Latent SVM for Object Detection
The Bag of Words Algorithm and Semantic Categorization
Summary
Exercises
Chapter 23. Future of OpenCV
Past and Present
OpenCV 3.x
How Well Did Our Predictions Go Last Time?
Future Functions
Current GSoC Work
Community Contributions
OpenCV.org
Some AI Speculation
Afterword
Appendix A. Planar Subdivisions
Delaunay Triangulation, Voronoi Tesselation
Creating a Delaunay or Voronoi Subdivision
Navigating Delaunay Subdivisions
Usage Examples
Exercises
Appendix B. opencv_contrib
An Overview of the opencv_contrib Modules
Contents of opencv_contrib
Appendix C. Calibration Patterns
Calibration Patterns Used by OpenCV
Bibliography
Index
About the Authors
Colophon