This textbook offers advanced content on computer vision (basic content can be found in its prerequisite textbook, "2D Computer Vision: Principles, Algorithms and Applications"), including the basic principles, typical methods and practical techniques. It is intended for graduate courses on related topics, e.g. Computer Vision, 3-D Computer Vision, Graphics, Artificial Intelligence, etc.
This book mainly covers the higher level of computer vision from the selection of materials. This book is self-contained, mainly for information majors, but also takes into account learners of different professional backgrounds, and also considers the needs of self-study readers. After learning the fundamental content of computer vision in this book, readers can carry out scientific research and solve more and even difficult specific problems in practical applications.
This book pays more attention to practicality in writing. Considering that computer vision technology has been involved in many professional fields in recent years, but many working people are not specialized in computer vision technology, this book does not emphasize the theoretical system too much, minimizes the formula derivation, and focuses on commonly used techniques. This book has many sample questions and uses intuitive explanation to help readers understand abstract concepts.
This book can consider three aspects from the knowledge requirements of the prerequisite courses: (1) Mathematics: including linear algebra and matrix theory, as well as basic knowledge about statistics, probability theory, and random modeling; (2) Computer science: including the mastery of computer software technology, the understanding of computer structure system, and the application of computer programming methods; (3) Electronics: On the one hand, the characteristics and principles of electronic equipment; on the other hand, circuit design and other content. In addition, the book 2D Computer Vision: Principles, Algorithms and Applications can be counted as the discipline prerequisite of this book.
Computer vision began to be studied as an Artificial Intelligence problem, so it is often called image understanding. In fact, the terms image understanding and computer vision are often mixed. In essence, they are connected to each other. In many cases, they have overlapped coverages and contents, and they have not absolute boundary in concept or practicality. In many occasions and situations, although they have their own emphasis, they often complement each other. Therefore, it is more appropriate to regard them as different terms used by people with different professions and backgrounds.
This book mainly introduces the three-dimensional part of computer vision and also corresponds to the basic concepts, basic theories, and practical techniques of high-level image engineering. Through the comprehensive use of these theories and technologies, various computer vision systems can be constructed to explore and solve practical application problems. In addition, through the introduction of the high-level content of image engineering, it can also help readers to obtain more information based on the results obtained from the low-level and middle-level technologies of image engineering, as well as combine and integrate the technologies at all levels.
Author(s): Yu-Jin Zhang
Publisher: Springer
Year: 2023
Language: English
Pages: 453
Preface
Contents
Chapter 1: Computer Vision Overview
1.1 Human Vision and Characteristics
1.1.1 Visual Characteristics
1.1.1.1 Vision and Other Sensations
1.1.1.2 Vision and Computer Vision
1.1.1.3 Vision and Machine Vision
1.1.1.4 Vision and Image Generation
1.1.2 Brightness Properties of Vision
1.1.2.1 Simultaneous Contrast
1.1.2.2 Mach Band Effect
1.1.2.3 Contrast Sensitivity
1.1.3 Spatial Properties of Vision
1.1.3.1 Spatial Cumulative Effect
1.1.3.2 Spatial Frequency
1.1.3.3 Visual Acuity
1.1.4 Temporal Properties of Vision
1.1.4.1 Visual Phenomena That Change Over Time
Brightness Adaptation
Time Resolution of the Eyes
1.1.4.2 Time Cumulative Effect
1.1.4.3 Time Frequency
1.1.5 Visual Perception
1.1.5.1 Visual Sensation and Visual Perception
1.1.5.2 The Complexity of Visual Perception
Perception of the Visual Edge
Perception of Brightness Contrast
1.2 Computer Vision Theory and Framework
1.2.1 Reaserch Goals, Tasks, and Methods of Computer Vision
1.2.2 Visual Computational Theory
1.2.2.1 Vision Is a Complex Information Processing Process
1.2.2.2 Three Essential Factors of Visual Information Processing
1.2.2.3 Three-Level Internal Expression of Visual Information
1.2.2.4 Visual Information Understanding Is Organized in the Form of Functional Modules
1.2.2.5 The Formal Representation of Computational Theory Must Consider Constraints
1.2.3 Framework Problems and Improvements
1.3 Three-Dimensional Vision System and Image Technology
1.3.1 Three-Dimensional Vision System Process
1.3.2 Computer Vision and Image Technology Levels
1.3.3 Image Technology Category
1.4 Overview of the Structure and Content of This Book
1.4.1 Structural Framework and Content of This Book
1.4.2 Chapter Overview
1.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 2: Camera Calibration
2.1 Linear Camera Model
2.1.1 Complete Imaging Model
2.1.2 Basic Calibration Procedure
2.1.3 Internal and External Parameters
2.1.3.1 External Parameters
2.1.3.2 Internal Parameters
2.2 Non-Linear Camera Model
2.2.1 Type of Distortion
2.2.1.1 Radial Distortion
2.2.1.2 Tangential Distortion
2.2.1.3 Eccentric Distortion
2.2.1.4 Thin Prism Distortion
2.2.2 Calibration Steps
2.2.3 Classification of Calibration Methods
2.3 Traditional Calibration Methods
2.3.1 Basic Steps and Parameters
2.3.2 Two-Stage Calibration Method
2.3.3 Precision Improvement
2.4 Self-Calibration Methods
2.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 3: Three-Dimensional Image Acquisition
3.1 High-Dimensional Image
3.2 Depth Image
3.2.1 Depth Image and Grayscale Image
3.2.2 Intrinsic Image and Non-Intrinsic Image
3.2.3 Depth Imaging Modes
3.3 Direct Depth Imaging
3.3.1 Time-of-Flight Method
3.3.1.1 Pulse Time Interval Measurement Method
3.3.1.2 Phase Measurement Method of Amplitude Modulation
3.3.1.3 Coherent Measurement Method of Frequency Modulation
3.3.2 Structured Light Method
3.3.2.1 Structured Light Imaging
3.3.2.2 Imaging Width
3.3.3 Moiré Contour Stripes Method
3.3.3.1 Basic Principles
3.3.3.2 Basic Method
3.3.3.3 Improvement Methods
3.3.4 Simultaneous Acquisition of Depth and Brightness Images
3.4 Stereo Vision Imaging
3.4.1 Binocular Horizontal Mode
3.4.1.1 Disparity and Depth
3.4.1.2 Angular Scanning Imaging
3.4.2 Binocular Convergence Horizontal Mode
3.4.2.1 Disparity and Depth
3.4.2.2 Image Rectification
3.4.3 Binocular Axial Mode
3.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 4: Video Image and Motion Information
4.1 Video Basic
4.1.1 Video Expression and Model
4.1.1.1 Video Representation Function
4.1.1.2 Video Color Model
4.1.1.3 Video Space Sampling Rate
4.1.2 Video Display and Format
4.1.2.1 Video Display
4.1.2.2 Video Bit Rate
4.1.2.3 Video Format
4.1.3 Color TV System
4.2 Motion Classification and Representation
4.2.1 Motion Classification
4.2.2 Motion Vector Field Representation
4.2.3 Motion Histogram Representation
4.2.3.1 Histogram of Motion Vector Direction
4.2.3.2 Histogram of Movement Area Types
4.2.4 Motion Track Description
4.3 Motion Information Detection
4.3.1 Motion Detection Based on Camera Model
4.3.1.1 Camera Motion Type
4.3.1.2 Motion Camera
4.3.2 Frequency Domain Motion Detection
4.3.2.1 Detection of Translation
4.3.2.2 Detection of Rotation
4.3.2.3 Detection of Scale Changes
4.3.3 Detection of Movement Direction
4.4 Motion-Based Filtering
4.4.1 Motion Detection-Based Filtering
4.4.1.1 Direct Filtering
4.4.1.2 Using Motion Detection Information
4.4.2 Motion Compensation-Based Filtering
4.4.2.1 Motion Trajectory and Time-Space Spectrum
4.4.2.2 Filtering Along the Motion Trajectory
4.4.2.3 Motion Compensation Filter
4.4.2.4 Spatial-Temporal Adaptive Linear Minimum Mean Square Error Filtering
4.4.2.5 Adaptive Weighted Average Filtering
4.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 5: Moving Object Detection and Tracking
5.1 Differential Image
5.1.1 Calculation of Difference Image
5.1.2 Calculation of Accumulative Difference Image
5.2 Background Modeling
5.2.1 Basic Principle
5.2.2 Typical Practical Methods
5.2.2.1 Method Based on Single Gaussian Model
5.2.2.2 Method Based on Video Initialization
5.2.2.3 Method Based on Gaussian Mixture Model
5.2.2.4 Method Based on Codebook
5.2.3 Effect Examples
5.2.3.1 No Moving Foreground in Static Background
5.2.3.2 There Is a Moving Foreground in a Static Background
5.2.3.3 There Is a Moving Foreground in the Moving Background
5.3 Optical Flow Field and Motion
5.3.1 Optical Flow Equation
5.3.2 Optical Flow Estimation with Least Square Method
5.3.3 Optical Flow in Motion Analysis
5.3.3.1 Mutual Velocity
5.3.3.2 Focus of Expansion
5.3.3.3 Collision Distance
5.3.4 Dense Optical Flow Algorithm
5.3.4.1 Solving the Optical Flow Equation
5.3.4.2 Global Motion Compensation
5.4 Moving Object Tracking
5.4.1 Kalman Filter
5.4.2 Particle Filter
5.4.3 Mean Shift and Kernel Tracking
5.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 6: Binocular Stereo Vision
6.1 Stereo Vision Process and Modules
6.1.1 Camera Calibration
6.1.2 Image Acquisition
6.1.3 Feature Extraction
6.1.4 Stereo Matching
6.1.5 3-D Information Recovery
6.1.6 Post-Processing
6.1.6.1 Depth Interpolation
6.1.6.2 Error Correction
6.1.6.3 Precision Improvement
6.2 Region-Based Stereo Matching
6.2.1 Template Matching
6.2.2 Stereo Matching
6.2.2.1 Epipolar Line Constraint
6.2.2.2 Essential Matrix and Fundamental Matrix
6.2.2.3 Influencing Factors in Matching
6.2.2.4 Calculation of Surface Optical Properties
6.3 Feature-Based Stereo Matching
6.3.1 Basic Steps and Methods
6.3.1.1 Matching with Edge Points
6.3.1.2 Matching with Zero-Crossing Points
6.3.1.3 Depth of Feature Points
6.3.1.4 Sparse Matching Points
6.3.2 Scale Invariant Feature Transformation
6.3.3 Dynamic Programming Matching
6.4 Error Detection and Correction of Parallax Map
6.4.1 Error Detection
6.4.2 Error Correction
6.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 7: Monocular Multiple Image Recovery
7.1 Photometric Stereo
7.1.1 Light Source, Scenery, Lens
7.1.2 Scene Brightness and Image Brightness
7.1.2.1 The Relationship Between Scene Brightness and Image Brightness
7.1.2.2 Bidirectional Reflectance Distribution Function
7.1.3 Surface Reflection Characteristics and Brightness
7.1.3.1 Ideal Scattering Surface
7.1.3.2 Ideal Specular Reflecting Surface
7.2 Shape from Illumination
7.2.1 Representation of the Surface Orientation of a Scene
7.2.2 Reflectance Map and Brightness Constraint Equation
7.2.2.1 Reflection Map
7.2.2.2 Image Brightness Constraint Equation
7.2.3 Solution of Photometric Stereo
7.3 Optical Flow Equation
7.3.1 Optical Flow and Motion Field
7.3.2 Solving Optical Flow Equation
7.3.2.1 Optical Flow Calculation: Rigid Body Motion
7.3.2.2 Optical Flow Calculation: Smooth Motion
7.3.2.3 Optical Flow Calculation: Gray Level Mutation
7.3.2.4 Optical Flow Calculation: Based on High-Order Gradient
7.4 Shape from Motion
7.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 8: Monocular Single Image Recovery
8.1 Shape from Shading
8.1.1 Shading and Orientation
8.1.2 Gradient Space Method
8.2 Solving Brightness Equation
8.2.1 Linearity Case
8.2.2 Rotational Symmetry Case
8.2.3 The General Case of Smoothness Constraints
8.3 Shape from Texture
8.3.1 Monocular Imaging and Distortion
8.3.2 Orientation Restoration from the Change of Texture
8.3.2.1 Three Typical Methods
8.3.2.2 Shape from Texture
Isotropic Assumption
Homogeneity Assumption
8.3.2.3 Texture Stereo Technology
8.4 Detection of Texture Vanishing Points
8.4.1 Detecting the Vanishing Point of Line Segment Texture
8.4.2 Determine the Vanishing Point Outside the Image
8.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 9: Three-Dimensional Scenery Representation
9.1 Local Features of the Surface
9.1.1 Surface Normal Section
9.1.2 Surface Principal Curvature
9.1.3 Mean Curvature and Gaussian Curvature
9.2 Three-Dimensional Surface Representation
9.2.1 Parameter Representation
9.2.1.1 The Parameter Representation of the Curve
9.2.1.2 Parameter Representation of Curved Surface
9.2.2 Surface Orientation Representation
9.2.2.1 Extended Gaussian Image
9.2.2.2 Spherical Projection and Stereographic Projection
9.3 Construction and Representation of Iso-surfaces
9.3.1 Marching Cube Algorithm
9.3.2 Wrapper Algorithm
9.4 Interpolating Three-Dimensional Surfaces from Parallel Contours
9.4.1 Contour Interpolation and Tiling
9.4.2 Problems That May Be Encountered
9.4.2.1 Corresponding Problems
9.4.2.2 Tiling Problem
9.4.2.3 Branching Problem
9.4.3 Delaunay Triangulation and Neighborhood Voronoï Diagram
9.5 Three-Dimensional Entity Representation
9.5.1 Basic Representation Scheme
9.5.1.1 Spatial Occupancy Array
9.5.1.2 Cell Decomposition
9.5.1.3 Geometric Model Method
9.5.2 Generalized Cylinder Representation
9.6 Key Points and References for Each Section
Self-Test Questions
References
Chapter 10: Generalized Matching
10.1 Matching Overview
10.1.1 Matching Strategies and Categories
10.1.1.1 Matching in Image Space
10.1.1.2 Matching in Object Space
10.1.1.3 Matching Based on Raster
10.1.1.4 Feature-Based Matching
10.1.1.5 Matching Based on Relationship
10.1.2 Matching and Registration
10.1.3 Matching Evaluation
10.2 Object Matching
10.2.1 Measure of Matching
10.2.1.1 Hausdorff Distance
10.2.1.2 Structural Matching Measure
10.2.2 Corresponding Point Matching
10.2.3 String Matching
10.2.4 Matching of Inertia Equivalent Ellipses
10.2.5 Shape Matrix Matching
10.3 Dynamic Pattern Matching
10.3.1 Matching Process
10.3.2 Absolute Pattern and Relative Pattern
10.4 Graph Theory and Graph Matching
10.4.1 Introduction to Graph Theory
10.4.1.1 Basic Definition
10.4.1.2 The Geometric Representation of the Graph
10.4.1.3 Colored Graph
10.4.1.4 Sub-Graph
10.4.2 Graph Isomorphism and Matching
10.4.2.1 The Identity and Isomorphism of Graph
10.4.2.2 Determination of Isomorphism
10.5 Line Drawing Signature and Matching
10.5.1 Contour Marking
10.5.1.1 Blade
10.5.1.2 Limb
10.5.1.3 Crease
10.5.1.4 Mark
10.5.1.5 Shade/Shadow
10.5.2 Structural Reasoning
10.5.3 Labeling via Backtracking
10.6 Key Points and References for Each Section
Self-Test Questions
References
Chapter 11: Knowledge and Scene Interpretation
11.1 Scene Knowledge
11.1.1 Model
11.1.2 Attribute Hypergraph
11.1.3 Knowledge-Based Modeling
11.2 Logic System
11.2.1 Predicate Calculation Rules
11.2.2 Inference by Theorem Proving
11.3 Fuzzy Reasoning
11.3.1 Fuzzy Sets and Fuzzy Operations
11.3.2 Fuzzy Reasoning Method
11.3.2.1 Basic Model
11.3.2.2 Fuzzy Combination
11.3.2.3 De-fuzzification
11.4 Scene Classification
11.4.1 Bag-of-Words/Feature Model
11.4.2 pLSA Model
11.4.2.1 Model Description
11.4.2.2 Model Calculation
11.4.2.3 Model Application Example
11.5 Key Points and References for Each Section
Self-Test Questions
References
Chapter 12: Spatial-Temporal Behavior Understanding
12.1 Spatial-Temporal Technology
12.1.1 New Research Field
12.1.2 Multiple Levels
12.2 Spatial-Temporal Interest Point Detection
12.2.1 Detection of Points of Interest in Space
12.2.2 Detection of Points of Interest in Space and Time
12.3 Spatial-Temporal Dynamic Trajectory Learning and Analysis
12.3.1 Automatic Scene Modeling
12.3.1.1 Object Tracking
12.3.1.2 Point of Interest Detection
12.3.1.3 Activity Path Learning
12.3.2 Path Learning
12.3.2.1 Trajectory Preprocessing
12.3.2.2 Trajectory Clustering
12.3.2.3 Path Modeling
12.3.3 Automatic Activity Analysis
12.4 Spatial-Temporal Action Classification and Recognition
12.4.1 Motion Classification
12.4.1.1 Direct Classification
12.4.1.2 Time State Model
12.4.1.3 Motion Detection
12.4.2 Action Recognition
12.4.2.1 Holistic Recognition
12.4.2.2 Posture Modeling
12.4.2.3 Activity Reconstruction
12.4.2.4 Interactive Activities
12.4.2.5 Group Activities
12.4.2.6 Scene Interpretation
12.5 Key Points and References for Each Section
Self-Test Questions
References
Appendix A: Visual Perception
A.1 Shape Perception
A.2 Spatial Perception
A.2.1 Nonvisual Indices of Depth
A.2.2 Binocular Indices of Depth
A.2.3 Monocular Indices of Depth
A.3 Motion Perception
A.3.1 The Condition of Motion Perception
A.3.2 Detection of Moving Objects
A.3.3 Depth Motion Detection
A.3.4 Real Motion and Apparent Motion
A.3.5 Correspondence Matching of Apparent Motion
A.3.6 Aperture Problem
A.3.7 Dynamic Indices of Depth
A.4 Key Points and References for Each Section
References
Answers to Self-Test Questions
Chapter 1 Computer Vision Overview
Chapter 2 Camera Calibration
Chapter 3 Three-Dimensional Image Acquisition
Chapter 4 Video Image and Motion Information
Chapter 5 Moving Object Detection and Tracking
Chapter 6 Binocular Stereo Vision
Chapter 7 Monocular Multiple Image Recovery
Chapter 8 Monocular Single Image Recovery
Chapter 9 Three-Dimensional Scene Representation
Chapter 10 Scene Matching
Chapter 11 Knowledge and Scene Interpretation
Chapter 12 Spatial-Temporal Behavior Understanding
Index
