This textbook introduces fundamental concepts, major models, and popular applications of pattern recognition for a one-semester undergraduate course. To ensure student understanding, the text focuses on a relatively small number of core concepts with an abundance of illustrations and examples. Concepts are reinforced with hands-on exercises to nurture the student's skill in problem solving. New concepts and algorithms are framed by real-world context and established as part of the big picture introduced in an early chapter. A problem-solving strategy is employed in several chapters to equip students with an approach for new problems in pattern recognition. This text also points out common errors that a new player in pattern recognition may encounter, and fosters the ability for readers to find useful resources and independently solve a new pattern recognition task through various working examples. Students with an undergraduate understanding of mathematical analysis, linear algebra, and probability will be well prepared to master the concepts and mathematical analysis presented here.
Author(s): Jianxin Wu
Edition: 1
Publisher: Cambridge University Press
Year: 2020
Language: English
Commentary: Publisher PDF
Pages: 398
City: Cambridge
Tags: Computational Statistics; Machine Learning and Information Science; Computer Science; Machine Learning and Pattern Recognition; Statistics and Probability
Cover
Half-title
Title page
Copyright information
Contents
List of Figures
List of Tables
Preface
Notation
Part I Introduction and Overview
1 Introduction
1.1 An Example: Autonomous Driving
1.2 Pattern Recognition and Machine Learning
Exercises
2 Mathematical Background
2.1 Linear Algebra
2.2 Probability
2.3 Optimization and Matrix Calculus
2.4 Complexity of Algorithms
2.5 Miscellaneous Notes and Additional Resources
Exercises
3 Overview of a Pattern Recognition System
3.1 Face Recognition
3.2 A Simple Nearest Neighbor Classifier
3.3 The Ugly Details
3.4 Making Assumptions and Simplifications
3.5 A Framework
3.6 Miscellaneous Notes and Additional Resources
Exercises
4 Evaluation
4.1 Accuracy and Error in the Simple Case
4.2 Minimizing the Cost/Loss
4.3 Evaluation in Imbalanced Problems
4.4 Can We Reach 100% Accuracy?
4.5 Confidence in the Evaluation Results
4.6 Miscellaneous Notes and Additional Resources
Exercises
Part II Domain-Independent Feature Extraction
5 Principal Component Analysis
5.1 Motivation
5.2 PCA to Zero-Dimensional Subspace
5.3 PCA to One-Dimensional Subspace
5.4 PCA for More Dimensions
5.5 The Complete PCA Algorithm
5.6 Variance Analysis
5.7 When to Use or Not to Use PCA?
5.8 The Whitening Transform
5.9 Eigen-Decomposition vs. SVD
5.10 Miscellaneous Notes and Additional Resources
Exercises
6 Fisher's Linear Discriminant
6.1 FLD for Binary Classification
6.2 FLD for More Classes
6.3 Miscellaneous Notes and Additional Resources
Exercises
Part III Classifiers and Tools
7 Support Vector Machines
7.1 The Key SVM Idea
7.2 Visualizing and Calculating the Margin
7.3 Maximizing the Margin
7.4 The Optimization and the Solution
7.5 Extensions for Linearly Inseparable and Multiclass Problems
7.6 Kernel SVMs
7.7 Miscellaneous Notes and Additional Resources
Exercises
8 Probabilistic Methods
8.1 The Probabilistic Way of Thinking
8.2 Choices
8.3 Parametric Estimation
8.4 Nonparametric Estimation
8.5 Making Decisions
8.6 Miscellaneous Notes and Additional Resources
Exercises
9 Distance Metrics and Data Transformations
9.1 Distance Metrics and Similarity Measures
9.2 Data Transformation and Normalization
9.3 Miscellaneous Notes and Additional Resources
Exercises
10 Information Theory and Decision Trees
10.1 Prefix Code and Huffman Tree
10.2 Basics of Information Theory
10.3 Information Theory for Continuous Distributions
10.4 Information Theory in ML and PR
10.5 Decision Trees
10.6 Miscellaneous Notes and Additional Resources
Exercises
Part IV Handling Diverse Data Formats
11 Sparse and Misaligned Data
11.1 Sparse Machine Learning
11.2 Dynamic Time Warping
11.3 Miscellaneous Notes and Additional Resources
Exercises
12 Hidden Markov Model
12.1 Sequential Data and the Markov Property
12.2 Three Basic Problems in HMM Learning
12.3 α, β, and the Evaluation Problem
12.4 γ, δ, ψ, and the Decoding Problem
12.5 ξ and Learning HMM Parameters
12.6 Miscellaneous Notes and Additional Resources
Exercises
Part V Advanced Topics
13 The Normal Distribution
13.1 Definition
13.2 Notation and Parameterization
13.3 Linear Operation and Summation
13.4 Geometry and the Mahalanobis Distance
13.5 Conditioning
13.6 Product of Gaussians
13.7 Application I: Parameter Estimation
13.8 Application II: Kalman Filter
13.9 Useful Math in This Chapter
Exercises
14 The Basic Idea behind Expectation-Maximization
14.1 GMM: A Worked Example
14.2 An Informal Description of the EM Algorithm
14.3 The Expectation-Maximization Algorithm
14.4 EM for GMM
14.5 Miscellaneous Notes and Additional Resources
Exercises
15 Convolutional Neural Networks
15.1 Preliminaries
15.2 CNN Overview
15.3 Layer Input, Output, and Notation
15.4 The ReLU Layer
15.5 The Convolution Layer
15.6 The Pooling Layer
15.7 A Case Study: The VGG16 Net
15.8 Hands-On CNN Experiences
15.9 Miscellaneous Notes and Additional Resources
Exercises
Bibliography
Index
