*Approaches pattern recognition from the designer's point of view
*New edition highlights latest developments in this growing field, including independent components and support vector machines, not available elsewhere
*Supplemented by computer examples selected from applications of interest
Pattern recognition is a scientific discipline that is becoming increasingly important in the age of automation and information handling and retrieval. This volume's unifying treatment covers the entire spectrum of pattern recognition applications, from image analysis to speech recognition and communications. This book presents cutting-edge material on neural networks, - a set of linked microprocessors that can form associations and uses pattern recognition to "learn". A direct result of more than 10 years of teaching experience, the text was developed by the authors through use in their own classrooms.
*Approaches pattern recognition from the designer's point of view
*New edition highlights latest developments in this growing field, including independent components and support vector machines, not available elsewhere
*Supplemented by computer examples selected from applications of interest
Author(s): Sergios Theodoridis, Konstantinos Koutroumbas
Edition: 2
Publisher: Academic Press
Year: 2003
Language: English
Pages: 689
Cover......Page 1
Half Title Page......Page 4
Title Page......Page 6
Copyright......Page 7
Table of Contents......Page 8
Preface......Page 16
1.1 Is Pattern Recognition Important?......Page 18
1.2 Features, Feature Vectors, and Classifiers......Page 20
1.3 Supervised versus Unsupervised Pattern Recognition......Page 23
1.4 Outline of the Book......Page 25
2.2 Bayes Decision Theory......Page 30
2.3 Discriminant Functions and Decision Surfaces......Page 36
2.4 Bayesian Classification for Normal Distributions......Page 37
2.5 Estimation of Unknown Probability Density Functions......Page 44
2.6 The Nearest Neighbor Rule......Page 61
3.2 Linear Discriminant Functions and Decision Hyperplanes......Page 72
3.3 The Perceptron Algorithm......Page 74
3.4 Least Squares Methods......Page 82
3.5 Mean Square Estimation Revisited......Page 89
3.6 Support Vector Machines......Page 94
4.2 The XOR Problem......Page 110
4.3 The Two-Layer Perceptron......Page 111
4.4 Three-Layer Perceptrons......Page 118
4.5 Algorithms Based on Exact Classification of the Training Set......Page 119
4.6 The Backpropagation Algorithm......Page 121
4.7 Variations on the Backpropagation Theme......Page 129
4.8 The Cost Function Choice......Page 132
4.9 Choice of the Network Size......Page 135
4.10 A Simulation Example......Page 141
4.11 Networks with Weight Sharing......Page 143
4.12 Generalized Linear Classifiers......Page 144
4.13 Capacity of the l-Dimensional Space in Linear Dichotomies......Page 146
4.14 Polynomial Classifiers......Page 148
4.15 Radial Basis Function Networks......Page 150
4.16 Universal Approximators......Page 154
4.17 Support Vector Machines: The Nonlinear Case......Page 156
4.18 Decision Trees......Page 160
4.19 Discussion......Page 167
5.1 Introduction......Page 180
5.2 Preprocessing......Page 181
5.3 Feature Selection Based on Statistical Hypothesis Testing......Page 183
5.4 The Receiver Operating Characteristics CROC Curve......Page 190
5.5 Class Separability Measures......Page 191
5.6 Feature Subset Selection......Page 198
5.7 Optimal Feature Generation......Page 204
5.8 Neural Networks and Feature Generation/Selection......Page 208
5.9 A Hint on the Vapnik-Chernovenkis Learning Theory......Page 210
6.1 Introduction......Page 224
6.2 Basis Vectors and Images......Page 225
6.3 The Karhunen-Love Transform......Page 0
6.4 The Singular Value Decomposition......Page 232
6.5 Independent Component Analysis......Page 236
6.6 The Discrete Fourier Transform (DFT)......Page 243
6.7 The Discrete Cosine and Sine Transforms......Page 247
6.8 The Hadamard Transform......Page 248
6.9 The Haar Transform......Page 250
6.10 The Haar Expansion Revisited......Page 252
6.11 Discrete Time Wavelet Transform (DTWT)......Page 256
6.12 The Multiresolution Interpretation......Page 266
6.14 A Look at Two-Dimensional Generalizations......Page 269
6.15 Applications......Page 272
7.1 Introduction......Page 286
7.2 Regional Features......Page 287
7.3 Features for Shape and Size Characterization......Page 311
7.4 A Glimpse at Fractals......Page 320
8.1 Introduction......Page 338
8.2 Measures Based on Optimal Path Searching Techniques......Page 339
8.3 Measures Based on Correlations......Page 354
8.4 Deformable Template Models......Page 360
9.2 The Bayes Classifier......Page 368
9.3 Markov Chain Models......Page 369
9.4 The Viterbi Algorithm......Page 370
9.5 Channel Equalization......Page 373
9.6 Hidden Markov Models......Page 378
9.7 Training Markov Models via Neural Networks......Page 390
9.8 A Discussion of Markov Random Fields......Page 392
10.2 Error Counting Approach......Page 402
10.3 Exploiting the Finite Size of the Data Set......Page 404
10.4 A Case Study from Medical Imaging......Page 407
11.1 Introduction......Page 414
11.2 Proximity Measures......Page 421
12.1 Introduction......Page 446
12.2 Categories of Clustering Algorithms......Page 448
12.3 Sequential Clustering Algorithms......Page 450
12.4 A Modification of BSAS......Page 454
12.5 A Two-Threshold Sequential Scheme......Page 455
12.6 Refinement Stages......Page 458
12.7 Neural Network Implementation......Page 460
13.1 Introduction......Page 466
13.2 Agglomerative Algorithms......Page 467
13.3 The Cophenetic Matrix......Page 493
13.4 Divisive Algorithms......Page 495
13.5 Choice of the Best Number of Clusters......Page 497
14.1 Introduction......Page 506
14.2 Mixture Decomposition Schemes......Page 508
14.3 Fuzzy Clustering Algorithms......Page 517
14.4 Possibilistic Clustering......Page 539
14.5 Hard Clustering Algorithms......Page 546
14.6 Vector Quantization......Page 550
15.2 Clustering Algorithms Based on Graph Theory......Page 562
15.3 Competitive Learning Algorithms......Page 569
15.4 Branch and Bound Clustering Algorithms......Page 578
15.5 Binary Morphology Clustering Algorithms (BMCAs)......Page 581
15.6 Boundary Detection Algorithms......Page 590
15.7 Valley-Seeking Clustering Algorithms......Page 593
15.8 Clustering via Cost Optimization (Revisited)......Page 595
15.9 Clustering Using Genetic Algorithms......Page 599
15.10 Other Clustering Algorithms......Page 600
16.1 Introduction......Page 608
16.2 Hypothesis Testing Revisited......Page 609
16.3 Hypothesis Testing in Cluster Validity......Page 611
16.4 Relative Criteria......Page 622
16.5 Validity of Individual Clusters......Page 638
16.6 Clustering Tendency......Page 641
Appendix A: Hints from Probability and Statistics......Page 660
Appendix B: Linear Algebra Basics......Page 672
Appendix C: Cost Function Optimization......Page 676
Appendix D: Basic Definitions from Linear Systems Theory......Page 694
Index......Page 698
Back Cover......Page 710
