This book organizes principles and methods of signal processing and machine learning into the framework of coherence. The book contains a wealth of classical and modern methods of inference, some reported here for the first time. General results are applied to problems in communications, cognitive radio, passive and active radar and sonar, multi-sensor array processing, spectrum analysis, hyperspectral imaging, subspace clustering, and related.
The reader will find new results for model fitting; for dimension reduction in models and ambient spaces; for detection, estimation, and space-time series analysis; for subspace averaging; and for uncertainty quantification. Throughout, the transformation invariances of statistics are clarified, geometries are illuminated, and null distributions are given where tractable. Stochastic representations are emphasized, as these are central to Monte Carlo simulations. The appendices contain a comprehensive account of matrix theory, the SVD, the multivariate normal distribution, and many of the important distributions for coherence statistics.
The book begins with a review of classical results in the physical and engineering sciences where coherence plays a fundamental role. Then least squares theory and the theory of minimum mean-squared error estimation are developed, with special attention paid to statistics that may be interpreted as coherence statistics. A chapter on classical hypothesis tests for covariance structure introduces the next three chapters on matched and adaptive subspace detectors. These detectors are derived from likelihood reasoning, but it is their geometries and invariances that qualify them as coherence statistics. A chapter on independence testing in space-time data sets leads to a definition of broadband coherence, and contains novel applications to cognitive radio and the analysis of cyclostationarity. The chapter on subspace averaging reviews basic results and derives an order-fitting rule for determining the dimension of an average subspace. These results are used to enumerate sources of acoustic and electromagnetic radiation and to cluster subspaces into similarity classes. The chapter on performance bounds and uncertainty quantification emphasizes the geometry of the Cramèr-Rao bound and its related information geometry.
Author(s): David Ramírez, Ignacio Santamaría, Louis Scharf
Publisher: Springer
Year: 2023
Language: English
Pages: 494
City: Cham
Preface
Contents
Acronyms
1 Introduction
1.1 The Coherer of Hertz, Branly, and Lodge
1.2 Interference, Coherence, and the Van Cittert-Zernike Story
1.3 Hanbury Brown-Twiss Effect
1.4 Tone Wobble and Coherence for Tuning
1.5 Beampatterns and Diffraction of Electromagnetic Radiation by a Slit
1.6 LIGO and the Detection of Einstein's Gravitational Waves
1.7 Coherence and the Heisenberg Uncertainty Relations
1.8 Coherence, Ambiguity, and the Moyal Identities
1.9 Coherence, Correlation, and Matched Filtering
1.10 Coherence and Matched Subspace Detectors
1.11 What Qualifies as a Coherence?
1.12 Why Complex?
1.13 What is the Role of Geometry?
1.14 Motivating Problems
1.15 A Preview of the Book
1.16 Chapter Notes
2 Least Squares and Related
2.1 The Linear Model
2.2 Over-Determined Least Squares and Related
2.2.1 Linear Prediction
2.2.2 Order Determination
2.2.3 Cross-Validation
2.2.4 Weighted Least Squares
2.2.5 Constrained Least Squares
2.2.6 Oblique Least Squares
2.2.7 The BLUE (or MVUB or MVDR) Estimator
2.2.8 Sequential Least Squares
2.2.9 Total Least Squares
2.2.10 Least Squares and Procrustes Problems for Channel Identification
2.2.11 Least Squares Modal Analysis
2.3 Under-determined Least Squares and Related
2.3.1 Minimum-Norm Solution
2.3.2 Sparse Solutions
2.3.3 Maximum Entropy Solution
2.3.4 Minimum Mean-Squared Error Solution
2.4 Multidimensional Scaling
2.5 The Johnson-Lindenstrauss Lemma
2.6 Chapter Notes
Appendices
2.A Completing the Square in Hermitian Quadratic Forms
2.A.1 Generalizing to Multiple Measurements and Other Cost Functions
2.A.2 LMMSE Estimation
3 Coherence, Classical Correlations, and their Invariances
3.1 Coherence Between a Random Variable and a Random Vector
3.2 Coherence Between Two Random Vectors
3.2.1 Relationship with Canonical Correlations
3.2.2 The Circulant Case
3.2.3 Relationship with Principal Angles
3.2.4 Distribution of Estimated Signal-to-Noise Ratio in Adaptive Matched Filtering
3.3 Coherence Between Two Time Series
3.4 Multi-Channel Coherence
3.5 Principal Component Analysis
3.6 Two-Channel Correlation
3.7 Multistage LMMSE Filter
3.8 Application to Beamforming and Spectrum Analysis
3.8.1 The Generalized Sidelobe Canceller
3.8.2 Composite Covariance Matrix
3.8.3 Distributions of the Conventional and Capon Beamformers
3.9 Canonical correlation analysis
3.9.1 Canonical Coordinates
3.9.2 Dimension Reduction Based on Canonical and Half-Canonical Coordinates
3.10 Partial Correlation
3.10.1 Regressing Two Random Vectors onto One
3.10.2 Regressing One Random Vector onto Two
3.11 Chapter Notes
4 Coherence and Classical Tests in the Multivariate Normal Model
4.1 How Limiting Is the Multivariate Normal Model?
4.2 Likelihood in the MVN Model
4.2.1 Sufficiency
4.2.2 Likelihood
4.3 Hypothesis Testing
4.4 Invariance in Hypothesis Testing
4.5 Testing for Sphericity of Random Variables
4.5.1 Sphericity Test: Its Invariances and Null Distribution
4.5.2 Extensions
4.6 Testing for sphericity of random vectors
4.7 Testing for Homogeneity of Covariance Matrices
4.8 Testing for Independence
4.8.1 Testing for Independence of Random Variables
4.8.2 Testing for Independence of Random Vectors
4.9 Cross-Validation of a Covariance Model
4.10 Chapter Notes
5 Matched Subspace Detectors
5.1 Signal and Noise Models
5.2 The Detection Problem and Its Invariances
5.3 Detectors in a First-Order Model for a Signal in a Known Subspace
5.3.1 Scale-Invariant Matched Subspace Detector
5.3.2 Matched Subspace Detector
5.4 Detectors in a Second-Order Model for a Signal in a Known Subspace
5.4.1 Scale-Invariant Matched Subspace Detector
5.4.2 Matched Subspace Detector
5.5 Detectors in a First-Order Model for a Signal in a Subspace Known Only by its Dimension
5.5.1 Scale-Invariant Matched Direction Detector
5.5.2 Matched Direction Detector
5.6 Detectors in a Second-Order Model for a Signal in a Subspace Known Only by its Dimension
5.6.1 Scale-Invariant Matched Direction Detector
5.6.2 Matched Direction Detector
5.7 Factor Analysis
5.8 A MIMO Version of the Reed-Yu Detector
5.9 Chapter Notes
Appendices
5.A Variations on Matched Subspace Detectors in a First-Order Model for a Signal in a Known Subspace
5.A.1 Scale-Invariant, Geometrically Averaged, Matched Subspace Detector
5.A.2 Refinement: Special Signal Sequences
5.A.3 Rapprochement
5.B Derivation of the Matched Subspace Detector in a Second-Order Model for a Signal in a Known Subspace
5.C Variations on Matched Direction Detectors in a Second-Order Model for a Signal in a Subspace Known Only by its Dimension
6 Adaptive Subspace Detectors
6.1 Introduction
6.2 Adaptive Detection Problems
6.2.1 Signal Models
6.2.2 Hypothesis Tests
6.3 Estimate and Plug (EP) Solutions for Adaptive Subspace Detection
6.3.1 Detectors in a First-Order Model for a Signal in a Known Subspace
6.3.2 Detectors in a Second-Order Model for a Signal in a Known Subspace
6.3.3 Detectors in a First-Order Model for a Signal in a Subspace Known Only by its Dimension
6.3.4 Detectors in a Second-Order Model for a Signal in a Subspace Known Only by its Dimension
6.4 GLR Solutions for Adaptive Subspace Detection
6.4.1 The Kelly and ACE Detector Statistics
6.4.2 Multidimensional and Multiple Measurement GLR Extensions of the Kelly and ACE Detector Statistics
6.5 Chapter Notes
7 Two-Channel Matched Subspace Detectors
7.1 Signal and Noise Models for Two-Channel Problems
7.1.1 Noise Models
7.1.2 Known or Unknown Subspaces
7.2 Detectors in a First-Order Model for a Signal in a Known Subspace
7.2.1 Scale-Invariant Matched Subspace Detector for Equal and Unknown Noise Variances
7.2.2 Matched Subspace Detector for Equal and Known Noise Variances
7.3 Detectors in a Second-Order Model for a Signal in a Known Subspace
7.3.1 Scale-Invariant Matched Subspace Detector for Equal and Unknown Noise Variances
7.3.2 Scale-Invariant Matched Subspace Detector for Unequal and Unknown Noise Variances
7.4 Detectors in a First-Order Model for a Signal in a Subspace Known Only by its Dimension
7.4.1 Scale-Invariant Matched Direction Detector for Equal and Unknown Noise Variances
7.4.2 Matched Direction Detector for Equal and Known Noise Variances
7.4.3 Scale-Invariant Matched Direction Detector in Noises of Different and Unknown Variances
7.4.4 Matched Direction Detector in Noises of Known but Different Variances
7.5 Detectors in a Second-Order Model for a Signal in a Subspace Known Only by its Dimension
7.5.1 Scale-Invariant Matched Direction Detector for Equal and Unknown Noise Variances
7.5.2 Matched Direction Detector for Equal and Known Noise Variances
7.5.3 Scale-Invariant Matched Direction Detector for Uncorrelated Noises Across Antennas (or White Noises with Different Variances)
7.5.4 Transformation-Invariant Matched Direction Detector for Noises with Arbitrary Spatial Correlation
7.6 Chapter Notes
8 Detection of Spatially Correlated Time Series
8.1 Introduction
8.2 Testing for Independence of Multiple Time Series
8.2.1 The Detection Problem and its Invariances
8.2.2 Test Statistic
8.3 Approximate GLR for Multiple WSS Time Series
8.3.1 Limiting Form of the Nonstationary GLR for WSS Time Series
8.3.2 GLR for Multiple Circulant Time Series and an Approximate GLR for Multiple WSS Time Series
8.4 Applications
8.4.1 Cognitive Radio
8.4.2 Testing for Impropriety in Time Series
8.5 Extensions
8.6 Detection of Cyclostationarity
8.6.1 Problem Formulation and Its Invariances
8.6.2 Test Statistics
8.6.3 Interpretation of the Detectors
8.7 Chapter Notes
9 Subspace Averaging
9.1 The Grassmann and Stiefel Manifolds
9.1.1 Statistics on the Grassmann and Stiefel Manifolds
9.2 Principal Angles, Coherence, and Distances Between Subspaces
9.3 Subspace Averages
9.3.1 The Riemannian Mean
9.3.2 The Extrinsic or Chordal Mean
9.4 Order Estimation
9.5 The Average Projection Matrix
9.6 Application to Subspace Clustering
9.7 Application to Array Processing
9.8 Chapter Notes
10 Performance Bounds and Uncertainty Quantification
10.1 Conceptual Framework
10.2 Fisher Information and the Cramér-Rao Bound
10.2.1 Properties of Fisher Score
10.2.2 The Cramér-Rao Bound
10.2.3 Geometry
10.3 MVN Model
10.4 Accounting for Bias
10.5 More General Quadratic Performance Bounds
10.5.1 Good Scores and Bad Scores
10.5.2 Properties and Interpretations
10.6 Information Geometry
10.7 Chapter Notes
11 Variations on Coherence
11.1 Coherence in Compressed Sensing
11.2 Multiset CCA
11.2.1 Review of Two-Channel CCA
11.2.2 Multiset CCA (MCCA)
11.3 Coherence in Kernel Methods
11.3.1 Kernel Functions, Reproducing Kernel Hilbert Spaces (RKHS), and Mercer's Theorem
11.3.2 Kernel CCA
11.3.3 Coherence Criterion in KLMS
11.4 Mutual Information as Coherence
11.5 Coherence in Time-Frequency Modeling of a Nonstationary Time Series
11.6 Chapter Notes
12 Epilogue
A Notation
Sets
Scalars, Vectors, Matrices, and Functions
Probability, Random Variables, and Distributions
B Basic Results in Matrix Algebra
B.1 Matrices and their Diagonalization
B.2 Hermitian Matrices and their Eigenvalues
B.2.1 Characterization of Eigenvalues of Hermitian Matrices
B.2.2 Hermitian Positive Definite Matrices
B.3 Traces
B.4 Inverses
B.4.1 Patterned Matrices and their Inverses
B.4.2 Matrix Inversion Lemma or Woodbury Identity
B.5 Determinants
B.5.1 Some Useful Determinantal Identities and Inequalities
B.6 Kronecker Products
B.7 Projection Matrices
B.7.1 Gramian, Pseudo-Inverse, and Projection
B.8 Toeplitz, Circulant, and Hankel Matrices
B.9 Important Matrix Optimization Problems
B.9.1 Trace Optimization
B.9.2 Determinant Optimization
B.9.3 Minimize Trace or Determinant of Error Covariance in Reduced-Rank Least Squares
B.9.4 Maximum Likelihood Estimation in a Factor Model
B.10 Matrix Derivatives
B.10.1 Differentiation with Respect to a Real Matrix
B.10.2 Differentiation with Respect to a Complex Matrix
C The SVD
C.1 The Singular Value Decomposition
C.2 Low-Rank Matrix Approximation
C.3 The CS Decomposition and the GSVD
C.3.1 CS Decomposition
C.3.2 The GSVD
D Normal Distribution Theory
D.1 Introduction
D.2 The Normal Random Variable
D.3 The Multivariate Normal Random Vector
D.3.1 Linear Transformation of a Normal Random Vector
D.3.2 The Bivariate Normal Random Vector
D.3.3 Analysis and Synthesis
D.4 The Multivariate Normal Random Matrix
D.4.1 Analysis and Synthesis
D.5 The Spherically Invariant Bivariate Normal Experiment
D.5.1 Coordinate Transformation: The Rayleigh and Uniform Distributions
D.5.2 Geometry and Spherical Invariance
D.5.3 Chi-Squared Distribution of uTu
Beta Distribution of ρ2 = uT P1u
D.5.5 F-Distribution of f = uT (I2−P1)u
D.5.6 Distributions for Other Derived Random Variables
D.5.7 Generation of Standard Normal Random Variables
D.6 The Spherically Invariant Multivariate Normal Experiment
D.6.1 Coordinate Transformation: The Generalized Rayleigh and Uniform Distributions
D.6.2 Geometry and Spherical Invariance
D.6.3 Chi-Squared Distribution of uTu
D.6.4 Beta Distribution of ρ2p
= uT Ppu
D.6.5 F-Distribution of fp = p
L−p
uT (IL−Pp)u
D.6.6 Distributions for Other Derived Random Variables
D.7 The Spherically Invariant Matrix-Valued Normal Experiment
D.7.1 Coordinate Transformation: Bartlett's Factorization
D.7.2 Geometry and Spherical Invariance
D.7.3 Wishart Distribution of UUT
D.7.4 The Matrix Beta Distribution
D.7.5 The Matrix F-Distribution
D.7.6 Special Cases
D.7.7 Summary
D.8 Spherical, Elliptical, and Compound Distributions
D.8.1 Spherical Distributions
D.8.2 Elliptical Distributions
D.8.3 Compound Distributions
E The complex normal distribution
E.1 The Complex MVN Distribution
E.2 The Proper Complex MVN Distribution
E.3 An Example from Signal Theory
E.4 Complex Distributions
F Quadratic Forms, Cochran's Theorem, and Related
F.1 Quadratic Forms and Cochran's Theorem
F.2 Decomposing a Measurement into Signal and Orthogonal Subspaces
F.3 Distribution of Squared Coherence
F.4 Cochran's Theorem in the Proper Complex Case
G The Wishart distribution, the Bartlett factorization, and related
G.1 Bartlett's Factorization
G.2 Real Wishart Distribution and Related
G.3 Complex Wishart Distribution and Related
G.4 Distribution of Sample Mean and Sample Covariance
H Null Distribution of Coherence Statistics
H.1 Null Distribution of the Tests for Independence
H.1.1 Testing Independence of Random Variables
H.1.2 Testing Independence of Random Vectors
H.2 Testing for Block-Diagonal Matrices of Different Block Sizes
H.3 Testing for Block-Sphericity
References
Alphabetical Index
