Linear Algebra and Matrix Analysis for Statistics

Content: Matrices, Vectors, and Their Operations Basic definitions and notations Matrix addition and scalar-matrix multiplication Matrix multiplication Partitioned matrices The "trace" of a square matrix Some special matrices Systems of Linear Equations Introduction Gaussian elimination Gauss-Jordan elimination Elementary matrices Homogeneous linear systems The inverse of a matrix More on Linear Equations The LU decomposition Crout's Algorithm LU decomposition with row interchanges The LDU and Cholesky factorizations Inverse of partitioned matrices The LDU decomposition for partitioned matrices The Sherman-Woodbury-Morrison formula Euclidean Spaces Introduction Vector addition and scalar multiplication Linear spaces and subspaces Intersection and sum of subspaces Linear combinations and spans Four fundamental subspaces Linear independence Basis and dimension The Rank of a Matrix Rank and nullity of a matrix Bases for the four fundamental subspaces Rank and inverse Rank factorization The rank-normal form Rank of a partitioned matrix Bases for the fundamental subspaces using the rank normal form Complementary Subspaces Sum of subspaces The dimension of the sum of subspaces Direct sums and complements Projectors Orthogonality, Orthogonal Subspaces, and Projections Inner product, norms, and orthogonality Row rank = column rank: A proof using orthogonality Orthogonal projections Gram-Schmidt orthogonalization Orthocomplementary subspaces The fundamental theorem of linear algebra More on Orthogonality Orthogonal matrices The QR decomposition Orthogonal projection and projector Orthogonal projector: Alternative derivations Sum of orthogonal projectors Orthogonal triangularization Revisiting Linear Equations Introduction Null spaces and the general solution of linear systems Rank and linear systems Generalized inverse of a matrix Generalized inverses and linear systems The Moore-Penrose inverse Determinants Definitions Some basic properties of determinants Determinant of products Computing determinants The determinant of the transpose of a matrix - revisited Determinants of partitioned matrices Cofactors and expansion theorems The minor and the rank of a matrix The Cauchy-Binet formula The Laplace expansion Eigenvalues and Eigenvectors Characteristic polynomial and its roots Spectral decomposition of real symmetric matrices Spectral decomposition of Hermitian and normal matrices Further results on eigenvalues Singular value decomposition Singular Value and Jordan Decompositions Singular value decomposition (SVD) The SVD and the four fundamental subspaces SVD and linear systems SVD, data compression and principal components Computing the SVD The Jordan canonical form Implications of the Jordan canonical form Quadratic Forms Introduction Quadratic forms Matrices in quadratic forms Positive and nonnegative definite matrices Congruence and Sylvester's law of inertia Nonnegative definite matrices and minors Extrema of quadratic forms Simultaneous diagonalization The Kronecker Product and Related Operations Bilinear interpolation and the Kronecker product Basic properties of Kronecker products Inverses, rank and nonsingularity of Kronecker products Matrix factorizations for Kronecker products Eigenvalues and determinant The vec and commutator operators Linear systems involving Kronecker products Sylvester's equation and the Kronecker sum The Hadamard product Linear Iterative Systems, Norms, and Convergence Linear iterative systems and convergence of matrix powers Vector norms Spectral radius and matrix convergence Matrix norms and the Gerschgorin circles SVD - revisited Web page ranking and Markov chains Iterative algorithms for solving linear equations Abstract Linear Algebra General vector spaces General inner products Linear transformations, adjoint and rank The four fundamental subspaces - revisited Inverses of linear transformations Linear transformations and matrices Change of bases, equivalence and similar matrices Hilbert spaces References Exercises appear at the end of each chapter