If you want to work in any computational or technical field, you need to understand linear algebra. As the study of matrices and operations acting upon them, linear algebra is the mathematical basis of nearly all algorithms and analyses implemented in computers. But the way it's presented in decades-old textbooks is much different from how professionals use linear algebra today to solve real-world modern applications.
This practical guide from Mike X Cohen teaches the core concepts of linear algebra as implemented in Python, including how they're used in data science, machine learning, deep learning, computational simulations, and biomedical data processing applications. Armed with knowledge from this book, you'll be able to understand, implement, and adapt myriad modern analysis methods and algorithms.
Ideal for practitioners and students using computer technology and algorithms, this book introduces you to:
• The interpretations and applications of vectors and matrices
• Matrix arithmetic (various multiplications and transformations)
• Independence, rank, and inverses
• Important decompositions used in applied linear algebra (including LU and QR)
• Eigendecomposition and singular value decomposition
• Applications including least-squares model fitting and principal components analysis
Author(s): Mike Cohen
Edition: 1
Publisher: O'Reilly Media
Year: 2022
Language: English
Commentary: Publisher's PDF
Pages: 328
City: Sebastopol, CA
Tags: Linear Algebra; Data Science; Python; Vectors; Matrices; Matrix Arithmetic; Orthogonal Matrices; QR Decomposition; Row Reduction; LU Decomposition; General Linear Models; Least Squares; Eigendecomposition; Singular Value Decomposition; Regression; NumPy
Cover
Copyright
Table of Contents
Preface
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
Chapter 1. Introduction
What Is Linear Algebra and Why Learn It?
About This Book
Prerequisites
Math
Attitude
Coding
Mathematical Proofs Versus Intuition from Coding
Code, Printed in the Book and Downloadable Online
Code Exercises
How to Use This Book (for Teachers and Self Learners)
Chapter 2. Vectors, Part 1
Creating and Visualizing Vectors in NumPy
Geometry of Vectors
Operations on Vectors
Adding Two Vectors
Geometry of Vector Addition and Subtraction
Vector-Scalar Multiplication
Scalar-Vector Addition
Transpose
Vector Broadcasting in Python
Vector Magnitude and Unit Vectors
The Vector Dot Product
The Dot Product Is Distributive
Geometry of the Dot Product
Other Vector Multiplications
Hadamard Multiplication
Outer Product
Cross and Triple Products
Orthogonal Vector Decomposition
Summary
Code Exercises
Chapter 3. Vectors, Part 2
Vector Sets
Linear Weighted Combination
Linear Independence
The Math of Linear Independence
Independence and the Zeros Vector
Subspace and Span
Basis
Definition of Basis
Summary
Code Exercises
Chapter 4. Vector Applications
Correlation and Cosine Similarity
Time Series Filtering and Feature Detection
k-Means Clustering
Code Exercises
Correlation Exercises
Filtering and Feature Detection Exercises
k-Means Exercises
Chapter 5. Matrices, Part 1
Creating and Visualizing Matrices in NumPy
Visualizing, Indexing, and Slicing Matrices
Special Matrices
Matrix Math: Addition, Scalar Multiplication, Hadamard Multiplication
Addition and Subtraction
“Shifting” a Matrix
Scalar and Hadamard Multiplications
Standard Matrix Multiplication
Rules for Matrix Multiplication Validity
Matrix Multiplication
Matrix-Vector Multiplication
Matrix Operations: Transpose
Dot and Outer Product Notation
Matrix Operations: LIVE EVIL (Order of Operations)
Symmetric Matrices
Creating Symmetric Matrices from Nonsymmetric Matrices
Summary
Code Exercises
Chapter 6. Matrices, Part 2
Matrix Norms
Matrix Trace and Frobenius Norm
Matrix Spaces (Column, Row, Nulls)
Column Space
Row Space
Null Spaces
Rank
Ranks of Special Matrices
Rank of Added and Multiplied Matrices
Rank of Shifted Matrices
Theory and Practice
Rank Applications
In the Column Space?
Linear Independence of a Vector Set
Determinant
Computing the Determinant
Determinant with Linear Dependencies
The Characteristic Polynomial
Summary
Code Exercises
Chapter 7. Matrix Applications
Multivariate Data Covariance Matrices
Geometric Transformations via Matrix-Vector Multiplication
Image Feature Detection
Summary
Code Exercises
Covariance and Correlation Matrices Exercises
Geometric Transformations Exercises
Image Feature Detection Exercises
Chapter 8. Matrix Inverse
The Matrix Inverse
Types of Inverses and Conditions for Invertibility
Computing the Inverse
Inverse of a 2 × 2 Matrix
Inverse of a Diagonal Matrix
Inverting Any Square Full-Rank Matrix
One-Sided Inverses
The Inverse Is Unique
Moore-Penrose Pseudoinverse
Numerical Stability of the Inverse
Geometric Interpretation of the Inverse
Summary
Code Exercises
Chapter 9. Orthogonal Matrices and QR Decomposition
Orthogonal Matrices
Gram-Schmidt
QR Decomposition
Sizes of Q and R
QR and Inverses
Summary
Code Exercises
Chapter 10. Row Reduction and LU Decomposition
Systems of Equations
Converting Equations into Matrices
Working with Matrix Equations
Row Reduction
Gaussian Elimination
Gauss-Jordan Elimination
Matrix Inverse via Gauss-Jordan Elimination
LU Decomposition
Row Swaps via Permutation Matrices
Summary
Code Exercises
Chapter 11. General Linear Models and Least Squares
General Linear Models
Terminology
Setting Up a General Linear Model
Solving GLMs
Is the Solution Exact?
A Geometric Perspective on Least Squares
Why Does Least Squares Work?
GLM in a Simple Example
Least Squares via QR
Summary
Code Exercises
Chapter 12. Least Squares Applications
Predicting Bike Rentals Based on Weather
Regression Table Using statsmodels
Multicollinearity
Regularization
Polynomial Regression
Grid Search to Find Model Parameters
Summary
Code Exercises
Bike Rental Exercises
Multicollinearity Exercise
Regularization Exercise
Polynomial Regression Exercise
Grid Search Exercises
Chapter 13. Eigendecomposition
Interpretations of Eigenvalues and Eigenvectors
Geometry
Statistics (Principal Components Analysis)
Noise Reduction
Dimension Reduction (Data Compression)
Finding Eigenvalues
Finding Eigenvectors
Sign and Scale Indeterminacy of Eigenvectors
Diagonalizing a Square Matrix
The Special Awesomeness of Symmetric Matrices
Orthogonal Eigenvectors
Real-Valued Eigenvalues
Eigendecomposition of Singular Matrices
Quadratic Form, Definiteness, and Eigenvalues
The Quadratic Form of a Matrix
Definiteness
? T ? Is Positive (Semi)definite
Generalized Eigendecomposition
Summary
Code Exercises
Chapter 14. Singular Value Decomposition
The Big Picture of the SVD
Singular Values and Matrix Rank
SVD in Python
SVD and Rank-1 “Layers” of a Matrix
SVD from EIG
SVD of ? T ?
Converting Singular Values to Variance, Explained
Condition Number
SVD and the MP Pseudoinverse
Summary
Code Exercises
Chapter 15. Eigendecomposition and SVD Applications
PCA Using Eigendecomposition and SVD
The Math of PCA
The Steps to Perform a PCA
PCA via SVD
Linear Discriminant Analysis
Low-Rank Approximations via SVD
SVD for Denoising
Summary
Exercises
PCA
Linear Discriminant Analyses
SVD for Low-Rank Approximations
SVD for Image Denoising
Chapter 16. Python Tutorial
Why Python, and What Are the Alternatives?
IDEs (Interactive Development Environments)
Using Python Locally and Online
Working with Code Files in Google Colab
Variables
Data Types
Indexing
Functions
Methods as Functions
Writing Your Own Functions
Libraries
NumPy
Indexing and Slicing in NumPy
Visualization
Translating Formulas to Code
Print Formatting and F-Strings
Control Flow
Comparators
If Statements
For Loops
Nested Control Statements
Measuring Computation Time
Getting Help and Learning More
What to Do When Things Go Awry
Summary
Index
About the Author
Colophon
