Taken literally, the title "All of Statistics" is an exaggeration. But in spirit, the title is apt, as the book does cover a much broader range of topics than a typical introductory book on mathematical statistics. This book is for people who want to learn probability and statistics quickly. It is suitable for graduate or advanced undergraduate students in computer science, mathematics, statistics, and related disciplines. The book includes modern topics like nonparametric curve estimation, bootstrapping, and clas sification, topics that are usually relegated to follow-up courses. The reader is presumed to know calculus and a little linear algebra. No previous knowledge of probability and statistics is required. Statistics, data mining, and machine learning are all concerned with collecting and analyzing data. For some time, statistics research was con ducted in statistics departments while data mining and machine learning re search was conducted in computer science departments. Statisticians thought that computer scientists were reinventing the wheel. Computer scientists thought that statistical theory didn't apply to their problems. Things are changing. Statisticians now recognize that computer scientists are making novel contributions while computer scientists now recognize the generality of statistical theory and methodology. Clever data mining algo rithms are more scalable than statisticians ever thought possible. Formal sta tistical theory is more pervasive than computer scientists had realized.
Author(s): Larry Wasserman
Publisher: Springer
Year: 2010
Language: English
Pages: 442
Cover
Title
Copyright
Dedication
Preface
Contents
I Probability
1 Probability
1.1 Introduction
1.2 Sample Spaces and Events
1.3 Probability
1.4 Probability on Finite Sample Spaces
1.5 Independent Events
1.6 Conditional Probability
1.7 Bayes' Theorem
1.8 Bibliographic Remarks
1.9 Appendix
1.10 Exercises
2 Random Variables
2.1 Introduction
2.2 Distribution Functions and Probability Functions
2.3 Some Important Discrete Random Variables
2.4 Some Important Continuous Random Variables
2.5 Bivariate Distributions
2.6 Marginal Distributions
2.7 Independent Random Variables
2.8 Conditional Distributions
2.9 Multivariate Distributions and IID Samples
2.10 Two Important Multivariate Distributions
2.11 Transformations of Random Variables
2.12 Transformations of Several Random Variables
2.13 Appendix
2.14 Exercises
3 Expectation
3.1 Expectation of a Random Variable
3.2 Properties of Expectations
3.3 Variance and Covariance
3.4 Expectation and Variance of Important Random Variables
3.5 Conditional Expectation
3.6 Moment Generating Functions
3.7 Appendix
3.8 Exercises
4 Inequalities
4.1 Probability Inequalities
4.2 Inequalities For Expectations
4.3 Bibliographic Remarks
4.4 Appendix
4.5 Exercises
5 Convergence of Random Variables
5.1 Introduction
5.2 Types of Convergence
5.3 The Law of Large Numbers
5.4 The Central Limit Theorem
5.5 The Delta Method
5.6 Bibliographic Remarks
5.7 Appendix
5.7.1 Almost Sure and L1 Convergence
5.7.2 Proof of the Central Limit Theorem
5.8 Exercises
II Statistical Inference
6 Models, Statistical Inference and Learning
6.1 Introduction
6.2 Parametric and Nonparametric Models
6.3 Fundamental Concepts in Inference
6.3.1 Point Estimation
6.3.2 Confidence Sets
6.3.3 Hypothesis Testing
6.4 Bibliographic Remarks
6.5 Appendix
6.6 Exercises
7 Estimating the CDF and Statistical Functionals
7.1 The Empirical Distribution Function
7.2 Statistical Functionals
7.3 Bibliographic Remarks
7.4 Exercises
8 The Bootstrap
8.1 Simulation
8.2 Bootstrap Variance Estimation
8.3 Bootstrap Confidence Intervals
8.4 Bibliographic Remarks
8.5 Appendix
8.5.1 The Jackknife
8.5 .2 Justification For The Percentile Interval
8.6 Exercises
9 Parametric Inference
9.1 Parameter of Interest
9.2 The Method of Moments
9.3 Maximum Likelihood
9.4 Properties of Maximum Likelihood Estimators
9.5 Consistency of Maximum Likelihood Estimators
9.6 Equivariance of the MLE
9.7 Asymptotic Normality
9.8 Optimality
9.9 The Delta Method
9.10 Multiparameter Models
9.11 The Parametric Bootstrap
9.12 Checking Assumptions
9.13 Appendix
9.13.1 Proofs
9.13.2 Sufficiency
9.13.3 Exponential Families
9.13.4 Computing Maximum Likelihood Estimates
9.14 Exercises
10 Hypothesis Testing and p-values
10.1 The Wald Test
10.2 p-values
10.3 The X2 Distribution
10.4 Pearson's X2 Test For Multinomial Data
10.5 The Permutation Test
10.6 The Likelihood Ratio Test
10.7 Multiple Testing
10.8 Goodness-of-fit Tests
10.9 Bibliographic Remarks
10.10 Appendix
10.10.1 The Neyman-Pearson Lemma
10.10.2 The t-test
10.11 Exercises
11 Bayesian Inference
11.1 The Bayesian Philosophy
11.2 The Bayesian Method
11.3 Functions of Parameters
11.4 Simulation
11.5 Large Sample Properties of Bayes' Procedures
11.6 Flat Priors, Improper Priors, and "Noninformative" Priors
11.7 Multiparameter Problems
11.8 Bayesian Testing
11.9 Strengths and Weaknesses of Bayesian Inference
11.10 Bibliographic Remarks
11.11 Appendix
11.12 Exercises
12 Statistical Decision Theory
12.1 Preliminaries
12.2 Comparing Risk Functions
12.3 Bayes Estimators
12.4 Minimax Rules
12.5 Maximum Likelihood, Minimax, and Bayes
12.6 Admissibility
12.7 Stein's Paradox
12.8 Bibliographic Remarks
12.9 Exercises
III Statistical Models and Methods
13 Linear and Logistic Regression
13.1 Simple Linear Regression
13.2 Least Squares and Maximum Likelihood
13.3 Properties of the Least Squares Estimators
13.4 Prediction
13.5 Multiple Regression
13.6 Model Selection
13.7 Logistic Regression
13.8 Bibliographic Remarks
13.9 Appendix
13.10 Exercise
14 Multivariate Models
14.1 Random Vectors
14.2 Estimating the Correlation
14.3 Multivariate Normal
14.4 Multinomial
14.5 Bibliographic Remarks
14.6 Appendix
14.7 Exercises
15 Inference About Independence
15.1 Two Binary Variables
15.2 Two Discrete Variables
15.3 Two Continuous Variables
15.4 One Continuous Variable and One Discrete
15.5 Appendix
15.6 Exercises
16 Causal Inference
16.1 The Counterfactual Model
16.2 Beyond Binary Treatments
16.3 Observational Studies and Confounding
16.4 Simpson's Paradox
16.5 Bibliographic Remarks
16.6 Exercises
17 Directed Graphs and Conditional Independence
17.1 Introduction
17.2 Conditional Independence
17.3 DAGs
17.4 Probability and DAGs
17.5 More Independence Relations
17.6 Estimation for DAGs
17.7 Bibliographic Remarks
17.8 Appendix
17.9 Exercises
18 Undirected Graphs
18.1 Undirected Graphs
18.2 Probability and Graphs
18.3 Cliques and Potentials
18.4 Fitting Graphs to Data
18.5 Bibliographic Remarks
18.6 Exercises
19 Log-Linear Models
19.1 The Log-Linear Model
19.2 Graphical Log-Linear Models
19.3 Hierarchical Log-Linear Models
19.4 Model Generators
19.5 Fitting Log-Linear Models to Data
19.6 Bibliographic Remarks
19.7 Exercises
20 Nonparametric Curve Estimation
20.1 The Bias-Variance Tradeoff
20.2 Histograms
20.3 Kernel Density Estimation
20.4 Nonparametric Regression
20.5 Appendix
20.6 Bibliographic Remarks
20.7 Exercises
21 Smoothing Using Orthogonal Functions
21.1 Orthogonal Functions and L2 Spaces
21.2 Density Estimation
21.3 Regression
21.4 Wavelets
21.5 Appendix
21.6 Bibliographic Remarks
21.7 Exercises
22 Classification
22.1 Introduction
22.2 Error Rates and the Bayes Classifier
22.3 Gaussian and Linear Classifiers
22.4 Linear Regression and Logistic Regression
22.5 Relationship Between Logistic Regression and LDA
22.6 Density Estimation and Naive Bayes
22.7 Trees
22.8 Assessing Error Rates and Choosing a Good Classifier
22.9 Support Vector Machines
22.10 Kernelization
22.11 Other Classifiers
22.12 Bibliographic Remarks
22.13 Exercises
23 Probability Redux: Stochastic Processes
23.1 Introduction
23.2 Markov Chains
23.3 Poisson Processes
23.4 Bibliographic Remarks
23.5 Exercises
24 Simulation Methods
24.1 Bayesian Inference Revisited
24.2 Basic Monte Carlo Integration
24.3 Importance Sampling
24.4 MCMC Part I: The Metropolis- Hastings Algorithm
24.5 MCMC Part II: Different Flavors
24.6 Bibliographic Remarks
24.7 Exercises
Bibliography
Index
