This book is intended for graduate students in statistics and industrial mathematics, as well as researchers and practitioners in the field. We cover both theory and practice of nonparametric estimation. The text is novel in its use of maximum penalized likelihood estimation, and the theory of convex minimization problems (fully developed in the text) to obtain convergence rates. We also use (and develop from an elementary view point) discrete parameter submartingales and exponential inequalities. A substantial effort has been made to discuss computational details, and to include simulation studies and analyses of some classical data sets using fully automatic (data driven) procedures. Some theoretical topics that appear in textbook form for the first time are definitive treatments of I.J. Good's roughness penalization, monotone and unimodal density estimation, asymptotic optimality of generalized cross validation for spline smoothing and analogous methods for ill-posed least squares problems, and convergence proofs of EM algorithms for random sampling problems.
Uploaded by zhang.
Author(s): P. P. B. Eggermont ; V. N. LaRiccia
Publisher: springer
Year: 2001
Preface
Parametric density estimation
Nonparametric density estimation
Convexity and optimization
Why a new text?
Acknowledgments
Contents
Notations, Acronyms, and Conventions
1 Parametric and Nonparametric Estimation
1. Introduction
2. Indirect problems, EM algorithms, kernel density estimation, and roughness penalization
3. Consistency of nonparametric estimators
4. The usual nonparametric assumptions
5. Parametric vs nonparametric rates
6. Sieves and convexity
7. Additional notes and comments
Part I: Parametric Density Estimation
2 Parametric Maximum Likelihood Estimation
1. Introduction
2. Optimality of maximum likelihood estimators
3. Computing maximum likelihood estimators
4. The EM algorithm
5. Sensitivity to errors: M-estimators
6. Ridge regression
7. Right-skewed distributions with heavy tails
8. Additional comments
3. Parametric Maximum Likelihood Estimation in Action
1. Introduction
2. Best asymptotically normal estimators and small sample behavior
3. Mixtures of normals
4. Computing with the log-normal distribution
5. On choosing parametric families of distributions
6. Toward nonparametrics: mixtures revisited
Part II: Nonparametric Density Estimation
4. Kernel Density Estimation
1. Introduction
2. The expected L1 error in kernel density estimation
3. Integration by parts tricks
4. Submartingales, exponential inequalities, and almost sure bounds for the L1 error
5. Almost sure bounds for everything else
6. Nonparametric estimation of entropy
7. Optimal kernels
8. Asymptotic normality of the L1 error
9. Additional comments
5. Non parametric Maximum Penalized Likelihood Estimation
1. Introduction
2. Good's roughness penalization of root-densities
3. Roughness penalization of log-densities
4. Roughness penalization of bounded log-densities
5. Estimation under constraints
6. Additional notes and comments
6. Monotone and Unimodal Densities
1. Introduction
2. Monotone density estimation
3. Estimating smooth monotone densities
4. Algorithms and contractivity
5. Contractivity : the general case
6. Estimating smooth unimodal densities
7. Other unimodal density estimators
8. Afterthoughts: convex densities
9. Additional notes and comments
7. Choosing the Smoothing Parameter
1. Introduction
2. Least-squares cross-validation and plug-in methods
3. The double kernel method
4. Asymptotic plug-in methods
5. Away with pilot estimators!?
6. A discrepancy principle
7. The Good estimator
8. Additional notes and comments
8. Nonparametric Density Estimation in Action
1. Introduction
2. Finite-dimensional approximations
3. Smoothing parameter selection
4. Two data sets
5. Kernel selection
6. Unimodal density estimation
Part III: Convexity and Optimization
9. Convex Optimization in Finite-Dimensional Spaces
1. Convex sets and convex functions
2. Convex minimization problems
3. Lagrange multipliers
4. Strict and strong convexity
5. Compactness arguments
6. Additional notes and comments
10. Convex Optimization in Infinite-Dimensional Spaces
1. Convex functions
2. Convex integrals
3. Strong convexity
4. Compactness arguments
5. Euler equations
6. Finitely many constraints
7. Additional notes and comments
11. Convexity in Action
1. Introduction
2. Optimal kernels
3. Direct nonparametric maximum roughness penalized likelihood density estimation
4. Existence of roughness penalized log-densities
5. Existence of log-concave estimators
6. Constrained minimum distance estimation
Appendices
A1 Some Data Sets
1. Introduction
2. Old Faithfull geyser data
3. The Buffalo snow fall data
4. The rubber abbrasion data
5. Cloud seeding data
6. Texo oil field data
A2 The Fourier Transform
1. Introduction
2. Smooth functions
3. Integrable functions
4. Square integrable functions
5. Some examples
6. The Wiener theorem for L1(R)
A3 Banach Spaces, Dual Spaces, and Compactness
1. Banach spaces
2. Bounded linear operators
3. Compact operators
4. Dual spaces
5. Hilbert spaces
6. Compact Hermitian operators
7. Reproducing kernel Hilbert spaces
8. Closing Comments
References
Author Index
Subject Index
