Functional Data Analysis

Cover......Page 1
Springer Series in Statistics......Page 2
Functional Data Analysis (Second edition)......Page 4
Copyright......Page 5
Preface to the Second Edition......Page 6
Contents......Page 10
1.1 What are functional data?......Page 22
1.3 Some functional data analyses......Page 26
1.4 The goals of functional data analysis......Page 30
1.5.1 Data representation: smoothing and interpolation......Page 32
1.5.2 Data registration or feature alignment......Page 33
1.5.4 Plotting pairs of derivatives......Page 34
1.6.2 Functional principal components analysis......Page 36
1.7 Functional linear models......Page 37
1.8 Using derivatives in functional data analysis......Page 38
1.9 Concluding remarks......Page 39
2.1 Introduction......Page 40
2.2.2 Derivatives and integrals......Page 41
2.2.4 Functions of functions......Page 42
2.3.2 Covariance and correlation functions......Page 43
2.3.3 Cross-covariance and cross-correlation functions......Page 45
2.4.1 Functional features......Page 47
2.4.2 Data resolution and functional dimensionality......Page 48
2.4.3 The size of a function......Page 49
2.5.1 The log nondurable goods index......Page 50
2.5.2 Phase–plane plots show energy transfer......Page 51
2.5.3 The nondurable goods cycles......Page 54
2.6 Further reading and notes......Page 55
3.1 Introduction......Page 58
3.2.1 What makes discrete data functional?......Page 59
3.2.3 The interplay between smooth and noisy variation......Page 60
3.2.4 The standard model for error and its limitations......Page 61
3.2.6 Data resolution and derivative estimation......Page 62
3.3 Representing functions by basis functions......Page 64
3.4 The Fourier basis systemfor periodic data......Page 66
3.5 The spline basis systemfor open-ended data......Page 67
3.5.1 Spline functions and degrees of freedom......Page 68
3.5.2 The B-spline basis for spline functions......Page 70
3.6.1 Wavelets......Page 74
3.6.3 Polynomial bases......Page 75
3.6.6 The constant basis......Page 76
3.7 Choosing a scale for t......Page 77
3.8 Further reading and notes......Page 78
4.2 Fitting data using a basis systemby least squares......Page 80
4.2.1 Ordinary or unweighted least squares fits......Page 81
4.2.2 Weighted least squares fits......Page 82
4.3 A performance assessment of least squares smoothing......Page 83
4.4 Least squares fits as linear transformations of the data......Page 84
4.4.1 How linear smoothers work......Page 85
4.4.2 The degrees of freedomof a linear smooth......Page 87
4.5.1 The bias/variance trade-off......Page 88
4.5.2 Algorithms for choosing K......Page 90
4.6.1 Sampling variance estimates......Page 91
4.6.2 Estimating Σ_e......Page 92
4.6.3 Confidence limits......Page 93
4.7 Fitting data by localized least squares......Page 94
4.7.1 Kernel smoothing......Page 95
4.7.2 Localized basis function estimators......Page 97
4.7.3 Local polynomial smoothing......Page 98
4.7.5 Summary of localized basismethods......Page 99
4.8 Further reading and notes......Page 100
5.1 Introduction......Page 102
5.2 Spline smoothing......Page 103
5.2.1 Two competing objectives in function estimation......Page 104
5.2.3 The penalized sum of squared errors fitting criterion......Page 105
5.2.4 The structure of a smoothing spline......Page 106
5.2.5 How spline smooths are computed......Page 107
5.2.6 Spline smoothing as a linear operation......Page 108
5.2.7 Spline smoothing as an augmented least squares problem......Page 110
5.2.8 Estimating derivatives by spline smoothing......Page 111
5.3.1 Roughness penalties with fewer basis functions......Page 112
5.3.3 More general roughness penalties......Page 113
5.3.4 Computing the roughness penalty matrix......Page 114
5.4.1 Some limits imposed by computational issues......Page 115
5.4.2 The cross-validation or CV method......Page 117
5.4.3 The generalized cross-validation or GCV method......Page 118
5.4.4 Spline smoothing the simulated growth data......Page 120
5.5 Confidence intervals for function values and functional probes......Page 121
5.5.1 Linear functional probes......Page 122
5.5.2 Two linear mappings defining a probe value......Page 123
5.5.3 Computing confidence limits for function values......Page 124
5.6 A bi-resolution analysis with smoothing splines......Page 125
5.6.1 Complementary bases......Page 126
5.6.2 Specifying the roughness penalty......Page 127
5.6.3 Some properties of the estimates......Page 128
5.6.4 Relationship to the roughness penalty approach......Page 129
5.7 Further reading and notes......Page 130
6.2 Fitting positive functions......Page 132
6.2.1 A positive smoothing spline......Page 134
6.2.2 Representing a positive function by a differential equation......Page 135
6.3.2 Expressing a strictly monotone function explicitly......Page 136
6.3.3 Expressing a strictly monotone function as a differential equation......Page 137
6.4 The performance of spline smoothing revisited......Page 138
6.5 Fitting probability functions......Page 139
6.6 Estimating probability density functions......Page 140
6.7 Functional data analysis of point processes......Page 142
6.8 Fitting a linear model with estimation of the density of residuals......Page 144
6.9 Further notes and readings......Page 147
7.1 Introduction......Page 148
7.2 Shift registration......Page 150
7.2.1 The least squares criterion for shift alignment......Page 152
7.3 Feature or landmark registration......Page 153
7.5 A more general warping function h......Page 158
7.6 A continuous fitting criterion for registration......Page 159
7.7 Registering the height acceleration curves......Page 161
7.9.1 Shift registration by the Newton-Raphson algorithm......Page 163
7.10 Further reading and notes......Page 165
8.1 Introduction......Page 168
8.2.1 PCA for multivariate data......Page 169
8.2.2 Defining PCA for functional data......Page 170
8.2.3 Defining an optimal empirical orthonormal basis......Page 172
8.2.4 PCA and eigenanalysis......Page 173
8.3.1 Plotting components as perturbations of the mean......Page 175
8.3.3 Rotating principal components......Page 177
8.4 Computational methods for functional PCA......Page 181
8.4.2 Basis function expansion of the functions......Page 182
8.4.3 More general numerical quadrature......Page 185
8.5 Bivariate and multivariate PCA......Page 187
8.5.1 Defining multivariate functional PCA......Page 188
8.5.2 Visualizing the results......Page 189
8.5.3 Inner product notation: Concluding remarks......Page 191
8.6 Further readings and notes......Page 192
9.1 Introduction......Page 194
9.2 The results of smoothing the PCA......Page 196
9.3.2 Estimating subsequent principal components......Page 198
9.3.3 Choosing the smoothing parameter by CV......Page 199
9.4.1 The periodic case......Page 200
9.4.2 The nonperiodic case......Page 202
9.5.1 Smoothing the data rather than the PCA......Page 203
9.5.2 A stepwise roughness penalty procedure......Page 205
9.5.3 A further approach......Page 206
10.1 Introduction......Page 208
10.2 General approaches tomixed data......Page 210
10.3.1 Combining function and vector spaces......Page 211
10.3.2 Finding the principal components in practice......Page 212
10.3.4 Balance between functional and vector variation......Page 213
10.4.2 Balancing temperature and time shift effects......Page 215
10.5.1 Taking account of effects beyond phase shift......Page 216
10.5.2 Separating out the vector component......Page 219
11.1.1 The basic problem......Page 222
11.3.1 Notation and assumptions......Page 225
11.3.2 The naive approach does not give meaningful results......Page 226
11.3.3 Choice of the smoothing parameter......Page 227
11.3.4 The values of the correlations......Page 228
11.4 Application to the study of lupus nephritis......Page 229
11.5 Why is regularization necessary?......Page 230
11.6.1 Discretization and basis approaches......Page 231
11.6.2 The roughness of the canonical variates......Page 232
11.7.1 The optimal scoring problem......Page 234
11.7.3 The relationship with CCA......Page 235
11.8 Further readings and notes......Page 236
12.1 Introduction......Page 238
12.2 A functional response and a categorical independent variable......Page 239
12.3 A scalar response and a functional independent variable......Page 240
12.4.3 Short-termfeed-forward......Page 241
12.5 What about predicting derivatives?......Page 242
12.6 Overview......Page 243
13.2 Predicting temperature curves fromclimate zones......Page 244
13.2.2 Assessing the fit......Page 246
13.3.1 Structure of the data......Page 250
13.3.2 A functional linear model for the horse data......Page 252
13.3.3 Effects and contrasts......Page 254
13.4.1 The general model......Page 256
13.4.3 Functional linear modelling with regularized basis expansions......Page 257
13.4.4 Using the Kronecker product to express \hat{B}......Page 259
13.5.1 How to compute confidence intervals......Page 260
13.5.2 Confidence intervals for climate zone effects......Page 262
13.5.3 Some cautions on interpreting confidence intervals......Page 264
13.6 Further reading and notes......Page 265
14.1 Introduction......Page 268
14.2.2 Preliminary steps......Page 269
14.2.3 Fitting the model and assessing fit......Page 271
14.3 Long-term and seasonal trends in the nondurable goods index......Page 272
14.4 Computational issues......Page 276
14.5 Confidence intervals......Page 278
14.6 Further reading and notes......Page 279
15.1 Introduction......Page 282
15.2 A naive approach: Discretizing the covariate function......Page 283
15.3 Regularization using restricted basis functions......Page 285
15.4 Regularization with roughness penalties......Page 287
15.5 Computational issues......Page 289
15.5.1 Computing the regularized solution......Page 290
15.6 Cross-validation and regression diagnostics......Page 291
15.7 The direct penalty method for computing β......Page 292
15.7.2 The two-stage minimization process......Page 293
15.7.3 Functional interpolation revisited......Page 294
15.8 Functional regression and integral equations......Page 296
15.9 Further reading and notes......Page 297
16.1 Introduction: Predicting log precipitation from temperature......Page 300
16.1.1 Fitting themodel without regularization......Page 301
16.2.1 Restricting the basis η(s)......Page 303
16.2.2 Restricting the basis θ(t)......Page 304
16.2.3 Restricting both bases......Page 305
16.3 Assessing goodness of fit......Page 306
16.4 Computational details......Page 311
16.4.1 Fitting themodel without regularization......Page 312
16.4.2 Fitting themodel with regularization......Page 313
16.5 The general case......Page 314
16.6 Further reading and notes......Page 316
17.1 Introduction......Page 318
17.2 The oil refinery data......Page 319
17.3 Themelanoma data......Page 322
17.4 Some comparisons of the refinery and melanoma analyses......Page 326
18.1 Introduction......Page 328
18.2 Exploring a simple linear differential equation......Page 329
18.3.1 Nonconstant coefficients......Page 331
18.3.2 Higher order equations......Page 332
18.3.3 Systems of equations......Page 333
18.4.1 Differential operators to produce new functional observations......Page 334
18.4.2 The gross domestic product data......Page 335
18.4.3 Differential operators to regularize or smooth models......Page 337
18.4.4 Differential operators to partition variation......Page 338
18.5.1 Derivatives are rougher......Page 340
18.5.2 Finding a linear differential operator that annihilates known functions......Page 341
18.5.3 Finding the functions ξ_j satisfying Lξ_j =0......Page 343
18.6.1 Why additional constraints are needed to define a solution......Page 344
18.6.2 How L and B partition functions......Page 345
18.7 Further reading and notes......Page 346
19.1 Introduction......Page 348
19.2 Defining the problem......Page 349
19.3 A principal differential analysis of lip movement......Page 350
19.3.1 The biomechanics of lip movement......Page 351
19.3.2 Visualizing the PDA results......Page 353
19.4 PDA of the pinch force data......Page 355
19.5.1 PDA by point-wise minimization......Page 359
19.5.2 PDA using the concurrent functional linear model......Page 360
19.5.3 PDA by iterating the concurrent linear model......Page 361
19.6.1 PDA and PCA both minimize sums of squared errors......Page 364
19.6.2 PDA and PCA both involve finding linear operators......Page 365
19.6.3 Differences between differential operators (PDA) and projection operators (PCA)......Page 366
19.7 Further readings and notes......Page 369
20.1 Introduction......Page 370
20.2 The Green’s function for solving a linear differential equation......Page 371
20.2.1 The definition of the Green’s function......Page 372
20.2.3 A recipe for the Green’s function......Page 373
20.3 Reproducing kernels and Green’s functions......Page 374
20.3.1 What is a reproducing kernel?......Page 375
20.3.2 The reproducing kernel for ker B......Page 376
20.3.3 The reproducing kernel for ker L......Page 377
20.4 Further reading and notes......Page 378
21.1 Introduction......Page 380
21.1.1 The lip movement data......Page 381
21.1.2 The weather data......Page 382
21.2 The optimal basis for spline smoothing......Page 384
21.3.1 The need for a good algorithm......Page 385
21.3.2 Setting up the smoothing procedure......Page 387
21.3.4 The performance assessment phase......Page 388
21.4 A compact support basis for L-splines......Page 390
21.5.1 The gross domestic product data......Page 391
21.5.2 Themelanoma data......Page 392
21.5.3 The GDP data with seasonal effects......Page 394
21.5.4 Smoothing simulated human growth data......Page 395
22.1.1 Replication and regularity......Page 400
22.1.2 Some functional aspects elsewhere in statistics......Page 401
22.1.3 Functional analytic treatments......Page 402
22.2.1 Probability and inference......Page 403
22.2.3 Multidimensional arguments......Page 404
22.2.5 Back to the data!......Page 405
A.1 Inner products \langle x, y \rangle......Page 406
A.1.1 Some specific examples......Page 407
A.1.2 General properties......Page 408
A.1.3 Descriptive statistics in inner product notation......Page 410
A.1.4 Some extended uses of inner product notation......Page 411
A.2.1 Projections......Page 412
A.3.1 Singular value decompositions......Page 413
A.3.3 The QR decomposition......Page 414
A.4.1 Projection matrices......Page 415
A.4.3 Projections in more general inner product spaces......Page 416
A.5.2 The problemin amore general space......Page 417
A.5.3 Generalized eigenproblems......Page 418
A.6 Kronecker Products......Page 419
A.7.1 Linear models from a transformation perspective......Page 420
A.7.2 The least squares solution for B......Page 421
A.8.2 Hard-edged constraints......Page 422
A.8.3 Soft-edged constraints......Page 423
References......Page 426
Index......Page 440