Bayesian Nonparametrics

Half-title......Page 3
Series-title......Page 4
Title......Page 5
Copyright......Page 6
Contents......Page 7
Contributors......Page 10
What is it all about?......Page 11
Who needs it?......Page 14
The aims, purposes and contents of this book......Page 16
What does this book do?......Page 17
How do alternative models relate to each other?......Page 19
A brief history of Bayesian nonparametrics......Page 20
From the start to the present......Page 21
Applications......Page 22
Where does this book fit in the broader picture?......Page 23
Further topics......Page 24
Challenges and future developments......Page 26
References......Page 28
1.1 Introduction......Page 32
1.2 Bayesian choices......Page 34
1.3 Decision theory......Page 36
1.4 Asymptotics......Page 37
1.5 General posterior inference......Page 39
References......Page 43
2.1 Introduction......Page 45
2.2.1 Motivation......Page 46
Construction by normalization......Page 48
Conjugacy......Page 49
Limits of the posterior......Page 50
Discreteness......Page 51
Self-similarity......Page 52
Dirichlet samples and ties......Page 53
Sethuraman stick-breaking representation......Page 55
2.3.1 Mixtures of Dirichlet processes......Page 56
2.3.2 Dirichlet process mixtures......Page 57
2.3.3 Hierarchical Dirichlet processes......Page 58
2.4.1 Motivation and implications......Page 59
2.4.3 Instances of inconsistency......Page 60
2.4.4 Approaches to consistency......Page 61
2.4.5 Schwartz's theory......Page 62
Uniformly consistent tests......Page 63
Entropy and sieves......Page 64
Gaussian processes......Page 66
2.4.7 Semiparametric applications......Page 67
2.4.8 Non-i.i.d. observations......Page 68
Martingale method......Page 69
2.5.1 Motivation, description and consequences......Page 70
Prior concentration rate......Page 71
Sieves......Page 72
Finite-dimensional models......Page 73
Dirichlet mixtures......Page 74
2.5.4 Misspecified models......Page 75
2.5.5 Non-i.i.d. extensions......Page 76
2.6.1 Motivation and description......Page 77
2.6.2 Infinite-dimensional normal models......Page 78
2.6.3 General theory of Bayesian adaptation......Page 79
2.6.4 Density estimation using splines......Page 80
2.7.1 Parametric Bernshten-von Mises theorems......Page 81
2.7.2 Nonparametric Bernshten-von Mises theorems......Page 82
2.7.4 Nonexistence of Bernshten-von Mises theorems......Page 83
2.8 Concluding remarks......Page 84
References......Page 86
3.1 Introduction......Page 90
3.1.1 Exchangeability assumption......Page 91
3.1.2 A concise account of completely random measures......Page 93
3.2 Models for survival analysis......Page 96
3.2.1 Neutral-to-the-right priors......Page 97
3.2.2 Priors for cumulative hazards: the beta process......Page 102
3.2.3 Priors for hazard rates......Page 107
3.3 General classes of discrete nonparametric priors......Page 109
3.3.1 Normalized random measures with independent increments......Page 110
3.3.2 Exchangeable partition probability function......Page 114
3.3.3 Poisson-Kingman models and Gibbs-type priors......Page 116
3.3.4 Species sampling models......Page 121
3.4 Models for density estimation......Page 124
3.4.1 Mixture models......Page 125
3.4.2 Polya trees......Page 132
3.5 Random means......Page 136
3.6 Concluding remarks......Page 139
References......Page 140
4.1.1 Construction and interpretation......Page 147
4.1.2 Transitions and Markov processes......Page 148
4.1.3 Hazard regression models......Page 150
4.1.4 Semiparametric competing risks models......Page 152
4.2 Quantile inference......Page 154
4.3 Shape analysis......Page 158
4.4 Time series with nonparametric correlation function......Page 160
4.5 Concluding remarks......Page 162
4.5.2 Mixtures of beta processes......Page 163
4.5.4 From nonparametric Bayes to parametric survival models......Page 164
References......Page 165
5.1 Introduction......Page 168
5.2 Hierarchical Dirichlet processes......Page 170
5.2.1 Stick-breaking construction......Page 171
5.2.2 Chinese restaurant franchise......Page 172
5.2.3 Posterior structure of the HDP......Page 174
Information retrieval......Page 176
Multipopulation haplotype phasing......Page 178
Topic modeling......Page 179
5.3 Hidden Markov models with infinite state spaces......Page 181
Word segmentation......Page 185
Trees and grammars......Page 186
5.4.1 Pitman-Yor processes......Page 187
5.4.2 Hierarchical Pitman-Yor processes......Page 189
5.4.3 Applications of the hierarchical Pitman-Yor process......Page 190
Image segmentation......Page 191
5.5.1 The beta process and the Bernoulli process......Page 194
5.5.2 The Indian buffet process......Page 197
5.5.3 Stick-breaking constructions......Page 198
5.5.4 Hierarchical beta processes......Page 200
Sparse latent variable models......Page 201
Relational models......Page 202
5.6.1 Hierarchical DPs with random effects......Page 203
5.6.2 Analysis of densities and transformed DPs......Page 204
5.7 Inference for hierarchical Bayesian nonparametric models......Page 205
Chinese restaurant franchise sampler......Page 206
Posterior representation sampler......Page 208
5.7.2 Inference for HDP hidden Markov models......Page 209
5.7.3 Inference for beta processes......Page 210
5.7.4 Inference for hierarchical beta processes......Page 211
5.8 Discussion......Page 212
References......Page 213
6.1 Introduction......Page 218
6.2 Construction of finite-dimensional measures on observables......Page 219
6.3 Recent advances in computation for Dirichlet process mixture models......Page 221
References......Page 231
7.1 Introduction......Page 233
7.2.1 Illustration for simple repeated measurement models......Page 234
7.2.2 Posterior computation......Page 240
7.2.3 General random effects models......Page 243
7.2.4 Latent factor regression models......Page 245
7.3.1 Background......Page 246
7.3.2 Basis functions and clustering......Page 247
7.3.3 Functional Dirichlet process......Page 250
7.3.4 Kernel-based approaches......Page 251
7.3.5 Joint modeling......Page 253
7.4 Local borrowing of information and clustering......Page 255
7.5 Borrowing information across studies and centers......Page 258
7.6.1 Motivation......Page 260
7.6.2 Dependent Dirichlet processes......Page 261
7.6.3 Kernel-based approaches......Page 263
7.6.4 Conditional distribution modeling through DPMs......Page 266
7.6.5 Reproductive epidemiology application......Page 268
7.7 Bioinformatics......Page 270
7.7.1 Modeling of differential gene expression......Page 271
7.7.2 Analyzing polymorphisms and haplotypes......Page 273
7.7.3 New species discovery......Page 274
7.8 Nonparametric hypothesis testing......Page 275
7.9 Discussion......Page 277
References......Page 278
8.1 Introduction......Page 284
8.2 Random partitions......Page 285
8.3 Polya trees......Page 287
8.4 More DDP models......Page 289
8.4.1 The ANOVA DDP......Page 290
8.4.2 Classification with DDP models......Page 291
8.5 Other data formats......Page 293
8.6 An R package for nonparametric Bayesian inference......Page 296
8.7 Discussion......Page 299
References......Page 300
Author index......Page 302
Subject index......Page 307