Markov Chain Monte Carlo in Practice

Contents 5
Contributors 15
1 Introducing Markov chain Monte Carlo 18
1.1 Introduction 18
1.2 The problem 19
1.2.1 Bayesian inference 19
1.2.2 Calculating expectations 20
1.3 Markov chain Monte Carlo 21
1.3.1 Monte Carlo integration 21
1.3.2 Markov chains 22
1.3.3 The Metropolis-Hastings algorithm 22
1.4 Implementation 25
1.4.1 Canonical forms of proposal distribution 25
The Metropolis Algorithm 26
The independence sampler 26
Single-component Metropolis-Hastings 27
Gibbs sampling 29
1.4.2 Blocking 29
I.4.3 Updating order 29
1.4.4 Number of chains 30
1.4.5 Starting values 30
I.4.6 Determining burn-in 31
1.4.7 Determining stopping time 32
1.4.8 Output analysis 32
1.5 Discussion 33
References 34
2 Hepatitis B: a case study in MCMC methods 37
2.1 Introduction 37
2.2 Hepatitis B immunization 38
2.2.1 Background 38
2.2.2 Preliminary analysis 38
2.3 Modelling 41
2.3.1 Structural modelling 41
2.3.2 Probability modelling 43
2.3.3 Prior distributions 43
2.4 Fitting a model using Gibbs sampling 44
2.4.1 Initialization 44
2.4.2 Sampling from full conditional distributions 45
2.4-3 Monitoring the output 47
2.4-4 Inference from the output 50
2.4-5 Assessing goodness-of-fit 50
2.5 Model elaboration 52
2.5.1 Heavy-tailed distributions 52
2.5.2 Introducing a covariate 53
2.6 Conclusion 56
Acknowledgements 56
References 56
Appendix: BUGS 58
3 Markov chain concepts related to sampling algorithms 60
3.1 Introduction 60
3.2 Markov chains 60
3.3 Rates of convergence 63
3.4 Estimation 64
3.41 Batch means 65
3.4-2 Window estimators 65
3.5 The Gibbs sampler and Metropolis—Hastings algorithm 66
3.5.1 The Gibbs sampler 66
3.5.2 The Metropolis-Hastings algorithm 69
References 71
4 Introduction to general state-space Markov chain theory 73
4.1 Introduction 73
4.2 Notation and definitions 74
4.3 Irreducibility, recurrence and convergence 76
4.3.1 Irreducibility 76
4.3.2 Recurrence 77
4.3.3 Convergence 78
4.4 Harris recurrence 79
4.5 Mixing rates and central limit theorems 81
4.6 Regeneration 84
4.7 Discussion 85
References 87
5 Full conditional distributions 89
5.1 Introduction 89
5.2 Deriving full conditional distributions 89
5.2.2 Graphical models 91
Logistic regression model 92
Undirected graphical models 92
5.3 Sampling from full conditional distributions 92
5.3.1 Rejection sampling 93
5.3.2 Ratio-of-uniforms method 94
5.3.3 Adaptive rejection sampling 96
5.3.4 Metropolis-Hastings algorithm 98
5.3.5 Hybrid adaptive rejection and Metropolis-Hastings 99
5.4 Discussion 100
References 100
6 Strategies for improving MCMC 103
6.1 Introduction 103
6.2 Reparameterization 104
6.2.1 Correlations and transformations 104
6.2.2 Linear regression models 106
6.2.3 Random-effects models 107
Reparameterization by hierarchical centring 108
Reparameterization by sweeping 108
Unknown variance components 110
6.2.4 Nonlinear models 110
6.2.5 General comments on reparameterization 111
6.3 Random and adaptive direction sampling 112
6.3.1 The hit-and-run algorithm 112
6.3.2 Adaptive direction sampling (ADS) 113
6.4 Modifying the stationary distribution 115
6.4 1 Importance sampling 115
6.4.2 Metropolis-coupled MCMC 117
6.4.3 Simulated tempering 118
6.4.4 Auxiliary variables 119
6.5 Methods based on continuous-time processes 122
6.6 Discussion 124
Acknowledgement 125
References 125
7 Implementing MCMC 129
7.1 Introduction 129
7.2 Determining the number of iterations 130
7.3 Software and implementation 132
7.4 Output analysis 133
7.4.1 An example 134
7.5 Generic Metropolis algorithms 135
7.5.1 An example 138
7.6 Discussion 141
Acknowledgments 142
References 142
8 Inference and monitoring convergence 145
8.1 Difficulties in inference from Markov chain simulation 145
8.2 The risk of undiagnosed slow convergence 146
8.3 Multiple sequences and overdispersed starting points 149
8.4 Monitoring convergence using simulation output 150
8.5 Output analysis for inference 153
8.6 Output analysis for improving efficiency 154
References 155
9 Model determination using sampling-based methods 158
9.1 Introduction 158
9.2 Classical approaches 159
9.3 The Bayesian perspective and the Bayes factor 161
9.4 Alternative predictive distributions 162
9.4.1 Cross-validation predictive densities 163
9.4.2 Posterior predictive densities 164
9.4.3 Other predictive densities 164
9.5 How to use predictive distributions 164
9.6 Computational issues 167
9.6.1 Estimating predictive densities 167
Posterior predictive density estimation 167
Cross-validation predictive density estimation 167
Prior predictive density estimation 168
9.6.2 Computing expectations over predictive densities 168
9.6.3 Sampling from predictive densities 169
9.7 An example 170
9.8 Discussion 171
References 173
10 Hypothesis testing and model selection 175
10.1 Introduction 175
10.2 Uses of Bayes factors 177
10.3 Marginal likelihood estimation by importance sampling 179
10.4 Marginal likelihood estimation using maximum likelihood 182
10.4.1 The Laplace-Metropolis estimator 182
10.4.2 Candidate's estimator 184
10.4.3 The data-augmentation estimator 185
Latent data with conditional independence 185
Latent data without conditional independence 187
10.5 Application: how many components in a mixture? 188
10.5.1 Gibbs sampling for Gaussian mixtures 188
10.5.2 A simulated example 189
10.5.3 How many disks in the Galaxy? 192
10.6 Discussion 193
References 195
11 Model checking and model improvement 200
11.1 Introduction 200
11.2 Model checking using posterior predictive simulation 200
11.3 Model improvement via expansion 203
11.4 Example: hierarchical mixture modelling of reaction times 204
11.4.1 The data and the basic model 204
11.4.2 Model checking using posterior predictive simulation 207
11.4.3 Expanding the model 207
11.4.4 Checking the new model 209
References 211
12 Stochastic search variable selection 213
12.1 Introduction 213
12.2 A hierarchical Bayesian model for variable selection 214
12.3 Searching the posterior by Gibbs sampling 217
12.4 Extensions 219
12.4.1 SSVS for generalized linear models 219
12.4.2 SSVS across exchangeable regressions 220
12.5 Constructing stock portfolios with SSVS 221
12.6 Discussion 223
References 224
13 Bayesian model comparison via jump diffusions 225
13.1 Introduction 225
13.2 Model choice 226
13.2.1 Example 1: mixture deconvolution 226
13.2.2 Example 2: object recognition 228
13.2.3 Example 3: variable selection in regression 229
13.2.4 Example 4' change-point identification 230
13.3 Jump-diffusion sampling 231
13.3.1 The jump component 232
Gibbs jump dynamics 233
Metropolis jump dynamics 235
Choice of jump dynamics 236
13.3.2 Moving between jumps 236
13.4 Mixture deconvolution 236
13.4.1 Dataset 1: galaxy velocities 238
13.4.2 Dataset 2: length of porgies 238
13.5 Object recognition 243
13.5.1 Results 243
13.6 Variable selection 245
13.7 Change-point identification 245
13.7.1 Dataset 1: Nile discharge 246
13.7.2 Dataset 2: facial image 246
13.8 Conclusions 248
References 248
14 Estimation and optimization of functions 250
14.1 Non-Bayesian applications of MCMC 250
14.2 Monte Carlo optimization 250
14.3 Monte Carlo likelihood analysis 253
14.4 Normalizing-constant families 254
14.5 Missing data 258
14.6 Decision theory 260
14.7 Which sampling distribution? 260
14.8 Importance sampling 262
14.9 Discussion 264
References 265
15 Stochastic EM: method and application 268
15.1 Introduction 268
15.2 The EM algorithm 269
15.3 The stochastic EM algorithm 270
15.3.1 Stochastic imputation 270
15.3.2 Looking at the plausible region 271
15.3.3 Point estimation 272
15.3.4 Variance of the estimates 273
15.4 Examples 273
15.4.1 Type-I censored data 273
Point estimation 273
Standard errors 275
15.4.2 Empirical Bayes probit regression for cognitive diagnosis 277
Model and inference 278
Results 280
References 281
16 Generalized linear mixed models 283
16.1 Introduction 283
16.2 Generalized linear models (GLMs) 284
16.3 Bayesian estimation of GLMs 285
16.4 Gibbs sampling for GLMs 286
16.5 Generalized linear mixed models (GLMMs) 287
16.5.1 Frequentist GLMMs 287
16.5.2 Bayesian GLMMs 288
16.6 Specification of random-effect distributions 289
16.6.1 Prior precision 289
16.6.2 Prior means 291
16.6.3 Intrinsic aliasing and contrasts 291
16.6.4 Autocorrelated random effects 294
16.6.5 The first-difference prior 295
16.6.6 The second-difference prior 296
16.6.7 General Markov random field priors 297
16.6.8 Interactions 297
Interaction between an R-factor and a fixed covariate 298
Interaction between an R-factor and an F-factor 298
Interaction between two R-factors 299
16.7 Hyperpriors and the estimation of hyperparameters 299
16.8 Some examples 300
16.8.1 Longitudinal studies 301
16.8.2 Time trends for disease incidence and mortality 301
16.8.3 Disease maps and ecological analysis 302
16.8.4 Simultaneous variation in space and time 304
16.8.5 Frailty models in survival analysis 304
16.9 Discussion 306
References 307
17 Hierarchical longitudinal modelling 310
17.1 Introduction 310
17.2 Clinical background 312
17.3 Model detail and MCMC implementation 313
17 A Results 316
17.5 Summary and discussion 322
References 325
18 Medical monitoring 327
18.1 Introduction 327
18.2 Modelling medical monitoring 328
18.2.1 Nomenclature and data 328
18.2.2 Linear growth model 329
18.2.3 Marker growth as a stochastic process 330
18.3 Computing posterior distributions 333
18.3.1 Recursive updating 333
18.4 Forecasting 335
18.5 Model criticism 336
18.6 Illustrative application 336
18.6.1 The clinical problem 336
18.6.2 The model 338
18.6.3 Parameter estimates 338
18.6.4 Predicting deterioration 339
18.7 Discussion 341
References 342
19 MCMC for nonlinear hierarchical models 344
19.1 Introduction 344
19.2 Implementing MCMC 346
19.2.1 Method 1: Rejection Gibbs 347
19.2.2 Method 2: Ratio Gibbs 348
19.2.3 Method 3: Random-walk Metropolis 348
19.2.4 Method 4' Independence Metropolis-Hastings 349
19.2.5 Method 5: MLE/prior Metropolis-Hastings 349
19.3 Comparison of strategies 349
19.3.1 Guinea pigs data 350
19.4 A case study from pharmacokinetics-pharmacodynamics 353
19.5 Extensions and discussion 355
Second-stage dependence on covariates 357
Hybrid MCMC strategies 357
References 360
20 Bayesian mapping of disease 363
20.1 Introduction 363
20.2 Hypotheses and notation 364
20.3 Maximum likelihood estimation of relative risks 364
20.4 Hierarchical Bayesian model of relative risks 367
20.4.1 Bayesian inference for relative risks 367
20.4.2 Specification of the prior distribution 368
Unstructured heterogeneity of the relative risks 368
Spatially structured variation of the relative risks 369
20.4.3 Graphical representation of the model 370
20.5 Empirical Bayes estimation of relative risks 373
20.5.1 The conjugate gamma prior 373
20.5.2 Non-conjugate priors 374
20.5.3 Disadvantages of EB estimation 375
20.6 Fully Bayesian estimation of relative risks 375
20.6.1 Choices for hyperpriors 375
Uniform hyperpriors 375
Conjugate gamma hyperprior 376
20.6.2 Full conditional distributions for Gibbs sampling 376
Full conditionals for X_i 377
Full conditionals for 378
20.6.3 Example: gall-bladder and bile-duct cancer mortality 379
20.7 Discussion 380
References 380
21 MCMC in image analysis 384
21.1 Introduction 384
21.2 The relevance of MCMC to image analysis 385
21.3 Image models at different levels 386
21.3.1 Pixel-level models 386
21.3.2 Pixel-based modelling in SPECT 388
21.3.3 Template models 392
21.3.4 An example of template modelling 394
21.3.5 Stochastic geometry models 395
21.3.6 Hierarchical modelling 396
21.4 Methodological innovations in MCMC stimulated by imaging 397
21.5 Discussion 398
References 399
22 Measurement error 403
22.2 Conditional-independence modelling 405
22.2.1 Designs with individual-level surrogates 405
Conditional-independence assumptions 405
Conditional-independence graph 406
22.2.2 Designs using ancillary risk-factor information 408
Model conditionals 409
22.2.3 Estimation 409
22.3 Illustrative examples 410
22.3.1 Two measuring instruments with no validation group 410
Design set-up 411
Results 412
22.3.2 Influence of the exposure model 413
Simulation set-up 413
22.3.3 Ancillary risk-factor information and expert coding 414
Generating disease-study data 414
Generating the job-exposure survey data 415
Analysing the simulated data 415
22.4 Discussion 416
References 417
23 Gibbs sampling methods in genetics 420
23.1 Introduction 420
23.2 Standard methods in genetics 420
23.2.1 Genetic terminology 420
23.2.2 Genetic models 422
23.2.3 Genetic likelihoods 424
23.3 Gibbs sampling approaches 425
23.3.1 Gibbs sampling of genotypes 426
Segregation analysis 426
Linkage analysis 428
23.3.2 Gibbs sampling of parameters 430
23.3.3 Initialization, convergence, and fine tuning 432
23.4 MCMC maximum likelihood 434
23.5 Application to a family study of breast cancer 435
23.6 Conclusions 438
References 439
24 Mixtures of distributions: inference and estimation 442
24.1 Introduction 442
24.1.1 Modelling via mixtures 442
24.1.2 A first example: character recognition 443
24.1.3 Estimation methods 444
24.1.4 Bayesian estimation 446
24.2 The missing data structure 447
24.3 Gibbs sampling implementation 449
24-3.1 General algorithm 449
24-3.2 Extra-binomial variation 450
24-3.3 Normal mixtures: star clustering 450
24-8-4 Reparameterization issues 453
24-3.5 Extra-binomial variation: continued 456
24.4 Convergence of the algorithm 456
24.5 Testing for mixtures 457
24-5.1 Extra-binomial variation: continued 459
24.6 Infinite mixtures and other extensions 460
24-6.1 Dirichlet process priors and nonparametric models 460
24-6.2 Hidden Markov models 462
References 463
25 An archaeological example: radiocarbon dating 466
25.1 Introduction 466
25.2 Background to radiocarbon dating 467
25.3 Archaeological problems and questions 470
25.4 Illustrative examples 471
25.4-1 Example 1: dating settlements 471
25.4-2 Example 2: dating archaeological phases 474
25.4 5 Example 3: accommodating outliers 477
25.4-4 Practical considerations 478
25.5 Discussion 479
References 480
Index 482