Handbook of Statistics 25: Bayesian Thinking: Modeling and Computation

Preface......Page 1
Table of contents......Page 3
Contributors......Page 13
Units, treatments, potential outcomes......Page 17
Replication and the Stable Unit Treatment Value Assumption - SUTVA......Page 18
Assignment mechanisms - unconfounded and strongly ignorable......Page 19
Confounded and ignorable assignment mechanisms......Page 20
Before 1923......Page 21
The observed outcome notation......Page 22
Models for the underlying data - Bayesian inference......Page 23
The posterior predictive distribution of Ymis under ignorable treatment assignment......Page 24
Simple normal example - analytic solution......Page 25
Simple normal example with covariate - numerical example......Page 26
Nonignorable treatment assignment......Page 27
Multiple treatments......Page 28
Principal stratification......Page 29
References......Page 30
Introduction and notation......Page 33
Intrinsic discrepancy and expected information......Page 38
One parameter models......Page 45
Main properties......Page 50
Approximate location parametrization......Page 52
Numerical reference priors......Page 54
Reference priors under regularity conditions......Page 55
Reference priors and the likelihood principle......Page 57
Restricted reference priors......Page 59
One nuisance parameter......Page 60
Many parameters......Page 69
Discrete parameters taking an infinity of values......Page 72
Behaviour under repeated sampling......Page 73
Prediction and hierarchical models......Page 75
Point estimation......Page 77
Region (interval) estimation......Page 80
Hypothesis testing......Page 83
Related work......Page 87
References......Page 89
Further reading......Page 98
Introduction......Page 107
Rationale......Page 109
Exact probability matching priors......Page 110
One-sided parametric intervals......Page 111
Two-sided parametric intervals......Page 112
Parametric matching priors in the multiparameter case......Page 113
Matching for an interest parameter......Page 114
Probability matching priors in group models......Page 116
Probability matching priors and reference priors......Page 117
Simultaneous and joint matching priors......Page 118
Matching priors via Bartlett corrections......Page 120
Matching priors for highest posterior density regions......Page 121
Nonregular cases......Page 122
One-sided predictive intervals......Page 123
Highest predictive density regions......Page 124
Probability matching priors for random effects......Page 125
Concluding remarks......Page 126
References......Page 127
Introduction......Page 131
Basics of Bayes factors and posterior model probabilities......Page 132
Motivation for the Bayesian approach to model selection......Page 133
Motivation for objective Bayesian model selection......Page 134
Difficulties in objective Bayesian model selection......Page 135
Objective Bayesian model selection methods......Page 137
Well calibrated priors approach......Page 139
Conventional prior approach......Page 140
Intrinsic Bayes factor (IBF) approach......Page 144
The intrinsic prior approach......Page 147
Improper EP-priors are ratio normalized.......Page 151
The empirical EP-prior approach.......Page 152
The fractional Bayes factor (FBF) approach......Page 153
Fractional Bayes factor as an average of training samples for exchangeable observations.......Page 154
Intrinsic priors of the fractional Bayes factor approach.......Page 155
Asymptotic methods and BIC......Page 156
Lower bounds on Bayes factors......Page 157
Example of randomized training sample.......Page 159
Conclusions......Page 161
References......Page 162
Introduction......Page 166
Conflict between P-values and lower bounds to Bayes factors and posterior probabilities: Case of a sharp null......Page 168
Calibration of P-values......Page 173
General observations......Page 174
Comparison of P-value with likelihood ratio and Bayes factor in Bahadur's asymptotics......Page 176
One-sided null hypothesis......Page 178
Bayesian P-values......Page 180
Concluding remarks......Page 183
References......Page 184
Introduction......Page 186
Sensitivity to choice of prior distribution and likelihood......Page 187
Bayesian residual analysis......Page 188
Prior predictive checks......Page 189
Repeated data generation and analysis......Page 190
Description of posterior predictive model checking......Page 191
Properties of posterior predictive p-values......Page 192
Definition of replications......Page 193
Discrepancy measures......Page 194
Application 1......Page 195
Application 2......Page 197
Direct data display......Page 200
Item fit......Page 201
Studying the association among the items......Page 203
Conclusions......Page 205
References......Page 206
Synopsis......Page 208
Preliminaries......Page 209
Bayesian elimination of nuisance parameters......Page 211
Objective Bayes analysis......Page 214
Integrated likelihood......Page 215
Reference prior approach......Page 217
Comparison with other approaches......Page 219
The Neyman and Scott class of problems......Page 222
Semiparametric problems......Page 228
Prediction and model averaging......Page 230
Significance tests......Page 231
References......Page 232
Introduction......Page 235
Bayes, admissible and minimax estimation......Page 236
Stein estimation and the James-Stein estimator......Page 239
Bayes estimation and the James-Stein estimator for the mean of the multivariate normal distribution with identity covariance matrix......Page 244
Generalizations for Bayes and the James-Stein estimation or the mean for the multivariate normal distribution with known covariance matrix Sigma......Page 249
The unknown covariance case.......Page 256
References......Page 257
Introduction to Bayesian nonparametrics......Page 259
Probability measures on spaces of probability measures......Page 261
The Dirichlet process......Page 262
Mixtures of Dirichlet processes......Page 263
Dirichlet process mixture models......Page 264
Fitting DPM models......Page 266
Extensions......Page 268
Polya tree and mixtures of Polya tree models......Page 269
The gamma process model......Page 271
Two sample problem......Page 272
Regression examples......Page 275
Regression for survival data......Page 276
Nonparametric regression with known error distribution......Page 280
Nonparametric regression with unknown error distribution......Page 286
Concluding remarks......Page 287
References......Page 288
Introduction......Page 293
Nonparametric Bayes......Page 294
Random distribution functions......Page 295
The Dirichlet process......Page 296
Mixtures of Dirichlet processes......Page 298
Neutral to the right processes......Page 301
Specifying prior distributions......Page 302
The posterior distributions......Page 303
Simulating the jump component......Page 304
Simulating an id distribution.......Page 305
B: Calculating lambdaepsilon......Page 306
The Beta process......Page 307
Some insights into the Beta process......Page 312
Extended-gamma process......Page 313
The likelihood function......Page 315
The computational model......Page 316
Polya trees......Page 317
Specifying the Polya tree......Page 318
Posterior distributions......Page 319
Beyond NTR processes and Polya trees......Page 321
References......Page 322
Introduction......Page 329
Discrete wavelet transforms and wavelet shrinkage......Page 330
Bayes and wavelets......Page 331
An illustrative example......Page 332
Regression problems......Page 334
Bayesian thresholding rules......Page 337
Bayesian wavelet methods in functional data analysis......Page 338
The density estimation problem......Page 341
An application in geoscience......Page 345
Other problems......Page 347
References......Page 349
Introduction......Page 353
Notation......Page 354
Outline......Page 355
The Dirichlet process......Page 356
Posterior distribution......Page 358
The MDP model......Page 360
Neutral to the right processes......Page 362
Posterior distribution......Page 364
Alternative representation......Page 365
Simulation......Page 366
Log-Gaussian prior......Page 367
Pólya trees......Page 368
Lévy driven processes......Page 370
Consistency......Page 373
Illustration......Page 377
Case 1......Page 378
Reinforcement and exchangeability......Page 379
Acknowledgement......Page 381
References......Page 382
Introduction......Page 386
Priors on infinite-dimensional spaces......Page 387
Dirichlet process......Page 388
Mixtures of Dirichlet processes......Page 390
Pinned-down Dirichlet......Page 391
Tail-free and neutral to the right process......Page 392
Polya tree process......Page 393
Priors obtained from random series representation......Page 394
Independent increment process......Page 395
Some other processes......Page 396
Consistency and rates of convergence......Page 397
Dirichlet process prior......Page 407
Right censored data......Page 408
Dirichlet mixture......Page 409
Mixture of normal kernels......Page 410
Mixtures on the half line......Page 412
Random histograms......Page 413
Polya tree prior......Page 414
Normal regression......Page 415
Binary regression......Page 416
Spectral density estimation......Page 417
Bernstein polynomial prior......Page 418
Estimation of transition density......Page 419
Concluding remarks......Page 421
References......Page 422
Motivation......Page 428
Bayesian recipe......Page 429
How can the Bayesian pie burn......Page 430
Monte Carlo integration and Markov chains......Page 431
The Metropolis-Hastings algorithm......Page 434
The Gibbs sampler......Page 436
Auxiliary variables in MCMC......Page 437
Estimating the variance of MCMC estimators......Page 439
Reversible jump MCMC......Page 440
Adaptive MCMC and particle filters......Page 441
Importance sampling and population Monte Carlo......Page 443
The perfect Bayesian pie: How to avoid ``burn-in'' issues......Page 444
Conclusions......Page 445
References......Page 446
Introduction......Page 450
Marginal posterior densities......Page 451
Conditional marginal density estimation......Page 452
Importance weighted marginal density estimation......Page 453
The Gibbs stopper approach......Page 455
Estimating posterior densities from the Metropolis-Hastings output......Page 456
Marginal posterior densities for generalized linear models......Page 460
Savage-Dickey density ratio......Page 462
Computing marginal likelihoods......Page 463
Computing posterior model probabilities via informative priors......Page 464
Simulation study.......Page 466
References......Page 469
Introduction......Page 471
Definition......Page 472
Missing data approach......Page 474
Nonparametric approach......Page 475
Reading......Page 477
The mixture conundrum......Page 478
Combinatorics......Page 479
The EM algorithm......Page 483
An inverse ill-posed problem......Page 484
Identifiability......Page 485
Choice of priors......Page 487
Loss functions......Page 489
Reordering......Page 492
Data augmentation and Gibbs sampling approximations......Page 493
Metropolis-Hastings approximations......Page 500
Population Monte Carlo approximations......Page 504
Inference for mixture models with unknown number of components......Page 508
Reversible jump algorithms......Page 509
Extensions to the mixture framework......Page 513
References......Page 515
Introduction......Page 520
Augmented probability simulation......Page 522
Sequential design......Page 524
Multiple comparisons......Page 525
Calibrating decision rules by frequentist operating characteristics......Page 526
Discussion......Page 527
References......Page 528
Introduction......Page 530
Introduction......Page 532
Prior for the regression coefficients......Page 533
Prior for the vector of binary indicator variables......Page 534
Prior for the partial correlation matrix C......Page 535
Selecting variables in groups......Page 536
Sampling scheme......Page 537
Cow milk protein data......Page 538
Hip replacement data......Page 540
Cow diet data......Page 544
Pig bodyweight data......Page 545
Simulation study......Page 552
Cow milk protein data......Page 558
Cow diet data......Page 559
Pig bodyweight data......Page 560
Summary......Page 561
References......Page 562
Dynamic linear models: General notation......Page 564
Inference in DLM......Page 566
Evolution and updating equations......Page 567
Variance law......Page 568
Monitoring and interventions......Page 569
Multiprocess models......Page 570
Dynamic nonlinear/nonnormal models......Page 571
A practical example......Page 573
Dynamic hierarchical models......Page 574
Markov Chain Monte Carlo......Page 575
Componentwise sampling schemes......Page 576
Block sampling schemes......Page 577
Exponential-family models......Page 581
Sequential Monte Carlo......Page 584
SIR- and SIS-based filters......Page 585
Auxiliary particle filter......Page 586
Parameter estimation and sequential Monte Carlo......Page 587
Computing predictive densities......Page 589
Recent developments......Page 590
Dynamic spatio-temporal models......Page 591
Multi-scale modeling......Page 593
Connections between Gaussian Markov random fields and DLMs......Page 594
References......Page 595
Why spatial statistics?......Page 600
Features of spatial data and building blocks for inference......Page 601
Small area estimation and parameter estimation in regional data......Page 603
Covariance functions and variograms......Page 610
Kriging: Classical spatial prediction......Page 613
Bayesian kriging......Page 616
Homogeneous Poisson processes......Page 619
Cox processes......Page 621
Inferential issues......Page 622
Recent developments and future directions......Page 628
References......Page 629
Introduction......Page 634
Different approaches......Page 636
Prior robustness......Page 637
Priors with given functional forms.......Page 638
epsilon-contamination class.......Page 639
Topological neighbourhoods.......Page 640
Other classes.......Page 641
Global robustness......Page 642
Local robustness......Page 644
Parametric classes.......Page 645
Model robustness approaches......Page 646
Partially known class.......Page 647
Loss robustness studies......Page 648
Joint robustness......Page 649
Foundations......Page 650
Bayes and nondominated alternatives......Page 652
Extracting additional information......Page 654
Stability theory......Page 656
General computational issues......Page 658
Lavine's algorithm......Page 659
Betrò and Guglielmi's algorithm......Page 660
Parametric class.......Page 661
Computing nondominated alternatives......Page 662
Convex loss functions.......Page 663
MCMC and robustness......Page 665
Robust estimators......Page 668
Hierarchical approaches......Page 669
Asymptotics and robustness......Page 670
Bayesian Gamma-minimax......Page 671
Conclusions......Page 672
References......Page 674
Introduction......Page 679
Elliptical distributions......Page 681
Measurement error models......Page 682
Diffuse prior distribution for the incidental parameters......Page 683
Dependent elliptical MEM......Page 685
Equal variances case......Page 688
Independent elliptical MEM......Page 690
Representable elliptical MEM......Page 691
A WNDE Student-t model......Page 692
A NDE Student-t model......Page 694
Application......Page 696
References......Page 697
Introduction......Page 699
The skew elliptical distribution......Page 702
Univariate case......Page 706
The L1-distance for posterior distribution of ( µ,sigma) under skew-normal model......Page 708
Bayes factor......Page 709
Bayes factor for representable skew elliptical linear model......Page 714
Simulation results......Page 715
Conclusions......Page 716
Proof of Proposition 3.7......Page 717
References......Page 720
Introduction......Page 722
Biological principles......Page 723
Image analysis, data extraction and normalization......Page 724
Statistical analysis of microarray data......Page 725
Bayesian models for gene selection......Page 726
Gene selection for binary classification......Page 727
Gene selection for multicategory classification......Page 730
Gene selection for survival methods......Page 735
Weibull regression model......Page 736
Proportional hazards model......Page 737
Differential gene expression analysis......Page 739
Censored models......Page 741
Nonparametric empirical Bayes approaches......Page 742
Nonparametric Bayesian approaches......Page 743
Finite mixture models......Page 744
Infinite mixture models......Page 745
Functional models......Page 746
Regression for grossly overparametrized models......Page 747
References......Page 748
Introduction......Page 752
Generalized linear mixed models......Page 754
Covariance structure modeling......Page 755
Flexible parametric and semiparametric methods......Page 756
Continuous right-censored time to event data......Page 757
Complications......Page 758
Multiple event time data......Page 760
Nonlinear modeling......Page 761
Constrained regression......Page 762
Model averaging......Page 764
Bioinformatics......Page 765
Discussion......Page 766
References......Page 767
Introduction......Page 771
Meta-analysis and multicentre studies......Page 773
Spatial analysis for environmental epidemiology......Page 776
Adjusting for mismeasured variables......Page 777
Adjusting for missing data......Page 781
Sensitivity analysis for unobserved confounding......Page 783
Ecological inference......Page 785
Bayesian model averaging......Page 787
Survival analysis......Page 790
Case-control analysis......Page 792
Bayesian applications in health economics......Page 794
Discussion......Page 795
References......Page 797
Introduction: The frequentist development......Page 801
Early Bayesian work on a single binary exposure......Page 804
Models with continuous and categorical exposure......Page 806
Analysis of matched case-control studies......Page 811
A continuous exposure: The equine epidemiology example......Page 815
A binary exposure: Endometrial cancer study......Page 816
Another binary exposure: Low birthweight study......Page 817
Example of a matched case-control study with multiple disease states......Page 819
Equivalence of retrospective and prospective analysis......Page 821
Conclusion......Page 823
References......Page 824
Introduction......Page 828
A Bayesian hierarchical model......Page 833
Connection to the bivariate-binormal model......Page 837
MCMC details......Page 838
An example......Page 839
References......Page 840
Elements of Markov chain Monte Carlo......Page 841
Computation of the marginal likelihood......Page 843
Binary responses......Page 846
Marginal likelihood of the binary probit......Page 849
Other link functions......Page 850
Marginal likelihood of the student-t binary model......Page 851
Ordinal response data......Page 852
Marginal likelihood of the student-t ordinal model......Page 853
Sequential ordinal model......Page 854
Multivariate responses......Page 856
Multivariate probit model......Page 857
Dependence structures......Page 858
Estimation of the MVP model......Page 859
Marginal likelihood of the MVP model......Page 862
Binary outcome with a confounded binary treatment......Page 863
Longitudinal binary responses......Page 864
Longitudinal multivariate responses......Page 868
References......Page 871
Advances in simulation-based Bayesian calculation......Page 874
Early Bayesian analyses of categorical data......Page 875
Bayesian smoothing of contingency tables......Page 877
Bayesian interaction analysis......Page 881
Bayesian tests of equiprobability and independence......Page 884
Bayes factors for GLM's with application to log-linear models......Page 886
Use of BIC in sociological applications......Page 889
Bayesian model search for loglinear models......Page 890
References......Page 893
Introduction......Page 895
Practical examples......Page 896
Semiparametric models based on intensity functions......Page 898
Frequentist methods for analyzing multiple event data......Page 901
Gamma processes......Page 903
Correlated prior processes......Page 904
Bayesian solution......Page 905
Analysis of the data-example......Page 906
Discussions and future research......Page 908
References......Page 909
Introduction......Page 911
Models and latent variables......Page 914
Reparameterization......Page 916
Priors for (alpha,xi).......Page 917
Noninformative priors......Page 918
Propriety of posterior under the noninformative priors......Page 919
Posterior distributions and Bayesian computation......Page 923
Bobwhite example......Page 925
Simulation studies......Page 926
References......Page 931
Introduction......Page 933
Time domain models......Page 934
Models depending on the initial number of bugs......Page 935
Counting process models......Page 937
Model unification......Page 938
The Jelinski and Moranda model......Page 939
Simple Bayesian approach......Page 940
Hierarchical Bayesian approach......Page 941
Bayesian inference and prediction for the JM model......Page 942
Gibbs sampler for the LV model......Page 943
Bayesian inference and prediction for the LV models......Page 945
General order statistics and NHPP-I......Page 946
Record value statistics and NHPP-II......Page 948
Gibbs sampling for the general order statistics model......Page 949
The Pareto order statistics model.......Page 950
The exponential process.......Page 951
The extreme value process.......Page 952
Bayesian inference for NHPP......Page 953
Superposed NHPP processes......Page 954
Gibbs sampling for the full model......Page 955
Superposition of Musa-Okumoto and Weibull processes.......Page 956
Bayesian inference for the superposed models......Page 957
Gibbs sampling for nested models......Page 958
Bayesian inference for the nested model.......Page 959
Model selection......Page 960
Optimal release policy......Page 962
References......Page 963
Some areas of application......Page 968
Small area models......Page 969
Basic unit level model......Page 970
Inference from small area models......Page 971
Empirical Bayes small area estimation......Page 972
Hierarchical Bayes small area estimation......Page 976
Model MII......Page 982
Conclusion......Page 983
References......Page 984
Introduction......Page 986
Commonalities across groups in teaching Bayesian methods......Page 987
Motivation and conceptual explanations: One solution......Page 989
Active learning and repetition......Page 991
Assessment......Page 993
References......Page 994
Colour figures......Page 996
Subject Index......Page 1008
Handbook of Statistics: Contents of Previous Volumes......Page 1020