Generalized Linear Models and Extensions

Figures......Page 14
Tables......Page 16
Listings......Page 18
Preface......Page 20
1 Introduction......Page 23
1.1 Origins and motivation......Page 24
1.2 Notational conventions......Page 27
1.3 Applied or theoretical?......Page 28
1.4 Road map......Page 29
1.5 Installing the support materials......Page 32
I Foundations of Generalized Linear Models......Page 33
2 GLMs......Page 34
2.1 Components......Page 37
2.2 Assumptions......Page 39
2.3 Exponential family......Page 40
2.4 Example: Using an offset in a GLM......Page 43
2.5 Summary......Page 46
3 GLM estimation algorithms......Page 47
3.1 Newton–Raphson (using the observed Hessian)......Page 55
3.2 Starting values for Newton–Raphson......Page 57
3.3 IRLS (using the expected Hessian)......Page 59
3.4 Starting values for IRLS......Page 63
3.5 Goodness of fit......Page 64
3.6 Estimated variance matrices......Page 65
3.6.1 Hessian......Page 68
3.6.3 Sandwich......Page 69
3.6.4 Modified sandwich......Page 71
3.6.5 Unbiased sandwich......Page 72
3.6.6 Modified unbiased sandwich......Page 73
3.6.7 Weighted sandwich: Newey–West......Page 74
3.6.8 Jackknife......Page 76
One-step jackknife......Page 77
Variable jackknife......Page 78
Usual bootstrap......Page 79
Grouped bootstrap......Page 80
3.7 Estimation algorithms......Page 81
3.8 Summary......Page 83
4 Analysis of fit......Page 84
4.1 Deviance......Page 85
4.2.2 Overdispersion......Page 87
4.3 Assessing the link function......Page 90
4.4 Residual analysis......Page 92
4.4.2 Working residuals......Page 93
4.4.4 Partial residuals......Page 94
4.4.6 Deviance residuals......Page 95
4.4.9 Score residuals......Page 96
4.5 Checks for systematic departure from the model......Page 98
AIC......Page 99
BIC......Page 101
4.6.2 The interpretation of R......Page 102
A transformation of the likelihood ratio......Page 103
4.6.3 Generalizations of linear regression R......Page 104
Ben-Akiva and Lerman adjusted likelihood-ratio index......Page 105
Transformation of likelihood ratio......Page 106
The adjusted count R......Page 107
Cameron–Windmeijer R......Page 108
4.7.1 Marginal effects for GLMs......Page 109
4.7.2 Discrete change for GLMs......Page 114
II Continuous Response Models......Page 117
5 The Gaussian family......Page 118
5.1 Derivation of the GLM Gaussian family......Page 119
5.2 Derivation in terms of the mean......Page 121
5.3 IRLS GLM algorithm (nonbinomial)......Page 124
5.4 ML estimation......Page 127
5.5 GLM log-Gaussian models......Page 129
5.6 Expected versus observed information matrix......Page 131
5.7 Other Gaussian links......Page 133
5.8 Example: Relation to OLS......Page 134
5.9 Example: Beta-carotene......Page 137
6 The gamma family......Page 149
6.1 Derivation of the gamma model......Page 150
6.2 Example: Reciprocal link......Page 153
6.3 ML estimation......Page 158
6.4 Log-gamma models......Page 160
6.5 Identity-gamma models......Page 165
6.6 Using the gamma model for survival analysis......Page 167
7 The inverse Gaussian family......Page 171
7.1 Derivation of the inverse Gaussian model......Page 172
7.2 Shape of the distribution......Page 174
7.3 The inverse Gaussian algorithm......Page 179
7.4 Maximum likelihood algorithm......Page 180
7.5 Example: The canonical inverse Gaussian......Page 181
7.6 Noncanonical links......Page 183
8 The power family and link......Page 188
8.1 Power links......Page 189
8.2 Example: Power link......Page 191
8.3 The power family......Page 193
III Binomial Response Models......Page 195
9 The binomial–logit family......Page 196
9.1 Derivation of the binomial model......Page 198
9.2 Derivation of the Bernoulli model......Page 202
9.3 The binomial regression algorithm......Page 204
9.4 Example: Logistic regression......Page 206
9.4.2 Model producing logistic odds ratios......Page 207
9.5 GOF statistics......Page 211
9.6 Grouped data......Page 216
9.7 Interpretation of parameter estimates......Page 217
10 The general binomial family......Page 228
10.1 Noncanonical binomial models......Page 229
10.2 Noncanonical binomial links (binary form)......Page 231
10.3 The probit model......Page 233
10.4 The clog-log and log-log models......Page 239
10.5 Other links......Page 247
10.6.1 Identity link......Page 249
10.6.2 Logit link......Page 250
10.6.3 Log link......Page 251
10.6.4 Log complement link......Page 252
10.6.5 Log-log link......Page 253
10.6.6 Complementary log-log link......Page 254
10.6.7 Summary......Page 255
10.7 Generalized binomial regression......Page 256
10.8 Beta binomial regression......Page 263
10.9 Zero-inflated models......Page 266
11 The problem of overdispersion......Page 269
11.1 Overdispersion......Page 270
11.2 Scaling of standard errors......Page 278
11.3 Williams’ procedure......Page 286
11.4 Robust standard errors......Page 289
IV Count Response Models......Page 291
12 The Poisson family......Page 292
12.1 Count response regression models......Page 293
12.2 Derivation of the Poisson algorithm......Page 294
12.3 Poisson regression: Examples......Page 300
12.4 Example: Testing overdispersion in the Poisson model......Page 306
12.5 Using the Poisson model for survival analysis......Page 309
12.6 Using offsets to compare models......Page 311
12.7 Interpretation of coefficients......Page 314
13 The negative binomial family......Page 316
13.1 Constant overdispersion......Page 319
13.2.1 Derivation in terms of a Poisson–gamma mixture......Page 322
13.2.2 Derivation in terms of the negative binomial probability function......Page 325
13.2.3 The canonical link negative binomial parameterization......Page 328
13.3 The log-negative binomial parameterization......Page 330
13.4 Negative binomial examples......Page 333
13.5 The geometric family......Page 341
13.6 Interpretation of coefficients......Page 346
14 Other count-data models......Page 348
14.1 Count response regression models......Page 349
14.2 Zero-truncated models......Page 353
14.3 Zero-inflated models......Page 358
14.4 General truncated models......Page 369
14.5 Hurdle models......Page 375
14.6 Negative binomial(P) models......Page 382
14.7 Negative binomial(Famoye)......Page 390
14.8 Negative binomial(Waring)......Page 392
14.9 Heterogeneous negative binomial models......Page 394
14.10 Generalized Poisson regression models......Page 399
14.11 Poisson inverse Gaussian models......Page 403
14.12 Censored count response models......Page 406
14.13 Finite mixture models......Page 418
14.14 Quantile regression for count outcomes......Page 424
14.15 Heaped data models......Page 427
V Multinomial Response Models......Page 435
15 Unordered-response family......Page 436
15.1.1 Interpretation of coefficients: Single binary predictor......Page 437
15.1.2 Example: Relation to logistic regression......Page 440
15.1.3 Example: Relation to conditional logistic regression......Page 441
15.1.4 Example: Extensions with conditional logistic regression......Page 443
15.1.5 The independence of irrelevant alternatives......Page 444
15.1.6 Example: Assessing the IIA......Page 446
15.1.7 Interpreting coefficients......Page 448
15.1.8 Example: Medical admissions—introduction......Page 449
15.1.9 Example: Medical admissions—summary......Page 452
15.2 The multinomial probit model......Page 458
15.2.1 Example: A comparison of the models......Page 460
15.2.2 Example: Comparing probit and multinomial probit......Page 463
15.2.3 Example: Concluding remarks......Page 468
16 The ordered-response family......Page 470
16.1 Interpretation of coefficients: Single binary predictor......Page 472
16.2 Ordered outcomes for general link......Page 475
16.3.2 Ordered probit......Page 478
16.3.4 Ordered log-log......Page 479
16.3.5 Ordered cauchit......Page 480
16.4 Generalized ordered outcome models......Page 482
16.5 Example: Synthetic data......Page 485
16.6 Example: Automobile data......Page 494
16.7 Partial proportional-odds models......Page 503
16.8 Continuation-ratio models......Page 509
16.9 Adjacent category model......Page 516
VI Extensions to the GLM......Page 518
17 Extending the likelihood......Page 519
17.1 The quasilikelihood......Page 520
17.2 Example: Wedderburn’s leaf blotch data......Page 522
17.3 Example: Tweedie family variance......Page 533
17.4 Generalized additive models......Page 537
18 Clustered data......Page 538
18.1 Generalization from individual to clustered data......Page 539
18.2 Pooled estimators......Page 540
18.3.1 Unconditional fixed-effects estimators......Page 542
18.3.2 Conditional fixed-effects estimators......Page 543
18.4.1 Maximum likelihood estimation......Page 547
18.4.2 Gibbs sampling......Page 552
18.5 Mixed-effect models......Page 555
18.6 GEEs......Page 559
18.7 Other models......Page 564
19 Bivariate and multivariate models......Page 569
19.1 Bivariate and multivariate models for binary outcomes......Page 570
19.2 Copula functions......Page 571
19.3 Using copula functions to calculate bivariate probabilities......Page 572
19.4 Synthetic datasets......Page 574
19.5 Examples of bivariate count models using copula functions......Page 578
19.6 The Famoye bivariate Poisson regression model......Page 586
19.7 The Marshall–Olkin bivariate negative binomial regression model......Page 589
19.8 The Famoye bivariate negative binomial regression model......Page 593
20 Bayesian GLMs......Page 599
20.1 Brief overview of Bayesian methodology......Page 600
20.1.1 Specification and estimation......Page 603
20.1.2 Bayesian analysis in Stata......Page 606
20.2.1 Bayesian logistic regression—noninformative priors......Page 614
20.2.2 Diagnostic plots......Page 618
20.2.3 Bayesian logistic regression—informative priors......Page 622
20.3 Bayesian probit regression......Page 628
20.4 Bayesian complementary log-log regression......Page 631
20.5 Bayesian binomial logistic regression......Page 633
20.6.1 Bayesian Poisson regression with noninformative priors......Page 636
20.6.2 Bayesian Poisson with informative priors......Page 638
20.7 Bayesian negative binomial likelihood......Page 645
20.7.1 Zero-inflated negative binomial logit......Page 646
20.8 Bayesian normal regression......Page 650
20.9.1 Using the llf() option......Page 654
Bayesian logistic regression using llf()......Page 655
Bayesian zero-inflated negative binomial logit regression using llf()......Page 657
Logistic regression model using llevaluator()......Page 660
Bayesian clog-log regression with llevaluator()......Page 662
Bayesian Poisson regression with llevaluator()......Page 664
Bayesian negative binomial regression using llevaluator()......Page 665
Zero-inflated negative binomial logit using llevaluator()......Page 668
Bayesian gamma regression using llevaluator()......Page 672
Bayesian inverse Gaussian regression using llevaluator()......Page 675
Bayesian zero-truncated Poisson using llevaluator()......Page 678
Bayesian bivariate Poisson using llevaluator()......Page 681
VII Stata Software......Page 686
21 Programs for Stata......Page 687
21.1.1 Syntax......Page 688
21.1.2 Description......Page 689
21.1.3 Options......Page 690
21.2.2 Options......Page 696
21.3.2 User-written variance functions......Page 700
21.3.3 User-written programs for link functions......Page 703
21.3.4 User-written programs for Newey–West weights......Page 704
21.4.3 Special comments on family(binomial) models......Page 707
21.4.4 Special comments on family(nbinomial) models......Page 708
21.4.5 Special comment on family(gamma) link(log) models......Page 709
22 Data synthesis......Page 710
22.1 Generating correlated data......Page 711
22.2.1 Generating data for linear regression......Page 718
22.2.2 Generating data for logistic regression......Page 721
22.2.3 Generating data for probit regression......Page 723
22.2.4 Generating data for complimentary log-log regression......Page 725
22.2.5 Generating data for Gaussian variance and log link......Page 726
22.2.6 Generating underdispersed count data......Page 727
22.3.1 Heteroskedasticity in linear regression......Page 731
22.3.2 Power analysis......Page 734
22.3.3 Comparing fit of Poisson and negative binomial......Page 735
22.3.4 Effect of missing covariate on......Page 739
A Tables......Page 742
References......Page 757
Author index......Page 771
Subject index......Page 778