Probability and Random Processes

PROBABILITY AND RANDOM PROCESSES......Page 4
Contents......Page 10
Preface......Page 14
1.1 Set Definitions......Page 18
1.2 Set Operations......Page 20
1.3 Set Algebras, Fields, and Events......Page 25
2.1 Probability Space......Page 27
2.2 Conditional Probability......Page 31
2.3 Independence......Page 35
2.4 Total Probability and Bayes’ Theorem......Page 37
3.1 Basic Counting Principles......Page 42
3.2 Permutations......Page 43
3.3 Combinations......Page 45
4.1 Bernoulli Trials......Page 54
4.2 Binomial Distribution......Page 55
4.3 Multinomial Distribution......Page 58
4.4 Geometric Distribution......Page 59
4.5 Negative Binomial Distribution......Page 61
4.6 Hypergeometric Distribution......Page 63
4.7 Poisson Distribution......Page 65
4.8 Logarithmic Distribution......Page 72
4.9 Summary of Discrete Distributions......Page 79
5.1 Definition of Random Variables......Page 81
5.2 Determination of Distribution and Density Functions......Page 83
5.3 Properties of Distribution and Density Functions......Page 90
5.4 Distribution Functions from Density Functions......Page 92
6.2 Uniform Distribution......Page 96
6.3 Exponential Distribution......Page 97
6.4 Normal or Gaussian Distribution......Page 101
7.2 Triangular Distribution......Page 112
7.3 Laplace Distribution......Page 113
7.4 Erlang Distribution......Page 114
7.5 Gamma Distribution......Page 116
7.6 Weibull Distribution......Page 118
7.7 Chi-Square Distribution......Page 119
7.8 Chi and Other Allied Distributions......Page 121
7.9 Student-t Density......Page 127
7.10 Snedecor F Distribution......Page 128
7.11 Lognormal Distribution......Page 129
7.12 Beta Distribution......Page 131
7.13 Cauchy Distribution......Page 132
7.14 Pareto Distribution......Page 134
7.16 Mixed Distributions......Page 135
7.17 Summary of Distributions of Continuous Random Variables......Page 136
8.1 Conditional Distribution and Density for P(A) = 0......Page 139
8.2 Conditional Distribution and Density for P(A) = 0......Page 143
8.3 Total Probability and Bayes’ Theorem for Densities......Page 148
9.1 Joint Discrete Distribution Functions......Page 152
9.2 Joint Continuous Distribution Functions......Page 153
9.3 Bivariate Gaussian Distributions......Page 161
10.1 Expectations......Page 163
10.2 Variance......Page 166
10.3 Means and Variances of Some Distributions......Page 167
10.4 Higher-Order Moments......Page 170
10.5 Bivariate Gaussian......Page 171
11.1 Characteristic Functions......Page 172
11.2 Examples of Characteristic Functions......Page 174
11.3 Generating Functions......Page 178
11.4 Examples of Generating Functions......Page 179
11.5 Moment Generating Functions......Page 181
11.6 Cumulant Generating Functions......Page 184
11.7 Table of Means and Variances......Page 187
12.1 Random Variable g(X)......Page 190
12.2 Distribution of Y = g(X)......Page 191
12.3 Direct Determination of Density f(Y)(y) from f(X)(x)......Page 211
12.4 Inverse Problem: Finding g(x) Given f(X)(x) and f(Y)(y)......Page 217
12.5 Moments of a Function of a Random Variable......Page 219
13.1 Function of Two Random Variables, Z = g(X,Y)......Page 223
13.2 Two Functions of Two Random Variables, Z = g(X,Y), W = h(X,Y)......Page 239
13.3 Direct Determination of Joint Density f(ZW)(z,w) from f(XY)(x,y)......Page 244
13.4 Solving Z = g(X,Y) Using an Auxiliary Random Variable......Page 250
13.5 Multiple Functions of Random Variables......Page 255
14.1 Degenerate Random Variables......Page 258
14.2 Chebyshev and Allied Inequalities......Page 259
14.3 Markov Inequality......Page 263
14.4 Chernoff Bound......Page 265
14.5 Cauchy–Schwartz Inequality......Page 268
14.6 Jensen’s Inequality......Page 271
14.7 Convergence Concepts......Page 273
14.8 Limit Theorems......Page 276
15.1 Uniform-Distribution Random Variates......Page 281
15.2 Histograms......Page 283
15.3 Inverse Transformation Techniques......Page 286
15.4 Convolution Techniques......Page 296
15.5 Acceptance–Rejection Techniques......Page 297
16.1 Basic Theory of Matrices......Page 301
16.2 Eigenvalues and Eigenvectors of Matrices......Page 310
16.3 Vectors and Matrix Differentiations......Page 318
16.4 Block Matrices......Page 325
17.1 Distributions and Densities......Page 328
17.2 Moments of Random Vectors......Page 336
17.3 Vector Gaussian Random Variables......Page 340
17.4 Diagonalization of Covariance Matrices......Page 347
17.5 Simultaneous Diagonalization of Covariance Matrices......Page 351
17.6 Linear Estimation of Vector Variables......Page 354
18.1 Criteria of Estimators......Page 357
18.2 Estimation of Random Variables......Page 359
18.3 Estimation of Parameters (Point Estimation)......Page 367
18.4 Interval Estimation (Confidence Intervals)......Page 381
18.5 Hypothesis Testing (Binary)......Page 390
18.6 Bayesian Estimation......Page 401
19.1 Basic Definitions......Page 423
19.2 Stationary Random Processes......Page 437
19.3 Ergodic Processes......Page 456
19.4 Estimation of Parameters of Random Processes......Page 462
19.5 Power Spectral Density......Page 489
20.1 Specifications of Random Processes......Page 507
20.2 Poisson Process......Page 509
20.3 Binomial Process......Page 522
20.4 Independent Increment Process......Page 524
20.5 Random-Walk Process......Page 529
20.6 Gaussian Process......Page 538
20.7 Wiener Process (Brownian Motion)......Page 540
20.8 Markov Process......Page 544
20.9 Markov Chain......Page 553
20.10 Martingale Process......Page 568
20.11 Periodic Random Process......Page 574
20.12 Aperiodic Random Process (Karhunen–Loeve Expansion)......Page 580
21.1 Review of Linear Systems......Page 591
21.2 Random Processes through Linear Systems......Page 595
21.3 Linear Filters......Page 609
21.4 Bandpass Stationary Random Processes......Page 623
22.1 Review of Orthogonality Principle......Page 642
22.2 Wiener Filtering......Page 644
22.3 Discrete Kalman Filter......Page 664
22.4 Continuous Kalman Filter......Page 677
23.1 Introduction......Page 683
23.2 Stochastic Model......Page 684
23.3 Stochastic Estimation Algorithm......Page 688
23.4 Prior Distribution P{M}......Page 691
23.5 Computer Simulation......Page 693
23.6 Results and Conclusions......Page 695
23.8 References for Chapter 23......Page 698
A A Fourier Transform Tables......Page 700
B Cumulative Gaussian Tables......Page 704
C Inverse Cumulative Gaussian Tables......Page 709
D Inverse Chi-Square Tables......Page 711
E Inverse Student-t Tables......Page 718
F Cumulative Poisson Distribution......Page 721
G Cumulative Binomial Distribution......Page 725
References......Page 731
Index......Page 733