This book concerns discrete-time homogeneous Markov chains that admit an invariant probability measure. The main objective is to give a systematic, self-contained presentation on some key issues about the ergodic behavior of that class of Markov chains. These issues include, in particular, the various types of convergence of expected and pathwise occupation measures, and ergodic decompositions of the state space.
Author(s): Onésimo Hernández-Lerma, Jean B. Lasserre
Series: Progress in Mathematics
Edition: 1
Publisher: Birkhäuser Basel
Year: 2003
Language: English
Pages: 223
MARKOV CHAINS AND INVARIANT PROBABILITIES......Page 1
Title Page......Page 3
Copyright Page......Page 4
Contents......Page 6
Acknowledgements......Page 10
Preface......Page 12
List of Symbols......Page 16
1.2 Measures and Functions......Page 18
1.2.1 Measures and Signed Measures......Page 19
1.2.2 Function Spaces......Page 20
1.3 Weak Topologies......Page 21
1.4.1 Setwise Convergence......Page 23
1.4.2 Convergence in the Total Variation Norm......Page 25
1.4.3 Convergence of Measures in a Metric Space......Page 26
1.5.1 A Uniform Principle of Weak Convergence......Page 30
1.5.2 Fatou's Lemma, Monotone and Lebesgue Dominated Convergence for Measures......Page 32
1.6 Notes......Page 34
Part I: Markov Chains and Ergodicity......Page 36
2.1 Introduction......Page 38
2.2 Basic Notation and Definitions......Page 39
2.2.1 Examples......Page 41
2.2.2 Invariant Probability Measures......Page 42
2.3.1 The Chacon–Ornstein Theorem......Page 45
2.3.2 Ergodic Theorems for Markov Chains......Page 46
2.3.3 A "Dual" Ergodic Theorem......Page 48
2.4 The Ergodicity Property......Page 50
2.5 Pathwise Results......Page 53
2.6 Notes......Page 56
3.2.1 Communication......Page 58
3.2.3 Absorbing Sets and Irreducibility......Page 59
3.2.4 Recurrence and Transience......Page 60
3.3 Limit Theorems......Page 61
3.4 Notes......Page 63
4.2 Basic Definitions and Properties......Page 64
4.3 Characterization of Harris Recurrence via Occupation Measures......Page 68
4.3.1 Positive Harris Recurrent Markov Chains......Page 69
4.3.2 Aperiodic Positive Harris Recurrent Markov Chains......Page 70
4.3.3 Geometric Ergodicity......Page 71
4.4 Sufficient Conditions for P.H.R.......Page 73
4.5 Harris and Doeblin Decompositions......Page 77
4.6 Notes......Page 78
5.1 Introduction......Page 80
5.2.1 The Limiting Transition Probability Function......Page 81
5.3 Yosida's Ergodic Decomposition......Page 85
5.3.1 Additive-Noise Systems......Page 88
5.4 Pathwise Results......Page 90
5.5 Proofs......Page 91
5.5.1 Proof of Lemma 5.2.3......Page 92
5.5.2 Proof of Lemma 5.2.4......Page 93
5.5.3 Proof of Theorem 5.2.2......Page 95
5.5.4 Proof of Lemma 5.3.2......Page 96
5.5.5 Proof of Theorem 5.4.1......Page 97
5.6 Notes......Page 99
6.1 Introduction......Page 100
6.2 A Classification......Page 101
6.2.1 Examples......Page 104
6.3 On the Birkhoff Individual Ergodic Theorem......Page 105
6.3.1 Finitely-Additive Invariant Measures......Page 106
6.3.2 Discussion......Page 108
6.4 Notes......Page 109
Part II: Further Ergodicity Properties......Page 110
7.1 Introduction......Page 112
7.2.1 The Feller Property......Page 113
7.2.2 Sufficient Condition for Existence of an Invariant Probability Measure......Page 115
7.3 Quasi Feller Chains......Page 116
7.4 Notes......Page 119
8.2 The Poisson Equation......Page 120
8.3 Canonical Pairs......Page 122
8.4 The Cesàro-Averages Approach......Page 127
8.5 The Abelian Approach......Page 131
8.6 Notes......Page 136
9.1 Introduction......Page 138
9.2.1 Notation......Page 139
9.2.2 Ergodicity......Page 140
9.2.3 Strong Stability......Page 142
9.2.4 The Link with the Poisson Equation......Page 143
9.3 Weak and Weak Uniform Ergodicity......Page 144
9.3.1 Weak Ergodicity......Page 145
9.3.2 Weak Uniform Ergodicity......Page 146
9.4 Notes......Page 148
Part III: Existence and Approximation of Invariant Probability Measures......Page 150
10.1 Introduction and Statement of the Problems......Page 152
10.2 Notation and Definitions......Page 153
10.3.1 Problem P* 1......Page 155
10.3.2 Problem P* 2......Page 157
10.3.3 Problem P* 3......Page 158
10.4 Markov Chains in Locally Compact Separable Metric Spaces......Page 160
10.5.2 Lyapunov Sufficient Conditions......Page 162
10.6.1 On Finitely Additive Measures......Page 164
10.6.2 A Generalized Farkas Lemma......Page 165
10.7.1 Proof of Theorem 10.3.1......Page 166
10.7.2 Proof of Theorem 10.4.3......Page 169
10.7.3 Proof of Theorem 10.5.1......Page 171
10.8 Notes......Page 172
11.1 Introduction and Statement of the Problems......Page 174
11.2 Notation and Definitions......Page 175
11.3 Existence Results......Page 177
11.3.2 Problem P 1......Page 178
11.3.3 Problem P 2......Page 179
11.3.4 Problem P 3......Page 180
11.3.5 Example......Page 182
11.4.1 Proof of Theorem 11.3.1......Page 184
11.4.2 Proof of Theorem 11.3.2......Page 186
11.4.3 Proof of Theorem 11.3.3......Page 188
11.4.4 Proof of Theorem 11.3.5......Page 189
11.5 Notes......Page 191
12.1 Introduction......Page 192
12.2 Statement of the Problem and Preliminaries......Page 193
12.2.1 Constraint-Aggregation......Page 194
12.3 An Approximation Scheme......Page 195
12.3.1 Aggregation......Page 196
12.3.2 Aggregation-Relaxation-Inner Approximation......Page 197
12.4.1 Upper and Lower Bounds......Page 200
12.4.2 An Approximation Scheme......Page 202
12.4.3 Sharp Upper and Lower Bounds......Page 203
12.5 Notes......Page 207
Bibliography......Page 210
Index......Page 220
Back Cover......Page 223
