The purpose of this book is to present the theory of general irreducible Markov chains and to point out the connection between this and the Perron-Frobenius theory of nonnegative operators. The author begins by providing some basic material designed to make the book self-contained, yet his principal aim throughout is to emphasize recent developments. The technique of embedded renewal processes, common in the study of discrete Markov chains, plays a particularly important role. The examples discussed indicate applications to such topics as queueing theory, storage theory, autoregressive processes and renewal theory. The book will therefore be useful to researchers in the theory and applications of Markov chains. It could also be used as a graduate-level textbook for courses on Markov chains or aspects of operator theory.
Author(s): Esa Nummelin
Series: Cambridge Tracts in Mathematics
Publisher: Cambridge University Press
Year: 2004
Language: English
Pages: 168
Contents......Page 7
Preface......Page 9
1.1. Kernels......Page 13
1.2. Markov chains......Page 15
2.1. Closed sets......Page 20
2.2. \phi-irreducibility......Page 23
2.3. The small functions......Page 26
2.4. Cyclicity......Page 32
3.1. Some potential theory......Page 37
3.2. R-transience and R-recurrence......Page 39
3.3. Stopping times for Markov chains......Page 43
3.4. Hitting and exit times......Page 45
3.5. The dissipative and conservative parts......Page 50
3.6. Recurrence......Page 53
4.1. Renewal sequences and renewal processes......Page 59
4.2. Kernels and Markov chains having a proper atom......Page 63
4.3. The general regeneration scheme......Page 70
4.4. The split chain......Page 72
5 Positive and null recurrence......Page 80
5.1. Subinvariant and invariant functions......Page 81
5.2. Subinvariant and invariant measures......Page 84
5.3. Expectations over blocks......Page 87
5.4. Recurrence of degree 2......Page 96
5.5. Geometric recurrence......Page 98
5.6. Uniform recurrence......Page 103
5.7. Degrees of #-recurrence......Page 106
6.1. Renewal theory......Page 110
6.2. Convergence of the iterates K^n(x,A)......Page 120
6.3. Ergodic Markov chains......Page 126
6.4. Ergodicity of degree 2......Page 130
6.5. Geometric ergodicity......Page 131
6.6. Uniform ergodicity......Page 134
6.7. R-ergodic kernels......Page 135
7.1. Sums of transition probabilities......Page 138
7.2. Ratios of sums of transition probabilities......Page 141
7.3. Ratios of transition probabilities......Page 143
7.4. A central limit theorem......Page 146
Notes and comments......Page 153
List of symbols and notation......Page 159
Bibliography......Page 160
Index......Page 167
