This book concerns discrete-time homogeneous Markov chains that admit an invariant probability measure. The main objective is to give a systematic, self-contained presentation on some key issues about the ergodic behavior of that class of Markov chains. These issues include, in particular, the various types of convergence of expected and pathwise occupation measures, and ergodic decompositions of the state space. Some of the results presented appear for the first time in book form. A distinguishing feature of the book is the emphasis on the role of expected occupation measures to study the long-run behavior of Markov chains on uncountable spaces.
The intended audience are graduate students and researchers in theoretical and applied probability, operations research, engineering and economics.
Author(s): Onésimo Hernández-Lerma, Jean Bernard Lasserre (auth.)
Series: Progress in Mathematics 211
Edition: 1
Publisher: Birkhäuser Basel
Year: 2003
Language: English
Pages: 208
Tags: Probability Theory and Stochastic Processes;Operations Research, Management Science;Mathematical Methods in Physics
Front Matter....Pages i-xvi
Preliminaries....Pages 1-18
Front Matter....Pages 19-19
Markov Chains and Ergodic Theorems....Pages 21-39
Countable Markov Chains....Pages 41-46
Harris Markov Chains....Pages 47-61
Markov Chains in Metric Spaces....Pages 63-82
Classification of Markov Chains via Occupation Measures....Pages 83-92
Front Matter....Pages 93-93
Feller Markov Chains....Pages 95-102
The Poisson Equation....Pages 103-120
Strong and Uniform Ergodicity....Pages 121-131
Front Matter....Pages 133-133
Existence and Uniqueness of Invariant Probability Measures....Pages 135-155
Existence and Uniqueness of Fixed Points for Markov Operators....Pages 157-174
Approximation Procedures for Invariant Probability Measures....Pages 175-191
Back Matter....Pages 193-208
