This book gives an introduction to discrete-time Markov chains which evolve on a separable metric space.
The focus is on the ergodic properties of such chains, i.e., on their long-term statistical behaviour. Among the main topics are existence and uniqueness of invariant probability measures, irreducibility, recurrence, regularizing properties for Markov kernels, and convergence to equilibrium. These concepts are investigated with tools such as Lyapunov functions, petite and small sets, Doeblin and accessible points, coupling, as well as key notions from classical ergodic theory. The theory is illustrated through several recurring classes of examples, e.g., random contractions, randomly switched vector fields, and stochastic differential equations, the latter providing a bridge to continuous-time Markov processes.
The book can serve as the core for a semester- or year-long graduate course in probability theory with an emphasis on Markov chains or random dynamics. Some of the material is also well suited for an ergodic theory course. Readers should have taken an introductory course on probability theory, based on measure theory. While there is a chapter devoted to chains on a countable state space, a certain familiarity with Markov chains on a finite state space is also recommended.
Author(s): Michel Benaïm, Tobias Hurth
Series: Universitext
Edition: 1
Publisher: Springer Nature
Year: 2022
Language: English
Pages: 197
City: Cham
Tags: Markov Chains
Preface
Contents
Preliminaries
1 Markov Chains
1.1 Markov Kernels
1.2 Markov Chains
1.3 The Canonical Chain
1.4 Markov and Strong Markov Properties
1.5 Continuous Time: Markov Processes
2 Countable Markov Chains
2.1 Recurrence and Transience
2.1.1 Positive Recurrence
2.1.2 Null Recurrence
2.2 Subsets of Recurrent Sets
2.3 Recurrence and Lyapunov Functions
2.4 Aperiodic Chains
2.5 The Convergence Theorem
2.6 Application to Renewal Theory
2.6.1 Coupling of Renewal Processes
2.7 Convergence Rates for Positive Recurrent Chains
Notes
3 Random Dynamical Systems
3.1 General Definitions
3.2 Representation of Markov Chains by RDS
Notes
4 Invariant and Ergodic Probability Measures
4.1 Weak Convergence of Probability Measures
4.1.1 Tightness and Prohorov's Theorem
A Tightness Criterion
4.2 Invariant Measures
4.2.1 Tightness Criteria for Empirical Occupation Measures
4.3 Excessive Measures
4.4 Ergodic Measures
4.5 Unique Ergodicity
4.5.1 Unique Ergodicity of Random Contractions
4.6 Classical Results from Ergodic Theory
4.6.1 Poincaré, Birkhoff, and Ergodic Decomposition Theorems
4.7 Application to Markov Chains
4.8 Continuous Time: Invariant Probabilities for Markov Processes
Notes
5 Irreducibility
5.1 Resolvent and ξ-Irreducibility
5.2 The Accessible Set
5.2.1 Continuous Time: Accessibility
5.3 The Asymptotic Strong Feller Property
5.3.1 Strong Feller Implies Asymptotic Strong Feller
5.3.2 A Sufficient Condition for the Asymptotic Strong Feller Property
5.3.3 Unique Ergodicity of Asymptotic Strong Feller Chains
Notes
6 Petite Sets and Doeblin Points
6.1 Petite Sets, Small Sets, Doeblin Points
6.1.1 Continuous Time: Doeblin Points for Markov Processes
6.2 Random Dynamical Systems
6.3 Random Switching Between Vector Fields
6.3.1 The Weak Bracket Condition
6.4 Piecewise Deterministic Markov Processes
6.4.1 Invariant Measures
6.4.2 The Strong Bracket Condition
6.5 Stochastic Differential Equations
6.5.1 Accessibility
6.5.2 Hörmander Conditions
Notes
7 Harris and Positive Recurrence
7.1 Stability and Positive Recurrence
7.2 Harris Recurrence
7.2.1 Petite Sets and Harris Recurrence
7.3 Recurrence Criteria and Lyapunov Functions
7.4 Subsets of Recurrent Sets
7.5 Petite Sets and Positive Recurrence
7.6 Positive Recurrence for Feller Chains
7.6.1 Application to PDMPs
7.6.2 Application to SDEs
8 Harris Ergodic Theorem
8.1 Total Variation Distance
8.1.1 Coupling
8.2 Harris Convergence Theorems
8.2.1 Geometric Convergence
Aperiodic Small Sets
8.2.2 Continuous Time: Exponential Convergence
8.2.3 Coupling, Splitting, and Polynomial Convergence
8.3 Convergence in Wasserstein Distance
A Monotone Class and Martingales
A.1 Monotone Class Theorem
A.2 Conditional Expectation
A.3 Martingales
Bibliography
List of Symbols
List of Symbols
Index
