Non-Homogeneous Markov Chains and Systems: Theory and Applications fulfills two principal goals. It is devoted to the study of non-homogeneous Markov chains in the first part, and to the evolution of the theory and applications of non-homogeneous Markov systems (populations) in the second. The book is self-contained, requiring a moderate background in basic probability theory and linear algebra, common to most undergraduate programs in mathematics, statistics, and applied probability. There are some advanced parts, which need measure theory and other advanced mathematics, but the readers are alerted to these so they may focus on the basic results.
Features
- A broad and accessible overview of non-homogeneous Markov chains and systems
- Fills a significant gap in the current literature
- A good balance of theory and applications, with advanced mathematical details separated from the main results
- Many illustrative examples of potential applications from a variety of fields
- Suitable for use as a course text for postgraduate students of applied probability, or for self-study
- Potential applications included could lead to other quantitative areas
The book is primarily aimed at postgraduate students, researchers, and practitioners in applied probability and statistics, and the presentation has been planned and structured in a way to provide flexibility in topic selection so that the text can be adapted to meet the demands of different course outlines. The text could be used to teach a course to students studying applied probability at a postgraduate level or for self-study. It includes many illustrative examples of potential applications, in order to be useful to researchers from a variety of fields.
Author(s): P.-C.G. Vassiliou
Edition: 1
Publisher: CRC Press
Year: 2022
Language: English
Pages: 450
Tags: Markov Chains, Non-Homogeneous
Cover
Half Title
Title Page
Copyright Page
Dedication
Contents
Preface
Acknowledgments
1. FOUNDATIONS OF PROBABILITY THEORY
1.1. Introductory notes
1.2. Some set theory and topology
1.2.1. Set theory useful in probability
1.2.2. Interesting topological spaces
1.3. Important family of sets in probability theory
1.4. Measurable spaces
1.5. Probability spaces
1.5.1. Infinite probability spaces
1.6. Filtration
1.7. Random variables
1.8. Integration with respect to a probability measure
1.9. Indicator functions
1.10. The space L2 is a Hilbert space
1.11. Independent σ-algebras and random variables
1.12. Convergence of sequences of random variables
1.13. The laws of large numbers and the central limit theorem
1.14. Conditional distributions and conditional expectations
1.15. Change of measure
1.16. Existence and uniqueness of conditional expectations
1.17. Properties of conditional expectation
2. A SMALL REVIEW OF MATRIX ANALYSIS
2.1. Introductory notes
2.2. Matrices
2.2.1. Similarity
2.3. The minimal polynomial of A
2.3.1. Invariant polynomials and elementary divisors
2.3.2. The Jordan canonical form
2.3.3. Remarks on the Jordan canonical form of a matrix
2.3.4. Construction of the transformation matrix of A to its Jordan form
2.4. The norm of a vector
2.5. The matrix norm
2.6. The Kronecker product of two matrices
2.7. The Hadamard product of matrices
2.8. Canonical forms of a matrix
2.9. Generalized inverses
2.10. The Moore-Penrose generalized matrix
2.11. The Drazin inverse and the group inverse
2.12. The group inverse and Markov chains
2.13. Sensitivity of Markov chains
3. NON-HOMOGENEOUS MARKOV CHAINS; WEAK ERGODICITY
3.1. Introductory notes
3.2. Stochastic processes
3.3. Markov chain
3.3.1. A more advanced definition of a non-homogeneous Markov chain
3.4. The life and work of A.A. Markov
3.5. Probability distribution in the states of an NHMC
3.6. Examples
3.6.1. A more advanced result on the relation between a G-non-homogeneous Markov chain and martingales
3.7. Weak and strong ergodicity
3.8. Structures for coefficients of ergodicity
3.9. Conditions for weak ergodicity for general products
3.9.1. A synopsis of the life and mathematical legacy of Wolfgang Doeblin
3.9.2. General products of stochastic matrices
3.9.3. Consequences of the above general theorems to weak ergodicity of inhomogeneous Markov chains
3.9.4. An application of backward products in non-homogeneous Markov chains
3.10. The dominant role of the Dobrushin ergodicity coefficient
3.11. Transition probability matrices are in chronological order
3.12. Examples on the use of weak ergodicity theorems
4. NON-HOMOGENEOUS MARKOV CHAINS; STRONG ERGODICITY
4.1. Strong ergodicity
4.2. Ergodicity and geometric strong ergodicity
4.3. Criteria for strong ergodicity for NHMC
4.4. Convergence in the Cesaro sense
4.4.1. Cyclic subclasses
4.4.2. The non-homogeneous case
4.5. Uniform strong ergodicity with the use of mean visit times
4.6. Strong ergodicity for general products of matrices
4.7. Sets of matrices all products of which converge
4.8. A geometric approach to ergodic NHMC
4.9. Asymptotic behavior with arbitrary stochastic matrices
4.9.1. An illustrative example
5. THE NON-HOMOGENEOUS MARKOV SYSTEM
5.1. Introductory notes
5.2. The non-homogeneous Markov system in discrete time and space
5.3. The expected and relative expected population structure
5.3.1. An expanding NHMS
5.3.2. A fluctuating NHMS without forced wastage
5.4. A range of NHMS’s environment
5.5. The G-non-homogeneous Markov system
5.6. Change of measure in G-non-homogeneous Markov systems
5.7. The space of random population structures as a Hilbert space
5.8. Estimation of the transition probabilities of an NHMS
5.9. Research notes
6. ASYMPTOTIC BEHAVIOR OF A NON-HOMOGENEOUS MARKOV SYSTEM
6.1. Introductory notes
6.2. Asymptotic behavior of the expected population structure
6.3. The relative expected population structure
6.4. A contracting NHMS without forced wastage
6.5. NHMS with non-homogeneous Poisson recruitment
6.6. Asymptotic stability in NHMS’s
6.7. NHMS as a martingale with Poisson input
6.8. Research notes
7. ASYMPTOTIC VARIABILITY OF NHMS
7.1. Introductory notes
7.2. The V matrices and their properties
7.3. An illustrative probable application
7.4. Rate of convergence of the variability vector
7.5. Research notes
8. CYCLIC BEHAVIOR OF NON-HOMOGENEOUS MARKOV SYSTEMS
8.1. Introductory notes
8.2. Cyclic non-homogeneous Markov systems
8.3. An illustrative example from a manpower system
8.4. Rate of convergence of {μ(t)}∞t=0 in an NHMS under cyclic behavior
8.5. Research notes
9. STOCHASTIC CONTROL IN NHMS’S
9.1. Introductory notes
9.2. Maintainability in NHMS by input control
9.3. Attainability in NHMS by input control
9.4. Asymptotically attainable structures
9.5. Periodicity of asymptotically attainable structures
9.6. An illustrative example
9.7. Control of asymptotic variability in NHMS
9.8. The NHMS in a stochastic environment
9.8.1. The expected population structure of the S-NHMS
9.8.2. Evaluating the expected transition probability matrix E[P(t)].
9.8.3. Expected value of a random number of Bernoulli trials with probability of success a random variable
9.8.4. The expected population structure
9.9. Maintainability in a stochastic environment
9.10. Strategies for attaining a structure in an S-NHMS
9.11. An illustrative application
9.12. Research notes
10. LAWS OF LARGE NUMBERS FOR NON-HOMOGENEOUS MARKOV SYSTEMS
10.1. Introductory notes
10.2. Basic concepts and useful results
10.3. Laws of large numbers for an NHMS
10.4. An illustrative application
10.5. Laws of large numbers for a Cy-NHMS
10.6. An illustrative application
10.7. LLN for NHMS with arbitrary probability matrices
10.7.1. Cesaro convergence for Markov chains with arbitrary transition probability matrices
10.7.2. Cesaro convergence for an NHMS with arbitrary transition probability matrices for the inherent Markov chain
10.7.3. Law of large numbers for Cesaro sums of expected population structures
10.8. An illustrative example
11. THE S-NHMS IN CONTINUOUS TIME
11.1. Introductory notes
11.2. Asymptotic behavior of NHMP in continuous time
11.3. The S-NHMS in continuous time
11.4. The expected population structure of the S-NHMSC
11.5. The asymptotic behavior of the S-NHMSC
11.6. An illustrative example from manpower planning
11.7. Research notes
12. THE PERTURBED NON-HOMOGENEOUS MARKOV SYSTEM
12.1. Introductory notes
12.2. The group inverse and the asymptotic behavior
12.3. The perturbed non-homogeneous Markov system
12.4. The oscillation of the matrix Q in manpower systems
12.5. Asymptotic behavior of the P-NHMS
12.6. Sensitivity of the limiting distributions
12.7. Asymptotic variability of the P-NHMS
12.8. Research notes
13. NON-HOMOGENEOUS MARKOV SET SYSTEM
13.1. Introduction
13.2. Non-homogeneous Markov set system
13.3. The set of the expected relative population structures
13.4. Asymptotic behavior of NHMSS
13.5. Properties of the limiting set Rng∞ (S, R0)
13.6. An illustrative representative example
14. MARKOV SYSTEMS ON A GENERAL STATE SPACE
14.1. Introductory notes
14.2. A mental image for MSGS
14.3. The foundation of a MSGS
14.4. Asymptotic behavior or ergodicity of MSGS
14.5. Total variability from the invariant measure; a coupling theorem
14.6. Rate of convergence of MSGS
14.7. Asymptotic periodicity of a MSGS
14.8. Total variability from the invariant measures
15. THE G–NON-HOMOGENEOUS MARKOV SYSTEM OF HIGH ORDER
15.1. Introductory notes
15.2. The G-non-homogeneous Markov system of high order
15.3. The population structure of the G-NHMS
15.4. The inhomogeneous mixture transition distribution model
15.5. The asymptotic behavior of the inherent Markov chain
15.6. The asymptotic expected population structure
15.7. An illustrative example from manpower systems
15.8. Research notes
REFERENCES
INDEX
