Continuous-time Markov decision processes (MDPs), also known as controlled Markov chains, are used for modeling decision-making problems that arise in operations research (for instance, inventory, manufacturing, and queueing systems), computer science, communications engineering, control of populations (such as fisheries and epidemics), and management science, among many other fields. This volume provides a unified, systematic, self-contained presentation of recent developments on the theory and applications of continuous-time MDPs. The MDPs in this volume include most of the cases that arise in applications, because they allow unbounded transition and reward/cost rates. Much of the material appears for the first time in book form.
Author(s): Xianping Guo, Onésimo Hernández-Lerma (auth.)
Series: Stochastic Modelling and Applied Probability 62
Edition: 1
Publisher: Springer-Verlag Berlin Heidelberg
Year: 2009
Language: English
Pages: 234
Tags: Operations Research, Mathematical Programming; Probability Theory and Stochastic Processes
Front Matter....Pages I-XVII
Introduction and Summary....Pages 1-8
Continuous-Time Markov Decision Processes....Pages 9-18
Average Optimality for Finite Models....Pages 19-53
Discount Optimality for Nonnegative Costs....Pages 55-70
Average Optimality for Nonnegative Costs....Pages 71-86
Discount Optimality for Unbounded Rewards....Pages 87-103
Average Optimality for Unbounded Rewards....Pages 105-125
Average Optimality for Pathwise Rewards....Pages 127-142
Advanced Optimality Criteria....Pages 143-162
Variance Minimization....Pages 163-173
Constrained Optimality for Discount Criteria....Pages 175-186
Constrained Optimality for Average Criteria....Pages 187-194
Back Matter....Pages 195-234
