Markov Decision Processes with Their Applications (Advances in Mechanics and Mathematics)

Cover......Page 1
Advances in Mechanics and Mathematics VOLUME 14......Page 3
Markov Decision Processes with Their Applications......Page 4
9780387369501......Page 5
Contents......Page 6
List of Figures......Page 10
List of Tables......Page 12
Preface......Page 14
Acknowledgments......Page 16
1 A Brief Description of Markov Decision Processes......Page 18
2 Overview of the Book......Page 21
3 Organization of the Book......Page 23
1.1 System Model......Page 28
1.2 Some Concepts......Page 29
1.3 Finiteness of the Reward......Page 31
2.1 Validity of the Optimality Equation......Page 34
2.2 Properties of the Optimality Equation......Page 38
3 Properties of Optimal Policies......Page 42
4 Successive Approximation......Page 47
5 Sufficient Conditions......Page 49
6 Notes and References......Page 51
1 Model and Preliminaries......Page 56
2 Optimality Equation......Page 60
2.2 Sufficient Conditions......Page 65
2.3 Recurrent Conditions......Page 67
3 Optimality Inequalities......Page 70
3.1 Conditions......Page 71
3.2 Properties of ACOI and Optimal Policies......Page 74
4 Notes and References......Page 77
1.1 Model and Conditions......Page 80
1.2 Model Decomposition......Page 84
1.3 Some Properties......Page 88
1.4 Optimality Equation and Optimal Policies......Page 94
2.1 Model and Conditions......Page 102
2.2 Optimality Equation......Page 104
3 A Stationary Model: Average Criterion......Page 112
1.1 Model......Page 122
1.2 Regular Conditions......Page 124
1.3 Criteria......Page 127
2 Transformation......Page 128
2.1 Total Reward......Page 129
2.2 Average Criterion......Page 132
3 Notes and References......Page 136
1.1 Model......Page 138
1.2 Optimality Equation......Page 144
1.3 Approximation by Weak Convergence......Page 154
1.4 Markov Environment......Page 157
1.5 Phase Type Environment......Page 160
2.1 Model......Page 165
2.2 Optimality Equation......Page 169
2.3 Markov Environment......Page 175
3.1 Model......Page 177
3.2 Optimality Equation......Page 180
3.3 Markov Environment......Page 187
4 Notes and References......Page 191
1 System Model......Page 194
2 Optimality......Page 197
2.1 Maximum Discounted Total Reward......Page 199
3 Optimality in Event Feedback Control......Page 203
4 Link to Logic Level......Page 206
5 Resource Allocation System......Page 211
6 Notes and References......Page 218
1 System Model......Page 220
2 Optimality Equation and Optimal Supervisors......Page 224
3 Language Properties......Page 230
4 System Based on Automaton......Page 232
5.1 Event Feedback Control......Page 235
5.2 State Feedback Control......Page 239
6 Job-Matching Problem......Page 240
7. Notes and References......Page 247
9. OPTIMAL REPLACEMENT UNDERSTOCHASTIC ENVIRONMENTS......Page 250
1.1 Problem and Model......Page 251
1.2 Total Cost Criterion......Page 255
1.3 Average Criterion......Page 258
2.1 Problem......Page 261
2.2 Optimal Control Limit Policies......Page 264
2.3 Markov Environment......Page 267
2.4 Numerical Example......Page 275
3. Notes and References......Page 277
1. Problem and Model......Page 282
2. Analysis for Private Reserve Price......Page 284
3. Analysis for Announced Reserve Price......Page 288
4. Monotone Properties......Page 290
5. Numerical Results......Page 299
6. Notes and References......Page 301
References......Page 304
Index......Page 312