Dynamic Programming and the Calculus of Variations (Mathematics in Science and Engineering, Volume 21)

Dynamic Programming and the Calculus of Variations......Page 6
Copyright Page......Page 7
Contents......Page 16
Preface......Page 10
1. Introduction......Page 24
2. An Example of a Multistage Decision Process Problem......Page 25
3. The Dynamic Programming solution of the Example......Page 26
4. The Dynamic Programming Formalism......Page 30
5. Two Properties of the Optimal Value Function......Page 33
6. An Alternative Method of Solution......Page 36
8. A Property of Multistage Decision Processes......Page 38
9. Further Illustrative Examples......Page 39
10. Terminal Control Problems......Page 43
12. Solution of the Example......Page 44
13. Properties of the Solution of a Terminal Control Problem......Page 46
14. Summary......Page 47
1. Introduction......Page 48
3. Admissible Solutions......Page 49
5. Functionals......Page 50
7. Arc-Length......Page 51
8. The Simplest General Problem......Page 52
10. The Nature of Necessary Conditions......Page 53
11. Example......Page 54
14. The Absolute Minimum of a Functional......Page 55
15. A Relative Minimum of a Function......Page 56
17. A Weak Relative Minimum of a Functional......Page 57
18. Weak Variations......Page 59
19. The First and Second Variations......Page 61
20. The Euler-Lagrange Equation......Page 62
22. The Legendre Condition......Page 64
23. The Second Variation and the Second Derivative......Page 65
24. The Jacobi Necessary Condition......Page 66
26. Focal Point......Page 67
28. The Weierstrass Necessary Condition......Page 68
29. Example......Page 71
31. Transversality Conditions......Page 73
32. Corner Conditions......Page 74
33. Relative Summary......Page 75
34. Sufficient Conditions......Page 77
36. Other Problem Formulations......Page 78
37. Example of a Terminal Control Problem......Page 80
38. Necessary Conditions for the Problem of Mayer......Page 82
40. Two-Point Boundary Value Problems......Page 83
41. A Well-Posed Problem......Page 84
43. Computational Solution......Page 87
44. Summary......Page 88
References to Standard Texts......Page 89
1. Introduction......Page 92
2. Notation......Page 93
3. The Fundamental Partial Differential Equation......Page 94
4. A Connection with Classical Variations......Page 97
6. Two Kinds of Derivatives......Page 98
7. Discussion of the Fundamental Partial Differential Equation......Page 99
8. Characterization of the Optimal Policy Function......Page 101
9. Partial Derivatives along Optimal Curves......Page 102
10. Boundary Conditions for the Fundamental Equation: I......Page 104
11. Boundary Conditions: II......Page 106
12. An Illustrative Example—Variable End Point......Page 107
13. A Further Example—Fixed Terminal Point......Page 109
14. A Higher-Dimensional Example......Page 111
15. A Different Method of Analytic Solution......Page 112
16. An Example......Page 117
18. The Euler-Lagrange Equation......Page 120
19. A Second Derivation of the Euler-Lagrange Equation......Page 122
21. The Weierstrass Necessary Condition......Page 124
22. The Jacobi Necessary Condition......Page 126
23. Discussion of the Jacobi Condition......Page 129
25. An Illustrative Example......Page 130
26. Determination of Focal Points......Page 131
27. Example......Page 133
28. Discussion of the Example......Page 134
29. Transversality Conditions......Page 137
30. Second-Order Transversality Conditions......Page 139
31. Example......Page 141
32. The Weierstrass Erdmann Corner Conditions......Page 142
33. Summary......Page 143
34. A Rigorus Dynamic Programming Approach......Page 144
35. An Isoperimetric Problem......Page 148
36. The Hamilton-Jacobi Equation......Page 151
1. Introduction......Page 152
2. Statement of the Problem......Page 153
4. The Fundamental Partial Differential Equation......Page 155
6. Interpretation of the Fundamental Equation......Page 157
7. Boundary Conditions for the Fundamental Equation......Page 159
9. Two Necessary Conditions......Page 160
10. The Multiplier Rule......Page 161
12. The Weierstrass Necessary Condition......Page 163
14. The Second Partial Derivatives of the Optimal Value Function......Page 164
15. A Matrix Differential Equation of Riccati Type......Page 166
16. Terminal Values of the Second Partial Derivatives of the Optimal Value Function......Page 167
17. The Guidance Problem......Page 168
18. Terminal Transversality Conditions......Page 170
19. Initial Transversality Conditions......Page 173
21. A First Integral of the Solution......Page 174
22. The Variational Hamiltonian......Page 175
23. Corner Conditions......Page 176
24. An Example......Page 177
25. Second-Order Transversality Conditions......Page 179
26. Problem Discontinuities......Page 180
27. Optimization of Parameters......Page 182
28. A Caution......Page 184
29. Summary......Page 186
2. Control-Variable Inequality Constraints......Page 187
3. The Appropriate Multiplier Rule......Page 188
4. A Second Derivation of the Result of Section 3......Page 192
5. Discussion......Page 194
6. The Sign of the Control Impulse Response Function......Page 197
8. The Appropriate Modification of the Multiplier Rule......Page 198
9. The Conventional Notation......Page 199
10. A Second Derivation of the Result of Section 9......Page 200
11. Discussion......Page 201
12. The Sign of the New Multipier Function......Page 203
13. State-Variable Inequality Constraints......Page 204
14. The Optimal Value Function for a State-Constrained Problem......Page 205
15. Derivation of a Multiplier Rule......Page 206
16. Generalizations......Page 209
17. A Connection with Other Forms of the Results......Page 210
18. Summary......Page 211
2. Switching Manifolds......Page 213
3. A Problem That Is Linear in the Derivative......Page 214
4. Analysis of the Problem of Section 3......Page 215
5. Discussion......Page 218
6. A Problem with Linear Dynamics and Criterion......Page 219
7. Investigation of the Problem of Section 6......Page 220
8. Further Analysis of the Problem of Section 6......Page 221
9. Discussion......Page 226
10. Summary......Page 227
1. Introduction......Page 228
2. A Deterministic Problem......Page 230
3. A Stochastic Problem......Page 232
4. Discussion......Page 235
6. The Optimal Expected Value Function......Page 236
7. The Fundamental Recurrence Relation......Page 237
9. A Continuous Stochastic Control Problem......Page 238
10. The Optimal Expected Value Function......Page 240
11. The Fundamental Partial Differential Equation......Page 241
12. Discussion......Page 242
13. The Analytic Solution of an Example......Page 243
14. Discussion......Page 244
15. A Modification of an Earlier Problem......Page 245
17. A Poisson Process......Page 247
18. The Fundamental Partial Differential Equation for a Poisson Process......Page 248
20. A Numerical Problem......Page 249
21. The Appropriate Prior-Probability Density......Page 251
23. The Fundamental Recurrence Equation......Page 252
24. A Further Specialization......Page 254
25. Numerical Solution......Page 255
26. Discussion......Page 258
27. A Control Problem with Partially Observable States and with Deterministic Dynamics......Page 259
28. Discussion......Page 261
29. Sufficient Statistics......Page 262
31. A Warning......Page 263
32. Summary......Page 264
Bibliography......Page 265
Author Index......Page 268
Subject Index......Page 269