The concept of a system as an entity in its own right has emerged with increasing force in the past few decades in, for example, the areas of electrical and control engineering, economics, ecology, urban structures, automaton theory, operational research and industry. The more definite concept of a large-scale system is implicit in these applications, but is particularly evident in fields such as the study of communication networks, computer networks and neural networks. The Wiley-Interscience Series in Systems and Optimization has been established to serve the needs of researchers in these rapidly developing fields. It is intended for works concerned with developments in quantitative systems theory, applications of such theory in areas of interest, or associated methodology. This is the first book-length treatment of risk-sensitive control, with many new results. The quadratic cost function of the standard LQG (linear/quadratic/Gaussian) treatment is replaced by the exponential of a quadratic, giving the so-called LEQG formulation allowing for a degree of optimism or pessimism on the part of the optimiser. The author is the first to achieve formulation and proof of risk-sensitive versions of the certainty-equivalence and separation principles. Further analysis allows one to formulate the optimization as the extremization of a path integral and to characterize the solution in terms of canonical factorization. It is thus possible to achieve the long-sought goal of an operational stochastic maximum principle, valid for a higher-order model, and in fact only evident when the models are extended to the risk-sensitive class. Additional results include deduction of compact relations between value functions and canonical factors, the exploitation of the equivalence between policy improvement and Newton Raphson methods and the direct relation of LEQG methods to the H??? and minimum-entropy methods. This book will prove essential reading for all graduate students, researchers and practitioners who have an interest in control theory including mathematicians, engineers, economists, physicists and psychologists. 1990 Stochastic Programming Peter Kall, University of Zurich, Switzerland and Stein W. Wallace, University of Trondheim, Norway Stochastic Programming is the first textbook to provide a thorough and self-contained introduction to the subject. Carefully written to cover all necessary background material from both linear and non-linear programming, as well as probability theory, the book draws together the methods and techniques previously described in disparate sources. After introducing the terms and modelling issues when randomness is introduced in a deterministic mathematical programming model, the authors cover decision trees and dynamic programming, recourse problems, probabilistic constraints, preprocessing and network problems. Exercises are provided at the end of each chapter. Throughout, the emphasis is on the appropriate use of the techniques, rather than on the underlying mathematical proofs and theories, making the book ideal for researchers and students in mathematical programming and operations research who wish to develop their skills in stochastic programming. 1994
Author(s): Whittle P.
Publisher: Wiley
Year: 1996
Language: English
Pages: 476
Tags: Автоматизация;Теория автоматического управления (ТАУ);Книги на иностранных языках;
Title......Page 0
Optimal Control: Basics and Beyond......Page 4
Contents......Page 6
Preface......Page 8
1 CONTROL AS AN OPTIMISATION PROBLEM......Page 12
2 AN EXAMPLE: THE HARVESTING OF A RENEWABLE RESOURCE......Page 13
3 DYNAMIC OPTIMISATION TECHNIQUES......Page 16
4 ORGANISATION OF THE TEXT......Page 18
PART 1: Deterministic Models......Page 20
1 STATE STRUCTURE OPTIMISATION AND DYNAMICPROGRAMMING......Page 22
2 OPTIMAL CONSUMPTION......Page 28
3 PRODUCTION SCHEDULING......Page 31
4 LQ REGULATION......Page 33
5 LQ REGULATION IN THE SCALAR CASE......Page 36
6 DYNAMIC PROGRAMMING IN CONTINUOUS TIME......Page 39
7. OPTIMAL HARVESTING......Page 42
8 LQ REGULATION IN CONTINUOUS TIME......Page 44
9 OPTIMAL TRACKING FOR A DISTURBED LQ MODEL......Page 49
10 OPTIMAL EQUILIBRIUM POINTS: NEIGHBOURING OPTIMALCONTROL......Page 53
1 A FORMALISM FOR THE DYNAMIC PROGRAMMING EQUATION......Page 58
2 INFINITE-HORIZON LIMITS FOR TOTAL COST......Page 60
3 AVERAGE-COST OPTIMALITY......Page 64
4 GROWTH OPTIMALITY......Page 66
5 POLICY IMPROVEMENT......Page 67
6 POLICY IMPROVEMENT AND LQ MODELS......Page 70
7. POLICY IMPROVEMENT FOR THE HARVESTING EXAMPLE......Page 72
1 THE CLASSIC CONTROL FORMULATION......Page 74
2 FILTERS IN DISCRETE TIME: THE SCALAR (SISO) CASE......Page 78
3 FROM DYNAMIC MODEL TO THE INPUT/OUTPUT RELATION:FILTER INVERSION......Page 83
4 FILTERS IN DISCRETE TIME; THE VECTOR (MIMO) CASE......Page 85
5 COMPOSITION AND INVERSION OF FILTERS; z-TRANSFORMS......Page 88
6 FILTERS IN CONTINUOUS TIME......Page 90
7 DYNAMIC MODELS: THE INVERSION OF CONTINUOUS-TIMEFILTERS......Page 91
8 LAPLACE TRANSFORMS......Page 93
9 STABILITY OF CONTINUOUS-TIME FILTERS......Page 94
10 SYSTEMS STABILISED BY FEEDBACK......Page 96
1 STATE-STRUCTURE FOR THE UNCONTROLLED CASE: STABILTY;LINEARISATION......Page 102
2 CONTROL......Page 107
3 THE CAYLEY-HAMILTON THEOREM......Page 110
4 CONTROLLABILITY (DISCRETE TIME)......Page 112
5 CONTROLLABLITY (CONTINUOUS TIME)......Page 115
6 OBSERVABILITY......Page 117
1 INFINITE-HORIZON LIMITS FOR THE LQ REGULATION PROBLEM......Page 122
2. STATIONARY TRACKING RULES......Page 126
3 DIRECT TRAJECTORY OPTIMISATION: WHY THE OPTIMAL FEEDBACK/FEEDFORWARD CONTROL RULE HAS THE FORMIT DOES......Page 127
4 AN ALTERNATIVE FORM FOR THE RICCATI RECURSION......Page 132
5 DIRECT TRAJECTORY OPTIMISATION FOR HIGHER-ORDERCASES......Page 133
6 THE CONCLUSIONS IN TRANSFER FUNCTION TERMS......Page 134
7 DIRECT TRAJECTORY OPTIMISATION FOR THE INPUTOUTPUTFORMULATION: NOTES OF CAUTION......Page 137
CHAPTER 7: The Pontryagin Maximum Principle......Page 142
1 THE PRINCIPLE AS A DIRECT LAGRANGIAN OPTIMISATION......Page 143
2 THE PONTRYAGIN MAXIMUM PRINCIPLE......Page 146
3 TERMINAL CONDITIONS......Page 149
4 MINIMAL TIME AND DISTANCE PROBLEMS......Page 151
5 SOME MISCELLANEOUS PROBLEMS......Page 153
6 THE BUSH PROBLEM......Page 156
7 ROCKET THRUST PROGRAMME OPTIMISATION......Page 158
8 PROBLEMS WHICH ARE PARTIALLY LQ......Page 161
9 CONTROL OF THE INERTIALESS PARTICLE......Page 164
10 CONTROL OF THE INERTIAL PARTICLE......Page 166
11 AVOIDANCE OF THE SfOPPING SEf: A GENERAL RESULT......Page 171
12 CONSTRAINED PATHS: TRANSVERSALITY CONDITIONS......Page 174
13 REGULATION OF A RESERVOIR......Page 176
PART 2: Stochastic Models......Page 178
CHAPTER 8: Stochastic Dynamic Programming......Page 180
1 ONE-STAGE OPTIMISATION......Page 181
2 MULTI-STAGE OPTIMISATION; THE DYNAMIC PROGRAMMINGEQUATION......Page 184
3 STATE STRUCTURE......Page 186
4 THE DYNAMIC PROGRAMMING EQUATION IN CONTINUOUS TIME......Page 188
1 JUMP PROCESSES......Page 190
2 DETERMINISTIC AND PIECEWISE DETERMINISTIC PROCESSES......Page 191
3 THE DERIVATE CHARACTERISTIC FUNCTION......Page 192
4 PROCESSES OF INDEPENDENT INCREMENTS......Page 193
5 WIENER PROCESSES (BROWNIAN MOTION)......Page 196
1 LQG REGULATION WITH PLANT NOISE......Page 200
2 OPTIMAL EXERCISE OF A STOCK OPTION......Page 203
3 A QUEUEING MODEL......Page 204
4 THE HARVESTING EXAMPLE: A BIRTH-DEATH MODEL......Page 205
5 BEHAVIOUR FOR NEAR-DETERMINISTIC MODELS......Page 208
6 THE UNDERSTANDING OF THRESHOLD CHOICE......Page 210
7 CONTINUOUS-TIME LQG PROCESSES AND PASSAGE TO ASTOPPING SET......Page 212
8 THE HARVESTING EXAMPLE: A DIFFUSION MODEL......Page 217
9 THE HARVESTING EXAMPLE: A MODEL WITH A SWITCHINGENVIRONMENT......Page 220
1 THE TRANSFER OF DETERMINISTIC CONCLUSIONS......Page 226
2 AVERAGE-COST OPTIMALITY......Page 228
3 POLICY IMPROVEMENT FOR THE AVERAGE-COST CASE......Page 231
4 MACHINE MAINTENANCE......Page 232
5 CUSTOMER ALLOCATION BETWEEN QUEUES......Page 233
6 ALLOCATION OF SERVER EFFORT BETWEEN QUEUES......Page 234
7 REWARDED SERVICE RATHER THAN PENALISED WAITING......Page 236
8 CALL ROUTING IN LOSS NETWORKS......Page 237
1 AN INDICATION OF CONCLUSIONS......Page 240
2 LQG STRUCTURE AND IMPERFECT OBSERVATION......Page 242
3 THE CERTAINTY EQUIVALENCE PRINCIPLE......Page 245
4 APPLICATIONS: THE EXTENDED CEP......Page 248
5 CALCULATION OF ESTIMATES: THE KALMAN FILTER......Page 250
6 PROJECTION ESTIMATES......Page 253
7 INNOVATIONS......Page 257
9 ESTIMATION AS DUAL TO CONTROL......Page 259
1 STATIONARITY: THE AUTOCOVARIANCE FUNCTION......Page 266
2 THE ACTION OF FILTERS: THE SPECTRAL DENSITY MATRIX......Page 268
3 MOVING-AVERAGE AND AUTOREGRESSIVE REPRESENTATIONS:CANONICAL FACTORISATIONS......Page 272
4 THE WIENER FILTER......Page 275
5 OPTIMISATION CRITERIA IN THE STEADY STATE......Page 276
1 ALLOCATION AND CONTROL......Page 280
2 THE GITTINS INDEX......Page 281
3 OPTIMALITY OF THE GITTINS INDEX POLICY......Page 283
4 EXAMPLES......Page 286
5 RESTLESS BANDITS......Page 288
6 AN EXAMPLE: MACHINE MAINTENANCE......Page 292
7 QUEUEING PROBLEMS......Page 293
CHAPTER 15: Imperfect State Observation......Page 296
1 SUFFICIENCY OF THE POSTERIOR DISTRIBUTION OF STATE......Page 297
2 EXAMPLES: MACHINE SERVICING AND CHANGE-POINTDETECTION......Page 300
PART 3: Risk-Sensitive and H_infinity Criteria......Page 304
1 UTILITY AND RISK-SENSITIVITY......Page 306
2 THE RISK-SENSITIVE CERTAINTY-EQUIVALENCE PRINCIPLE......Page 309
3 SOME SIMPLE EXAMPLES......Page 313
4 THE CASE OF PERFECT STATE OBSERVATION......Page 315
5 THE DISTURBED (NON-HOMOGENEOUS) CASE......Page 318
6 IMPERFECT STATE OBSERVATION: THE SOLUTION OF THEP-RECURSION......Page 319
8 CONTINUOUS TIME......Page 321
9 SOME CONTINUOUS-TIME EXAMPLES......Page 323
10 AVERAGE COST......Page 325
11 TIME-INTEGRAL METHODS FOR THE LEQG MODEL......Page 327
12 WHY DISCOUNT?......Page 328
1 THE Hoo LIMIT......Page 332
2 CHARACTERISTICS OF THE Hoo NORM......Page 335
3 THE Hoo CRITERION AND ROBUSTNESS......Page 337
PART 4: Time-integral Methods and Optimal Stationary Policies......Page 342
1 QUADRATIC INTEGRALS IN DISCRETE TIME......Page 346
2 FACTORISATION AND RECURSION IN THE 'MARKOV' CASE......Page 348
3 VALUE FUNCTIONS AND RECURSIONS......Page 350
4 A CENTRAL RESULT: THE RICCATI/FACTORISATIONRELATIONSHIP......Page 352
5 POLICY IMPROVEMENT AND SUCCESSIVE APPROXIMATION TOTHE CANONICAL FACTORISATION......Page 353
6 THE INTEGRAL FORMALISM IN CONTINUOUS TIME......Page 355
7 RECURSIONS AND FACTORISATIONS IN CONTINUOUS TIME......Page 357
8 THE PIINR ALGORITHM IN CONTINUOUS TIME......Page 359
1 DERIVATION OF THE SOLUTION FROM THE TIME-INTEGRAL......Page 360
2 EXPRESSION OF THE OPTIMAL CONTROL RULE IN DISCRETETIME......Page 362
3 EXPRESSION OF THE OPTIMAL CONTROL RULE INCONTINUOUS TIME......Page 364
1 THE PROCESS/OBSERVATION MODEL: APPEAL TO CERTAINTY EQUIVALENCE......Page 368
2 PROCESS ESTIMATION IN TIME-INTEGRAL FORM(DISCRETE TIME)......Page 369
3 THE PARALLEL KALMAN FILTER (DISCRETE TIME)......Page 372
4 THE SERIAL KALMAN FILTER (DISCRETE TIME)......Page 374
5 THE CONTINUOUS-TIME CASE; INVALIDITY OF THE SERIAL FILTER......Page 376
6 THE PARALLEL KALMAN FILTER (CONTINUOUS TIME)......Page 377
1 DEDUCTION OF THE TIME-INTEGRAL......Page 382
2 THE STATIONARITY CONDITIONS......Page 384
3 A GENERAL FORMALISM......Page 386
PART 5: Near-determinism and LargeDeviation Theory......Page 390
1 THE LARGE DEVIATION PROPERTY......Page 394
2 THE STATIC CASE: CRAMER'S THEOREM......Page 395
3 OPERATORS AND SCALING FOR MARKOV PROCESSES......Page 400
4 THE RATE FUNCTION FOR A MARKOV PROCESS......Page 403
5 HAMILTONIAN ASPECTS......Page 406
6 REFINEMENTS OF THE LARGE DEVIATION PRINCIPLE......Page 408
7 EQUILIBRIUM DISTRIBUTIONS AND EXPECTED EXIT TIMES......Page 410
1 SPECIFICATION AND THE RISK-NEUTRAL CASE......Page 416
2 LARGE DEVIATION CONCLUSIONS IN THE RISK-SENSITIVECASE......Page 417
3 AN EXAMPLE: THE INERTIALESS LANDING PROBLEM......Page 419
4 THE OPTIMISATION OF CONSUMPTION OVER A LIFETIME......Page 422
5 A NON-DIFFUSION EXAMPLE: THE REGULATION OF PARTICLENUMBERS......Page 424
1 REDUCTION OF THE CONTROL PROBLEM......Page 426
2 THE LARGE DEVIATION EVALUATION......Page 429
3 THE CASE OF LINEAR DYNAMICS......Page 430
4 LANDING THE INERTIALESS PARTICLE......Page 432
5 OBSTACLE AVOIDANCE FOR mE INERTIALESS PARTICLE......Page 433
6 CRASH AVOIDANCE FOR THE INERTIAL PARTICLE......Page 437
7 THE AVOIDANCE OF ABSORPTION GENERALLY......Page 439
1 SPECIFICATION OF THE PLANT/OBSERVATION PROCESS......Page 442
2 'CERTAINTY EQUIVALENCE' AND THE SEPARATION PRINCIPLE......Page 443
3 THE RISK-SENSITIVE MAXIMUM PRINCIPLE......Page 446
5 PURE ESTIMATION: NON-LINEAR FILTERING......Page 448
APPENDIX 1: Notation and Conventions......Page 454
APPENDIX 2: The Structural Basis of Temporal Optimisation......Page 460
APPENDIX 3: Moment Generating Functions; Basic Properties......Page 466
Reforences......Page 468
Index......Page 472
Back Cover......Page 476
