This book intends to report new optimal control results with critic intelligence for complex discrete-time systems, which covers the novel control theory, advanced control methods, and typical applications for wastewater treatment systems. Therein, combining with artificial intelligence techniques, such as neural networks and reinforcement learning, the novel intelligent critic control theory as well as a series of advanced optimal regulation and trajectory tracking strategies are established for discrete-time nonlinear systems, followed by application verifications to complex wastewater treatment processes. Consequently, developing such kind of critic intelligence approaches is of great significance for nonlinear optimization and wastewater recycling. The book is likely to be of interest to researchers and practitioners as well as graduate students in automation, computer science, and process industry who wish to learn core principles, methods, algorithms, and applications in the field of intelligent optimal control. It is beneficial to promote the development of intelligent optimal control approaches and the construction of high-level intelligent systems.
Author(s): Ding Wang, Mingming Ha, Mingming Zhao
Series: Intelligent Control and Learning Systems, 6
Publisher: Springer
Year: 2023
Language: English
Pages: 282
City: Singapore
Preface
Acknowledgements
Contents
1 On the Critic Intelligence for Discrete-Time Advanced Optimal Control Design
1.1 Introduction
1.2 Discrete-Time Optimal Regulation Design
1.3 Discrete-Time Trajectory Tracking Design
1.4 The Construction of Critic Intelligence
1.4.1 Basis of Reinforcement Learning
1.4.2 The Neural Network Approximator
1.4.3 The Critic Intelligence Framework
1.5 The Iterative Adaptive Critic Formulation
1.6 Significance and Prospects
References
2 Event-Triggered Adaptive Optimal Regulation of Constrained Affine Systems
2.1 Introduction
2.2 Problem Description
2.3 Stability Analysis of the Event-Triggered Control System
2.4 Event-Triggered Constrained Control Design via HDP
2.4.1 The Proposed Control Structure
2.4.2 Neural Network Implementation
2.4.3 The Whole Design Procedure
2.5 Simulation Experiments
2.6 Conclusions
References
3 Self-Learning Optimal Regulation with Event-Driven Iterative Adaptive Critic
3.1 Introduction
3.2 Problem Description
3.3 Event-Driven Iterative Adaptive Critic Design via DHP
3.3.1 Derivation and Convergence Discussion
3.3.2 Neural Network Implementation
3.4 Event-Based System Stability Analysis
3.5 Special Discussion of the Affine Nonlinear Case
3.6 Highlighting the Mixed-driven Control Framework
3.7 Simulation Experiments
3.8 Conclusions
References
4 Near-Optimal Regulation with Asymmetric Constraints via Generalized Value Iteration
4.1 Introduction
4.2 Problem Statement
4.3 Properties of the Generalized Value Iteration Algorithm …
4.3.1 Derivation of the Generalized Value Iteration Algorithm
4.3.2 Properties of the Generalized Value Iteration Algorithm
4.3.3 Implementation of the Generalized Value Iteration Algorithm
4.4 Simulation Studies
4.5 Conclusion
References
5 Nonaffine Neuro-Optimal Tracking Control with Accuracy and Stability Guarantee
5.1 Introduction
5.2 Problem Formulation
5.3 Neuro-Optimal Tracking Control Based on the Value Iteration Algorithm
5.3.1 The Value Iteration Algorithm for Tracking Control
5.3.2 Convergence of the Value Iteration Algorithm
5.3.3 Closed-Loop Stability Analysis with Value Iteration
5.4 Neural Network Implementation of the Iterative HDP Algorithm
5.4.1 The Critic Network
5.4.2 The Action Network
5.5 Simulation Experiments
5.5.1 Example 1
5.5.2 Example 2
5.6 Conclusions
References
6 Data-Driven Optimal Trajectory Tracking via a Novel Self-Learning Approach
6.1 Introduction
6.2 Problem Description
6.3 The Optimal Tracking Control Based on the Iterative DHP Algorithm
6.3.1 Derivation of the Iterative ADP Algorithm
6.3.2 Derivation of the Iterative DHP Algorithm
6.4 Data-Based Iterative DHP Implementation
6.4.1 Neuro-Identifier for Estimation of Nonlinear Dynamics
6.4.2 The Critic Network
6.4.3 The Action Network
6.5 Simulation Studies
6.6 Conclusion
References
7 Adaptive Critic with Improved Cost for Discounted Tracking and Novel Stability Proof
7.1 Introduction
7.2 Problem Formulation and VI-Based Adaptive Critic Scheme
7.3 Novel Stability Analysis of VI-Based Adaptive Critic Designs
7.4 Discounted Tracking Control for the Special Case of Linear Systems
7.5 Simulation Studies
7.5.1 Example 1
7.5.2 Example 2
7.6 Conclusions
References
8 Iterative Adaptive Critic Control Towards an Urban Wastewater Treatment Plant
8.1 Introduction
8.2 Platform Description with Control Problem Statement
8.2.1 Platform Description
8.2.2 Control Problem Statement
8.3 The Data-Driven IAC Control Method
8.4 Application to the Proposed Wastewater Treatment Plant
8.5 Revisiting Wastewater Treatment via Mixed Driven NDP
8.6 Conclusions
References
9 Constrained Neural Optimal Tracking Control with Wastewater Treatment Applications
9.1 Introduction
9.2 Problem Statement with Asymmetric Control Constraints
9.3 Intelligent Optimal Tracking Design
9.3.1 Description of the Iterative ADP Algorithm
9.3.2 DHP Formulation of the Iterative Algorithm
9.4 Neural Network Implementation
9.5 A Wastewater Treatment Application
9.6 Conclusion
References
10 Data-Driven Hybrid Intelligent Optimal Tracking Design with Industrial Applications
10.1 Introduction
10.2 Problem Statement
10.3 Offline Learning of the Pre-designed Controller
10.4 Online Near-Optimal Tracking Control with Stability Analysis
10.4.1 The Critic Network
10.4.2 The Action Network
10.4.3 Uniformly Ultimately Bounded Stability of Weight Estimation Errors
10.4.4 Summary of the Proposed Tracking Approach
10.5 Experimental Simulation
10.5.1 Application to a Torsional Pendulum Device
10.5.2 Application to a Wastewater Treatment Plant
10.6 Concluding Remarks
References
Appendix Index
Index
