Contents

1 Introduction 1

1.1 Optimal Control 1

1.1.1 Continuous-Time LQR 1

1.1.2 Discrete-Time LQR 2

1.2 Adaptive Dynamic Programming 3

1.3 Review of Matrix Algebra 5

References 6

2 Neural-Network-BasedApproach for Finite-TimeOptimal Control 7

2.1 Introduction 7

2.2 Problem Formulation and Motivation 9

2.3 The Data-Based Identifier 9

2.4 Derivation of the Iterative ADP Algorithm with Convergence Analysis 11

2.5 Neural Network Implementation of theIterative Control Algorithm 17

2.6 Simulation Study 18

2.7 Conclusion 20

References 22

3 Nearly Finite-HorizonOptimalControlfor Nonafiine Time-Delay Nonlinear Systems 25

3.1 Introduction 25

3.2 Problem Statement 26

3.3 The Iteration ADP Algorithm and ItsConvergence 30

3.3.1 The Novel ADP Iteration Algorithm 30

3.3.2 Convergence Analysis of the Improved Iteration Algorithm 33

3.3.3 Neural Network Implementation of the Iteration ADP Algorithm 38

3.4 Simulation Study 40

3.5 Conclusion 48

References 48

4 Multi-objective Optimal Control for Time-Delay Systems 49

4.1 Introduction 49

4.2 Problem Formulation 50

4.3 Derivation of the ADP Algorithm for Time-Delay Systems 51

4.4 Neural Network Implementation for the Multi-objective Optimal Control Problem of Time-Delay Systems 54

4.5 Simulation Study 55

4.6 Conclusion 61

References 62

5 Multiple Actor-Critic Optimal Control via ADP 63

5.1 Introduction 63

5.2 Problem Statement 65

5.3 SIANN Architecture-Based Classification 66

5.4 Optimal Control Based on ADP 69

5.4.1 Model Neural Network 70

5.4.2 Critic Network and Action Network 74

5.5 Simulation Study 82

5.6 Conclusion 91

References 91

6 Optimal Control for a Class of Complex-Valued Nonlinear Systems 95

6.1 Introduction 95

6.2 Motivations and Preliminaries 96

6.3 ADP-Based Optimal Control Design 99

6.3.1 Critic Network 99

6.3.2 Action Network. 101

6.3.3 Design of the Compensation Controller 102

6.3.4 Stability Analysis 103

6.4 Simulation Study 107

6.5 Conclusion. 110

References 110

7 Off-Policy Neuro-Optimal Control for Unknown Complex-Valued Nonlinear Systems 113

7.1 Introduction 113

7.2 Problem Statement 114

7.3 Off-Policy Optimal Control Method 115

7.3.1 Convergence Analysis of Off-Policy PI Algorithm 117

7.3.2 Implementation Method of Off-Policy Iteration Algorithm 119

7.3.3 Implementation Process 122

7.4 Simulation Study 122

7.5 Conclusion 125

References 125

8 Approximation-Error-ADP-Based Optimal Tracking Control for Chaotic Systems 127

8.1 Introduction 127

8.2 Problem Formulation and Preliminaries 128

8.3 Optimal Tracking Control Scheme Basedon Approximation-Error ADP Algorithm 130

8.3.1 Description of Approximation-Error ADP Algorithm 130

8.3.2 Convergence Analysis of the Iterative ADP Algorithm 132

8.4 Simulation Study 136

8.5 Conclusion 144

References 144

9 Off-Policy Actor-Critic Structure for Optimal Controlof Unknown Systems with Disturbances 147

9.1 Introduction 147

9.2 Problem Statement 148

9.3 Off-Policy Actor-Critic Integral Reinforcement Learning 151

9.3.1 On-Policy IRL for Nonzero Disturbance 151

9.3.2 Off-Policy IRL for Nonzero Disturbance 152

9.3.3 NN Approximation for Actor-Critic Structure 154

9.4 Disturbance Compensation Redesign andStability Analysis 157

9.4.1 Disturbance Compensation Off-Policy Controller Design 157

9.4.2 Stability Analysis 158

9.5 Simulation Study 161

9.6 Conclusion 163

References 163

10 An Iterative ADP Method to Solve for a Class of Nonlinear Zero-Sum DifferentialGames 165

10.1 Introduction 165

10.2 Preliminaries and Assumptions 166

10.3 Iterative Approximate Dynamic Programming Method for ZS Differential Games 169

10.3.1 Derivation of the Iterative ADP Method 169

10.3.2 The Procedure of theMethod 174

10.3.3 The Properties of theIterativeADP Method 176

10.4 Neural Network Implementation 190

10.4.1 The Model Network 191

10.4.2 The Critic Network 192

10.4.3 The Action Network 193

10.5 Simulation Study 195

10.6 Conclusion 204

References 204

11 Neural-Network-Based Synchronous Iteration Learning Method for Multi-player Zero-Sum Games 207

11.1 Introduction 207

11.2 Motivations and Preliminaries 208

11.3 Synchronous Solution of Multi-playerZSGames 213

11.3.1 Derivation of Off-Policy Algorithm 213

11.3.2 Implementation Method for Off-Policy Algorithm 214

11.3.3 Stability Analysis 218

11.4 Simulation Study 219

11.5 Conclusion 224

References 224

12 Off-Policy Integral Reinforcement Learning Method for Multi-player Non-Zero-Sum Games 227

12.1 Introduction 227

12.2 Problem Statement 228

12.3 Multi-player Learning PI SolutionforNZSGames 229

12.4 Off-Policy Integral ReinforcementLearningMethod 234

12.4.1 Derivation of Off-Policy Algorithm 234

12.4.2 Implementation Method for Off-Policy Algorith

本书系统地介绍了基于自适应学习控制的单控制器与多控制器系统最优控制方法。分别针对单控制器系统与多控制器系统，提出了一系列自适应动态规划方法获得系统的最优控制策略。全书共分为11章，第1、2章主要讲述自适应动态规划方法的基本原理以及实现方式。第3~7章主要介绍单控制器非线性系统迭代自适应动态规划最优控制方法以及性能分析方案。第9~11章主要介绍多控制器系统迭代自适应动态规划最优控制方法以及性能分析方案。

JMB/C流量型控制器，采用先进的微电脑芯片技术，专门用于水处理设备的流量型再生的控制。该控制器可用于由多阀或多路阀组成的多罐过滤、软化、除盐等系统的独立控制，具有瞬时流量、剩余量和累积水量显示、再生程序预设定、系统运行状态显示与其他控制设备连锁控制等特殊功能。

控制器性能特性

1.l 设置密码保护

2.l JMB单通道或JMC1-3路多通道流量计脉冲接收

3.l 流量计探头K值选择（根据探头安装管道尺寸按附表查询）

4.l 系统流量显示（0-999t/h）及预先设定批量（1-9999.9t）的剩余量递减显示

5.l 累计流量查询及清零功能

6.l 各路系统（工作、再生、备用、反馈）状态显示

7.l 本地强制手动启动再生

8.l 再生启动信号输出时间（2-9999秒）设定

9.l 系统工作模式：JMC（C1、D2、D3、E2、E3）五种选择

JMB（C1、C2、C3、C4、C5、D2、D3、D4、D5）九种选择

注：C型：同时运行轮流再生；D型：多用一备；E型：同时运行分别再生

1.l 再生系统（出水电磁阀控制）辅助接点输出

2.l 停电保护功能：停电后设定的参数保持时间大于3年

3.l 仪表外壳采用工程塑料，具有防尘、防腐、防水溅射等功能

4.l 仪表采用墙壁挂装或支架悬挂安装形式

主要技术指标

1.l 流量传感器脉冲量输入：正弦波频率范围0-10K，探头（直流）电压幅度4-12V

2.l 采样周期：0.2秒

3.l 输出形式：继电器无源接点输出 触点容量，AC220V/2A

4.l 馈电输入：无源接点反馈信号

5.l 显示方式：背光液晶显示单元，使用寿命≥10年

6.l 仪表电源：开关电源 100-265V/AC 功耗5W

7.l 使用环境：环境温度4-60℃ 相对湿度 ≤85%RH

8.l 控制器外壳尺寸：260X170X105mm

JMA--JMC连接示意图

BACK TO TOP