Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains

Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Tempora Domains......Page 5
Copyright......Page 6
Contents......Page 9
Preface......Page 17
1.1.1 System Models and Simulation......Page 21
1.2 Linear System Identification......Page 23
1.3 Nonlinear System Identification......Page 25
1.4 NARMAX Methods......Page 27
1.5 The NARMAX Philosophy......Page 28
1.6 What is System Identification For?......Page 29
1.7 Frequency Response of Nonlinear Systems......Page 31
1.8 Continuous-Time, Severely Nonlinear, and Time-Varying Models and Systems......Page 32
1.10 Using Nonlinear System Identification in Practice and Case Study Examples......Page 33
References......Page 34
2.1 Introduction......Page 37
2.2.1 Autoregressive Moving Average with Exogenous Input Model......Page 38
2.2.1.5 ARX Model......Page 39
2.2.2.1 ARX Model Parameter Estimation – The Least Squares Algorithm......Page 40
2.2.2.2 ARMAX Model Parameter Estimation – The Extended Least Squares Algorithm......Page 41
2.3 Piecewise Linear Models......Page 42
2.3.1 Spatial Piecewise Linear Models......Page 43
2.3.1.2 Parameter Estimation......Page 44
2.3.1.3 Simulation Example......Page 45
2.3.2.1 Decomposition of Signal-Dependent Models......Page 46
2.3.2.2 Parameter Estimation of Signal-Dependent Models......Page 47
2.3.2.3 Simulation Example......Page 48
2.3.3 Remarks on Piecewise Linear Models......Page 49
2.4 Volterra Series Models......Page 50
2.5 Block-Structured Models......Page 51
2.5.2 Feedback Block-Structured Models......Page 52
2.6 NARMAX Models......Page 53
2.6.1 Polynomial NARMAX Model......Page 55
2.6.2 Rational NARMAX Model......Page 57
2.6.2.3 Output-affine Model......Page 58
2.6.3 The Extended Model Set Representation......Page 59
2.7 Generalised Additive Models......Page 60
2.8.1 Multi-layer Networks......Page 61
2.8.2 Single-Layer Networks......Page 62
2.8.2.1 Activation Functions......Page 63
2.8.2.2 Radial Basis Function Networks......Page 64
2.9 Wavelet Models......Page 65
2.9.1.1 Random Noise......Page 66
2.9.1.2 Coloured Noise......Page 67
2.10 State-Space Models......Page 68
2.12 Noise Modelling......Page 69
2.12.3 Additive Coloured Noise......Page 70
2.12.4 General Noise......Page 71
2.13 Spatio-temporal Models......Page 72
References......Page 73
3.1 Introduction......Page 81
3.2.1 Linear-in-the-Parameters Representation......Page 84
3.2.3 The Basic OLS Estimator......Page 85
3.2.4 The Matrix Formulation of the OLS Estimator......Page 87
3.2.5 The Error Reduction Ratio......Page 88
3.2.6 An Illustrative Example of the Basic OLS Estimator......Page 89
3.3 The Forward Regression OLS Algorithm......Page 90
3.3.1 Forward Regression with OLS......Page 92
3.3.1.1 The FROLS Algorithm......Page 93
3.3.1.2 Variants of the FROLS Algorithm......Page 96
3.3.2 An Illustrative Example of Forward Regression with OLS......Page 97
3.3.3 The OLS Estimation Engine and Identification Procedure......Page 98
3.4 Term and Variable Selection......Page 99
3.5 OLS and Sum of Error Reduction Ratios......Page 100
3.5.2 The Variance of the s -Step-Ahead Prediction Error......Page 102
3.5.4 The Variable Selection Algorithm......Page 103
3.6.1 The Noise Model......Page 104
3.7 An Example of Variable and Term Selection for a Real Data Set......Page 107
3.8 ERR is Not Affected by Noise......Page 114
3.9 Common Structured Models to Accommodate Different Parameters......Page 115
3.10.2 Parameter-Dependent Model Structure......Page 118
3.10.3 Modelling Auxetic Foams – An Example of External Parameter-Dependent Model Identification......Page 119
3.11 OLS and Model Reduction......Page 120
References......Page 122
4.1 Introduction......Page 125
4.2 Feature Selection and Feature Extraction......Page 126
4.3 Principal Components Analysis......Page 127
4.4.1 The Basic Idea of the FOS-MOD Algorithm......Page 128
4.4.2 Feature Detection and Ranking......Page 129
4.4.3 Monitoring the Search Procedure......Page 131
4.4.4 Illustrative Examples......Page 132
4.5.1 Principal Component-Derived Multiple Regression......Page 133
4.5.2 PCA-Based MFROLS Algorithms......Page 134
4.5.3 An Illustrative Example......Page 135
References......Page 137
5.1 Introduction......Page 139
5.2 Detection of Nonlinearity......Page 141
5.3 Estimation and Test Data Sets......Page 143
5.4.1 One-Step-Ahead Prediction......Page 144
5.4.2 Model Predicted Output......Page 146
5.5 Statistical Validation......Page 147
5.5.1 Correlation Tests for Input–Output Models......Page 148
5.5.2 Correlation Tests for Time Series Models......Page 152
5.5.3 Correlation Tests for MIMO Models......Page 153
5.5.4 Output-Based Tests......Page 154
5.6 Term Clustering......Page 155
5.7 Qualitative Validation of Nonlinear Dynamic Models......Page 157
5.7.2 Bifurcation Diagrams......Page 159
5.7.4.1 Poincaré Maps for Model Validation......Page 160
5.7.4.2 Bifurcation Diagrams for Model Validation......Page 162
5.7.4.3 Poincaré Maps and Bifurcation Diagrams for Model Validation of Chaotic Systems......Page 163
References......Page 165
6.1 Introduction......Page 169
6.2 Generalised Frequency Response Functions......Page 171
6.2.1.1 The Volterra Series......Page 173
6.2.1.2 Volterra Series Models of Continuous- and Discrete-Time Nonlinear Systems......Page 174
6.2.2 Generalised Frequency Response Functions......Page 176
6.2.3.1 The System Time Domain Output Response Representation Using GFRFs......Page 177
6.2.3.2 The Relationship Between GFRFs and the System Frequency Domain Output Response......Page 179
6.2.4 Interpretation of the Composition of the Output Frequency Response of Nonlinear Systems......Page 182
6.2.5.1 Multi-dimensional Spectral Estimation Approaches......Page 185
6.2.5.2 Frequency-Domain Volterra System Identification Approaches......Page 186
6.2.5.3 Parametric Model-Based Approach......Page 187
6.2.6.1 Summary of the Parametric Method of Estimating GFRFs......Page 196
6.2.6.2 Case Study of a Real System......Page 197
6.3 Output Frequencies of Nonlinear Systems......Page 204
6.3.1 Output Frequencies of Nonlinear Systems under Multi-tone Inputs......Page 205
6.3.2 Output Frequencies of Nonlinear Systems for General Inputs......Page 207
6.4 Nonlinear Output Frequency Response Functions......Page 211
6.4.1 Definition and Properties of NOFRFs......Page 212
6.4.2 Evaluation of NOFRFs......Page 215
6.4.3.1 Basic Idea......Page 216
6.4.3.2 Damage Detection Procedure......Page 217
6.4.3.3 An Experimental Case Study......Page 218
6.5 Output Frequency Response Function of Nonlinear Systems......Page 222
6.5.2 Determination of the OFRF......Page 223
6.5.2.1 Determining the OFRF Structure......Page 224
6.5.2.2 Determining the OFRF `Coefficients’......Page 226
6.5.3 Application of the OFRF to Analysis of Nonlinear Damping for Vibration Control......Page 227
References......Page 233
7.1 Introduction......Page 237
7.2 Energy Transfer Filters......Page 238
7.2.1 The Time and Frequency Domain Representation of the NARX Model with Input Nonlinearity......Page 240
7.2.2.1 The Problem Description......Page 242
7.2.2.2 ETF Design for a Specified Input......Page 243
7.2.2.3 ETF Designs Using Orthogonal Least Squares......Page 252
7.2.2.4 ETF Design for Several Specified Inputs......Page 257
7.3 Energy Focus Filters......Page 260
7.3.1 Output Frequencies of Nonlinear Systems with Input Signal Energy Located in Two Separate Frequency Intervals......Page 261
7.3.2 The Energy Focus Filter Design Procedure and an Example......Page 265
7.4.1 OFRF -Based Design of Nonlinear Systems in the Frequency Domain......Page 269
7.4.1.1 General Procedure for the OFRF -Based Design of Nonlinear Systems in the Frequency Domain......Page 270
7.4.2.1 Experimental Setup......Page 271
7.4.2.2 Modelling the Experimental Vibration Isolation System......Page 274
7.4.2.3 The OFRF -Based Design for Nonlinear Damping......Page 276
References......Page 279
8.1 Introduction......Page 281
8.2 The Multi-layered Perceptron......Page 283
8.3 Radial Basis Function Networks......Page 284
8.3.2 Fixed Kernel Centres with a Single Width......Page 286
8.3.3 Limitation of RBF Networks with a Single Kernel Width......Page 288
8.3.4 Fixed Kernel Centres and Multiple Kernel Widths......Page 289
8.4 Wavelet Networks......Page 290
8.4.1 Wavelet Decompositions......Page 291
8.4.2 Wavelet Networks......Page 292
8.4.3 Limitations of Fixed Grid Wavelet Networks......Page 293
8.4.4.1 The Structure of the New Wavelet Networks......Page 294
8.4.4.3 Determining Significant Wavelet Terms......Page 295
8.4.4.4 A Procedure to Construct a Wavelet Network......Page 296
8.5.1 Multi-resolution Wavelet Decompositions......Page 297
8.5.2 Multi-resolution Wavelet Models and Networks......Page 300
8.5.3 An Illustrative Example......Page 302
References......Page 304
9.1 Introduction......Page 309
9.2 Wavelet NARMAX Models......Page 311
9.2.1 Nonlinear System Identification Using Wavelet Multi-resolution NARMAX Models......Page 312
9.2.2 A Strategy for Identifying Nonlinear Systems......Page 319
9.3.1 Limitations of the Volterra Series Representation......Page 321
9.3.2 Time Domain Analysis......Page 322
9.4.1 Introduction......Page 325
9.4.2 Examples of the Response Spectrum Map......Page 326
9.5 A Modelling Framework for Sub-harmonic and Severely Nonlinear Systems......Page 333
9.5.1 Input Signal Decomposition......Page 334
9.5.2 MISO NARX Modelling in the Time Domain......Page 337
9.5.2.1 A Simulation Example......Page 338
9.6.1 MISO Frequency Domain Volterra Representation......Page 340
9.6.2 Generating the GFRFs from the MISO model......Page 342
9.7.1 Frequency Domain Response Synthesis......Page 346
9.7.2 An Example of Frequency Domain Analysis for Sub-harmonic Systems......Page 352
References......Page 354
10.1 Introduction......Page 357
10.2.1 Definitions......Page 358
10.2.2 Reconstructing the Linear Model Terms......Page 362
10.2.3 Reconstructing the Quadratic Model Terms......Page 366
10.2.4 Model Structure Determination......Page 368
10.3.1 Introduction......Page 372
10.3.2 Reconstructing the Linear Model Terms......Page 375
10.3.3 Reconstructing the Quadratic Model Terms......Page 378
10.3.4 Reconstructing the Higher-Order Model Terms......Page 381
10.3.5 A Real Application......Page 384
References......Page 387
11.1 Introduction......Page 391
11.2.1 The Kalman Filter Algorithm......Page 392
11.2.2 The RLS and LMS Algorithms......Page 395
11.3.1 A General Form of TV-ARX Model Using Wavelets......Page 396
11.3.2 A Multi-wavelet Approach for Time-Varying Parameter Estimation......Page 397
11.4.1 The Definition of a Time-Dependent Spectral Function......Page 398
11.5 Nonlinear Time-Varying Model Estimation......Page 400
11.6.1 Time-Varying Frequency Response Functions......Page 401
11.6.2 First- and Second-Order TV-GFRFs......Page 402
11.7 A Sliding Window Approach......Page 408
References......Page 409
12.1 Introduction......Page 411
12.2.2 Discrete Lattice......Page 413
12.2.3 Neighbourhood......Page 414
12.2.4.1 Truth Table......Page 416
12.2.4.2 Boolean Function......Page 417
12.2.4.3 Totalistic Rule......Page 418
12.2.5 Simulation Examples of Cellular Automata......Page 419
12.3.1 Introduction and Review......Page 422
12.3.2 Polynomial Representation......Page 423
12.3.3.2 Neighbourhood Detection Based on the CA-OLS Algorithm......Page 425
12.3.3.3 Neighbourhood Detection Based on Mutual Information......Page 427
12.3.3.4 Rule Identification Based on a Coarse-to-Fine Approach......Page 430
12.4.1 Introduction to Excitable Media Systems......Page 434
12.4.2.1 The Greenberg-Hasting Model......Page 435
12.4.2.2 Hodgepodge Machine Model......Page 438
12.4.3.1 Neighbourhood Detection......Page 439
12.4.3.2 Rule Identification......Page 441
12.4.4.1 Introduction......Page 444
12.4.4.2 Identification of n-State Spatio-temporal Systems......Page 445
References......Page 447
13.1 Introduction......Page 451
13.2 Spatio-temporal Patterns and Continuous-State Models......Page 452
13.2.1 Stem Cell Colonies......Page 453
13.2.3 Oxygenation in Brain......Page 454
13.2.5 A Simulated Example Showing Spatio-temporal Chaos from CML Models......Page 455
13.3.1 Deterministic CML Models......Page 457
13.3.1.1 Deterministic CML State-Space Models......Page 458
13.3.1.2 Input–Output Representation of CMLs......Page 460
13.3.1.3 Polynomial Representation......Page 461
13.3.1.4 B-Spline Wavelet Representation......Page 462
13.3.2 The Identification of Stochastic CML Models......Page 474
13.4.1 Model Structure......Page 478
13.4.3.1 Approximation of the Nonlinear Function......Page 479
13.4.3.2 Finite Difference Schemes for Spatial Derivatives......Page 480
13.4.3.3 Dealing with the Boundary......Page 481
13.5 Nonlinear Frequency Response Functions for Spatio-temporal Systems......Page 486
13.5.1 A One-Dimensional Example......Page 487
13.5.2 Higher-Order Frequency Response Functions......Page 488
References......Page 491
14.1 Introduction......Page 493
14.2 Practical System Identification......Page 494
14.3.1 Door Traversal......Page 498
14.3.2 Route Learning......Page 502
14.4 System Identification for Space Weather and the Magnetosphere......Page 504
14.5 Detecting and Tracking Iceberg Calving in Greenland......Page 513
14.5.1 Causality Detection......Page 514
14.5.2 Results......Page 515
14.6 Detecting and Tracking Time-Varying Causality for EEG Data......Page 518
14.6.1 Data Acquisition......Page 519
14.6.2 Causality Detection......Page 520
14.6.3 Detecting Linearity and Nonlinearity......Page 524
14.7 The Identification and Analysis of Fly Photoreceptors......Page 525
14.7.1 Identification of the Fly Photoreceptor......Page 526
14.7.2 Model-Based System Analysis in the Time and Frequency Domain......Page 527
14.8 Real-Time Diffuse Optical Tomography Using RBF Reduced-Order Models of the Propagation of Light for Monitoring Brain Haemodynamics......Page 534
14.8.1.1 The Forward Problem......Page 535
14.8.1.2 Image Reconstruction......Page 536
14.8.2.1 Tomographic Reconstruction Algorithm Using Reduced-Order Forward Models......Page 537
14.8.2.3 Incorporating the Anatomical and Functional a priori Information......Page 538
14.8.2.5 Results......Page 539
14.9 Identification of Hysteresis Effects in Metal Rubber Damping Devices......Page 542
14.9.1 Dynamic Modelling of Metal Rubber Damping Devices......Page 543
14.9.2 Model Identification of a Metal Rubber Specimen......Page 546
14.10 Identification of the Belousov–Zhabotinsky Reaction......Page 548
14.10.1 Data Acquisition......Page 549
14.10.2.1 Chemical Oscillation Frequency......Page 550
14.10.2.3 Model Validation......Page 552
14.11 Dynamic Modelling of Synthetic Bioparts......Page 554
14.11.1 The Biopart and the Experiments......Page 555
14.11.2 NARMAX Model of the Synthetic Biopart......Page 556
14.12 Forecasting High Tides in the Venice Lagoon......Page 559
14.12.1 Time Series Forecasting Problem......Page 560
14.12.2.2 The Model......Page 561
14.12.2.3 Prediction Results......Page 562
References......Page 563
Index......Page 569
Supplemental Images......Page 576