Computational intelligence paradigms : theory & applications using MATLAB

COMPUTATIONAL INTELLIGENCE PARADIGMS: THEORY AND APPLICATIONS USING MATLAB®......Page 2
Contents......Page 4
About This Book......Page 16
Organization of the Book......Page 17
About the Authors......Page 20
Acknowledgment......Page 21
1.1 Introduction......Page 23
1.2.1 Optimization......Page 27
Verifier-Based Definition......Page 28
1.3 Neural Networks......Page 29
1.3.1 Feed Forward Neural Networks......Page 30
1.4.1 Fuzzy Sets......Page 31
1.4.2 Fuzzy Controllers......Page 33
1.5 Evolutionary Computing......Page 34
1.5.1 Genetic Algorithms......Page 35
1.5.2 Genetic Programming......Page 36
1.5.4 Evolutionary Strategies......Page 37
1.6 Swarm Intelligence......Page 38
1.7 Other Paradigms......Page 39
1.7.1 Granular Computing......Page 40
1.7.2 Chaos Theory......Page 42
1.7.3 Artificial Immune Systems......Page 43
1.8 Hybrid Approaches......Page 44
1.9 Relationship with Other Paradigms......Page 45
1.10 Challenges To Computational Intelligence......Page 47
Summary......Page 48
Review Questions......Page 49
2.2 A Brief History of Neural Networks......Page 51
2.3 Artificial Neural Networks......Page 54
2.3.2 Artificial Neurons......Page 55
2.3.3 Implementation of Artificial Neuron Electronically......Page 57
2.3.4 Operations of Artificial Neural Network......Page 59
2.3.5 Training an Artificial Neural Network......Page 62
Supervised Training......Page 63
Unsupervised or Adaptive Training......Page 64
2.3.6 Comparison between Neural Networks, Traditional Computing, and Expert Systems......Page 66
2.4 Neural Network Components......Page 68
Supervised Learning......Page 74
Unsupervised Learning......Page 75
2.4.2 Learning Rates......Page 76
Hebb’s Rule......Page 77
Delta Rule for Single Output Unit......Page 78
Extended Delta Rule......Page 79
Competitive Learning Law or Kohonen’s Learning Law......Page 80
Boltzmann Learning Law......Page 82
MATLAB Snippet to Implement Hebb Rule Syntax......Page 83
Review Questions......Page 89
3.1 Introduction......Page 91
3.2.1 Single-Layer Networks......Page 93
3.2.2 Multilayer Networks......Page 95
Architecture......Page 97
Algorithm......Page 98
Training Algorithm......Page 99
Perceptron Training Algorithm for Several Output Classes......Page 100
3.3.2 MATLAB Implementation of a Perceptron Network......Page 101
Architecture......Page 105
Training Algorithm......Page 107
Parameters......Page 108
Choice of Parameters......Page 110
Number of Training Pairs......Page 111
Momentum Factor......Page 112
3.3.4 Implementation of BPN Using MATLAB......Page 113
Limitations of Back Propagation Network......Page 116
3.3.5 Delta Bar Delta Network......Page 117
Learning Rule of the Delta Bar Delta Network......Page 118
3.3.6 Extended Delta Bar Delta......Page 119
3.3.7 Directed Random Search Network......Page 122
3.3.8 Functional Link Artificial Neural Network (FLANN) or Higher-Order Neural Network......Page 124
Architecture......Page 125
Review Questions......Page 128
4.2 Neural Networks Based on Classification......Page 130
4.2.1 Learning Vector Quantization......Page 131
Architecture......Page 133
Training Algorithm......Page 134
LVQ3......Page 135
4.2.2 Implementation of LVQ in MATLAB......Page 136
4.2.3 Counter-Propagation Network......Page 137
Training Phases of Full CPN......Page 140
Training Algorithm......Page 141
Initializing the Weight Vectors......Page 143
4.2.4 Probabilistic Neural Network......Page 144
Training Algorithm......Page 147
Application Algorithm......Page 148
4.2.5 Implementation of the Probabilistic Neural Net Using MATLAB......Page 149
4.3 Data Association Networks......Page 150
4.3.1 Hopfield Network......Page 151
Training Algorithm......Page 152
Application Algorithm......Page 153
4.3.2 Implementation of Hopfield Network in MATLAB......Page 154
4.3.3 Boltzmann Machine......Page 155
Architecture......Page 156
Application Algorithm......Page 157
4.3.4 Hamming Network......Page 158
Architecture......Page 160
Application Procedure......Page 161
FIGURE 4.11: Bi-Directional Associative Memory Example......Page 162
Architecture......Page 163
Activation Function......Page 164
Continuous BAM......Page 165
Application Algorithm......Page 166
Background......Page 167
ART1: The Simplified Neural Network Model......Page 168
Supplemental Units......Page 169
Training Algorithm......Page 170
4.4.2 Implementation of ART Algorithm in MATLAB......Page 172
4.4.3 Self-Organizing Map......Page 174
Architecture......Page 177
Training Algorithm......Page 178
4.5 Applications Areas of ANN......Page 179
4.5.1 Language Processing......Page 180
4.5.4 Pattern Recognition......Page 181
4.5.6 Financial......Page 182
Review Questions......Page 183
5.1 Illustration 1: Coin Detection Using Euclidean Distance (Hamming Net)......Page 185
Observations......Page 189
5.2 Illustration 2: Learning Vector Quantization - Clustering Data Drawn from Different Regions......Page 191
5.3 Illustration 3: Character Recognition Using Kohonen Som Network......Page 194
Observations......Page 201
5.4 Illustration 4: The Hopfield Network as an Associative Memory......Page 202
Observations......Page 205
5.5 Illustration 5: Generalized Delta Learning Rule and Back-Propagation of Errors for a Multilayer Network......Page 207
Observations......Page 208
5.6 Illustration 6: Classification of Heart Disease Using Learning Vector Quantization......Page 209
5.7 Illustration 7: Neural Network Using MATLAB Simulink......Page 218
Observations......Page 219
Review Questions......Page 220
6.1 Introduction......Page 222
6.3 Crisp and Fuzzy Logic......Page 224
6.4 Fuzzy Sets......Page 226
6.5 Universe......Page 228
6.6 Membership Functions......Page 229
Triangular Function......Page 231
Gaussian Function......Page 232
6.6.2 Membership Functions in the MATLAB Fuzzy Logic Toolbox......Page 233
Trapezoidal Membership Function......Page 236
Gaussian Membership Function Syntax......Page 237
Generalized Bell Shaped Membership Function......Page 238
Sigmoidal Shaped Membership Function......Page 239
Z Curve Membership Function......Page 240
S Curve Membership Function......Page 241
6.6.4 Translation of Parameters between Membership Functions Using MATLAB......Page 242
6.7 Singletons......Page 243
6.9 Operations on Fuzzy Sets......Page 244
6.9.1 Fuzzy Complements......Page 246
6.9.2 Fuzzy Intersections: t-norms......Page 249
6.9.3 Fuzzy Unions: t-conorms......Page 252
6.9.4 Combinations of Operations......Page 254
Complement......Page 255
Union......Page 256
Intersection......Page 257
6.9.6 Aggregation Operations......Page 258
6.10 Fuzzy Arithmetic......Page 261
6.10.1 Arithmetic Operations on Intervals......Page 262
6.10.2 Arithmetic Operations on Fuzzy Numbers......Page 263
Illustration......Page 265
6.11 Fuzzy Relations......Page 266
6.12 Fuzzy Composition......Page 270
6.12.1 MATLAB Code to Implement Fuzzy Composition......Page 273
Review Questions......Page 279
7.2 Fuzzy Rules......Page 280
7.2.2 Disintegration of Rules......Page 281
7.2.3 Aggregation of Rules......Page 282
7.3.1 Fuzzification......Page 283
7.3.2 Fuzzy Rule Base and Fuzzy IF-THEN Rules......Page 284
7.3.3 Fuzzy Inference Machine......Page 285
Max-Membership principle......Page 286
Center of gravity method (COG)......Page 287
Weighted Average method......Page 288
Center of sums......Page 289
Center of largest area......Page 290
First or Last of maxima......Page 291
7.3.5 Implementation of Defuzzification using MATLAB Fuzzy Logic Toolbox......Page 292
Centroid method......Page 293
Mean of Maximum methods......Page 294
7.4 Fuzzy Inference Methods......Page 295
7.4.1 Mamdani’s Fuzzy Inference Method......Page 297
7.4.2 Takagi–Sugeno Fuzzy Inference Method......Page 300
7.4.3 Tsukamoto Fuzzy Inference Method......Page 301
7.5 Fuzzy Inference Systems in MATLAB......Page 302
7.5.1 Mamdani-Type Fuzzy Inference......Page 305
Application of the Fuzzy Operator......Page 307
Application of Implication......Page 308
Defuzzify......Page 309
Advantages of the Mamdani Method......Page 310
7.5.2 Sugeno-Type Fuzzy Inference......Page 311
7.5.3 Conversion of Mamdani to Sugeno System......Page 314
7.6 Fuzzy Automata and Languages......Page 316
7.7 Fuzzy Control......Page 317
7.7.1 Fuzzy Controllers......Page 319
7.7.2 A Fuzzy Controller in MATLAB......Page 321
Review Questions......Page 324
8.1 Illustration 1: Application of Fuzzy Controller Using MATLAB — Fuzzy Washing Machine......Page 326
8.2 Illustration 2 - Fuzzy Control System for a Tanker Ship......Page 336
8.3 Illustration 3 - Approximation of Any Function Using Fuzzy Logic......Page 355
8.4 Illustration 4 - Building Fuzzy Simulink Models......Page 362
Review Questions......Page 367
9.1 Introduction......Page 369
9.3 Fused Neuro-Fuzzy Systems......Page 370
9.3.2 Adaptive Neuro-Fuzzy Inference System (ANFIS)......Page 371
9.3.3 Generalized Approximate Reasoning-Based Intelligent Control (GARIC)......Page 373
NEFCON Learning Algorithm......Page 374
Decremental Rule Learning......Page 375
Incremental Rule Learning......Page 376
9.3.5 Fuzzy Inference and Neural Network in Fuzzy Inference Software (FINEST)......Page 378
9.3.6 Fuzzy Net (FUN)......Page 380
9.3.7 Evolving Fuzzy Neural Network (EFuNN)......Page 381
9.3.9 Evolutionary Design of Neuro-Fuzzy Systems......Page 382
9.4.1 Architecture of Adaptive Neuro-Fuzzy Inference System......Page 385
Layer 2 - Product Layer......Page 386
Layer 5 - Total Output Layer......Page 387
Forward Pass: Least Squares Estimate Method......Page 388
Backward Pass: Steepest Descent Method......Page 389
9.5.1 CART — Introduction......Page 390
9.5.2 Node Splitting Criteria......Page 391
9.5.3 Classification Trees......Page 392
9.5.5 Computational Issues of CART......Page 393
9.5.6 Computational Steps......Page 394
9.5.7 Accuracy Estimation in CART......Page 397
9.5.8 Advantages of Classification and Regression Trees......Page 398
9.6.1 System Identification Using Fuzzy Clustering......Page 400
9.6.2 Hard C-Means Clustering......Page 401
9.6.3 Fuzzy C-Means (FCM) Clustering......Page 404
Algorithm......Page 405
9.6.4 Subtractive Clustering......Page 406
9.6.5 Experiments......Page 407
Review Questions......Page 411
10.1 Illustration 1 - Fuzzy Art Map......Page 413
10.2 Illustration 2: Fuzzy C-Means Clustering - Comparative Case Study......Page 420
10.3 Illustration 3 - Kmeans Clustering......Page 421
10.4 Illustration 4 - Neuro-Fuzzy System Using Simulink......Page 429
10.5 Illustration 5 - Neuro-Fuzzy System Using Takagi–Sugeno and ANFIS GUI of MATLAB......Page 432
Summary......Page 435
Review Questions......Page 436
11.1 Introduction......Page 437
11.2 Evolutionary Computation......Page 438
11.3 Brief History of Evolutionary Computation......Page 440
11.4.2 Biological Evolution Inspired by Nature......Page 441
11.4.3 Evolutionary Biology......Page 443
11.5 Flow Diagram of a Typical Evolutionary Algorithm......Page 446
11.6.1 Genetic Algorithms (GA)......Page 448
11.6.2 Genetic Programming (GP)......Page 449
11.6.3 Evolutionary Programming (EP)......Page 453
11.6.4 Evolutionary Strategies (ESs)......Page 454
Randomness......Page 456
11.7 Evolutionary Algorithms......Page 457
11.7.2 Solution Representation......Page 460
11.7.3 Fitness Function......Page 461
11.7.5 Selection Mechanisms......Page 462
Illustration......Page 463
Local Selection......Page 465
Truncation Selection......Page 467
Tournament Selection......Page 468
11.7.6 Crossover Technique......Page 469
Discrete Recombination......Page 470
Arithmetic Crossover......Page 471
Crossover Probability......Page 472
Real Valued Mutation......Page 473
Boundary......Page 474
Mutation Probability......Page 475
11.7.8 Reproduction Operator......Page 476
11.8.1 History......Page 477
11.8.2 Procedure of Evolutionary Programming......Page 478
Choosing a Solution’s Coding Scheme......Page 479
Extending the Coding Scheme......Page 483
Generating an Initial Population......Page 484
The Current Population......Page 487
Offspring Generation in Evolutionary Programming Methods......Page 488
Merging the Populations......Page 493
Choosing a Current Population......Page 495
11.9 Evolutionary Strategies......Page 496
11.9.2 Mutation......Page 498
Discrete Recombination......Page 499
11.9.4 Population Assessment......Page 500
11.9.6 Computational Considerations......Page 501
11.9.7 Algorithm Performance......Page 502
11.10 Advantages and Disadvantages of Evolutionary Computation......Page 503
Summary......Page 505
Review Questions......Page 506
12.1 Illustration 1: Differential Evolution Optimizer......Page 508
12.2 Illustration 2: Design of a Proportional-Derivative Controller Using Evolutionary Al-gorithm for Tanker Ship Heading Regulation......Page 519
12.3 Illustration 3: Maximizing the Given One-Dimensional Function with the Boundaries Using Evolutionary Algorithm......Page 540
12.4 Illustration 4: Multiobjective Optimization Using Evolutionary Algorithm (MOEA)......Page 543
12.5 Illustration 5: Evolutionary Strategy for Nonlinear Function Minimization......Page 558
Review Questions......Page 562
13.1 Introduction......Page 564
13.2 Encoding and Optimization Problems......Page 565
13.3 Historical Overview of Genetic Algorithm......Page 566
13.4 Genetic Algorithm Description......Page 567
13.5 Role of Genetic Algorithms......Page 569
13.6 Solution Representation of Genetic Algorithms......Page 570
Grefenstette Settings......Page 571
13.8 Schema Theorem and Theoretical Background......Page 572
13.8.1 Building Block Hypothesis......Page 573
13.8.2 The Dynamics of a Schema......Page 574
13.8.3 Illustrations Based on Schema Theorem......Page 576
13.9.2 2-Point Crossover......Page 580
13.9.3 Linkage and Defining Length......Page 581
13.9.4 Linkage and Inversion......Page 582
13.10 Genotype and Fitness......Page 583
13.11.2 Epistasis......Page 584
13.11.3 Deception......Page 585
13.11.7 Diploidy and Dominance......Page 586
13.12.1 Neural Nets......Page 587
13.12.4 Iterated Search......Page 588
13.13 Benefits of GA......Page 589
13.14.1 Illustration 1: Maximizing the Given One-Dimensional Function within Given Boundaries......Page 590
13.14.2 Illustration 2: Solving Economic Dispatch Problem Using Genetic Algorithm......Page 593
13.14.3 Illustration 3: Traveling Salesman Problem......Page 597
Summary......Page 604
Review Questions......Page 605
14.1 Introduction......Page 607
14.2 Growth of Genetic Programming......Page 610
14.3 The Lisp Programming Language......Page 611
14.4.1 Generation of an Individual and Population......Page 612
14.4.2 Creating a Random Population......Page 613
14.4.5 The Genetic Operations......Page 615
14.4.6 Selection Functions......Page 616
14.4.7 MATLAB Routine for Selection......Page 617
14.4.8 Crossover Operation......Page 620
14.4.9 MATLAB Routine for Crossover......Page 623
14.4.11 MATLAB Routine for Mutation......Page 624
Connectionist Model......Page 626
Inductive Model......Page 627
14.6 Elementary Steps of Genetic Programming......Page 628
14.6.3 The Fitness Function......Page 629
14.6.5 The Termination Criterion......Page 630
14.7 Flowchart of Genetic Programming......Page 631
14.9.1 Illustration 1: Static Function Identification......Page 634
14.9.2 Illustration 2: Dynamical Input-Output Model Identification......Page 645
14.9.3 Illustration 3 - Symbolic Regression Problem Using Genetic Programming Toolbox......Page 648
Summary......Page 661
Review Questions......Page 662
15.1 Introduction to Swarms......Page 664
15.2 Biological Background......Page 665
15.3 Swarm Robots......Page 667
15.4 Stability of Swarms......Page 668
15.5 Swarm Intelligence......Page 669
15.5.1 Properties of a Swarm Intelligence System......Page 670
15.6 Particle Swarm Optimization (PSO)......Page 671
15.6.1 Mathematical Model of PSO......Page 672
15.6.3 Parameters and Tuning of Parameters in PSO......Page 673
15.6.4 Neighborhood Topologies......Page 674
Von Neumann Topology......Page 675
15.7.2 PSO-II......Page 676
15.7.3 PSO-III......Page 677
15.8 Ant Colony Optimization......Page 678
15.8.2 Ant Colony Optimization Algorithm......Page 679
15.9.2 Clustering Behavior of Ants......Page 680
15.9.4 Machine Scheduling......Page 681
15.9.5 Quadratic Knapsack Problem......Page 682
15.10 MATLAB Examples of Swarm Intelligence......Page 683
15.10.1 Illustration 1: Simulation of the Movement of Swarm to Minimize the Objective Function......Page 684
15.10.2 Illustration 2: Behavior of Particle Swarm Optimization......Page 686
15.10.3 Illustration 3: Ant Colony Optimization to Determine the Shortest Path......Page 699
15.10.4 Illustration 4: Ant Algorithm for the Quadratic Assignment Problem (QAP)......Page 702
Review Questions......Page 706
A......Page 708
B......Page 710
C......Page 712
D......Page 717
E......Page 718
F......Page 722
G......Page 725
H......Page 728
I......Page 730
L......Page 732
M......Page 734
N......Page 736
O......Page 738
P......Page 740
R......Page 742
S......Page 743
T......Page 747
V......Page 749
X......Page 750
Appendix B: List of Abbreviations......Page 751
C.1 Genetic Algorithm Toolbox for MATLAB......Page 754
Crossover Operators......Page 755
C.2 Fuzzy Logic Toolbox 2.2.7......Page 756
Membership Functions......Page 757
Working in Simulink Environment......Page 758
C.3 Neural Network Toolbox 6.0......Page 759
Distance Functions......Page 760
Line Search Functions......Page 761
New Networks Functions......Page 762
Plotting Functions......Page 763
Training Functions......Page 764
Transfer Functions......Page 765
Weight and Bias Initialization Functions......Page 766
C.4 Genetic Algorithm and Direct Search Toolbox 2.2......Page 767
Simulated Annealing......Page 768
C.5 GPLAB - A Genetic Programming Toolbox for MATLAB......Page 769
Running the Algorithm and Testing Result......Page 770
Description of Parameter and State Variables......Page 771
Filtering of New Individuals......Page 772
Artificial Ant Functions......Page 773
Expected Number of Children......Page 774
Fitness......Page 775
Automatic Operator Probability Adaptation......Page 776
Utilitarian Functions......Page 777
D.2 ComputerAnts......Page 778
D.5 Evolution Machine......Page 779
D.7 GAC, GAL......Page 780
D.9 GAGS......Page 781
D.11 GALOPPS......Page 782
D.12 GAMusic......Page 783
D.15 GECO......Page 784
D.17 GENEsYs......Page 785
D.20 Genitor......Page 786
D.23 GPEIST......Page 787
D.25 JAG......Page 788
D.28 PGA......Page 789
D.29 PGAPack......Page 790
D.31 Splicer......Page 791
D.33 WOLF......Page 792
D.35 XFUZZY: A Simulation Environment for Fuzzy Logic Control Systems......Page 793
D.37 COMDALE/C, COMDALE/X, and ProcessVision......Page 794
E.1 An Evolutionary Algorithm for Global Optimization Based on Level-Set Evolution and Latin Squares......Page 796
E.3 Analog Genetic Encoding for the Evolution of Circuits and Networks......Page 797
E.5 Solving the Register Allocation Problem for Embedded Systems Using a Hybrid Evolutionary Algorithm......Page 798
E.6 Semantic Understanding of General Linguistic Items by Means of Fuzzy Set Theory......Page 799
E.8 An Ant Colony Optimization Approach to the Probabilistic Traveling Salesman Problem......Page 800
E.9 Neural Processing of Symbolic Data......Page 801
References......Page 802
References......Page 830