This text serves as a cookbook for neural network solutions to practical problems using C++. It will enable those with moderate programming experience to select a neural network model appropriate to solving a particular problem, and to produce a working program implementing that network. The book provides guidance along the entire problem-solving path, including designing the training set, preprocessing variables, training and validating the network, and evaluating its performance. Though the book is not intended as a general course in neural networks, no background in neural works is assumed and all models are presented from the ground up.The principle focus of the book is the three layer feedforward network, for more than a decade as the workhorse of professional arsenals. Other network models with strong performance records are also included.Bound in the book is an IBM diskette that includes the source code for all programs in the book. Much of this code can be easily adapted to C compilers. In addition, the operation of all programs is thoroughly discussed both in the text and in the comments within the code to facilitate translation to other languages.
Author(s): Masters
Edition: Book & Disk 1st
Publisher: Morgan Kaufmann
Year: 1993
Language: English
Pages: 509
Tags: Информатика и вычислительная техника;Искусственный интеллект;Нейронные сети;
Contents......Page all_10073_to_00502.cpc0006.djvu
Preface......Page all_10073_to_00502.cpc0013.djvu
1. Foundations......Page all_10073_to_00502.cpc0015.djvu
Motivation......Page all_10073_to_00502.cpc0016.djvu
New Life for Old Techniques......Page all_10073_to_00502.cpc0017.djvu
Perceptrons and Linear Separability......Page all_10073_to_00502.cpc0018.djvu
Neural Network Capabilities......Page all_10073_to_00502.cpc0020.djvu
Basic Structure of a Neural Network......Page all_10073_to_00502.cpc0022.djvu
Training......Page all_10073_to_00502.cpc0023.djvu
Validation......Page all_10073_to_00502.cpc0024.djvu
Leave-k-out Method......Page all_10073_to_00502.cpc0026.djvu
2. Classification......Page all_10073_to_00502.cpc0029.djvu
Binary Decisions......Page all_10073_to_00502.cpc0030.djvu
Making the Decision......Page all_10073_to_00502.cpc0031.djvu
Reject Category......Page all_10073_to_00502.cpc0032.djvu
Other Encoding Schemes......Page all_10073_to_00502.cpc0033.djvu
Supervised versus Unsupervised Training......Page all_10073_to_00502.cpc0035.djvu
3. Autoassociation......Page all_10073_to_00502.cpc0037.djvu
Autoassociative Filtering......Page all_10073_to_00502.cpc0038.djvu
Code for Autoassociative Filtering......Page all_10073_to_00502.cpc0042.djvu
Noise Reduction......Page all_10073_to_00502.cpc0043.djvu
Learning a Prototype from Exemplars......Page all_10073_to_00502.cpc0045.djvu
Exposing Isolated Events......Page all_10073_to_00502.cpc0046.djvu
Pattern Completion......Page all_10073_to_00502.cpc0054.djvu
Error Correction......Page all_10073_to_00502.cpc0055.djvu
Encoding Words......Page all_10073_to_00502.cpc0056.djvu
Data Compression......Page all_10073_to_00502.cpc0058.djvu
4. Time-Series Prediction......Page all_10073_to_00502.cpc0060.djvu
The Basic Model......Page all_10073_to_00502.cpc0062.djvu
Input Data......Page all_10073_to_00502.cpc0063.djvu
Trend Elimination......Page all_10073_to_00502.cpc0064.djvu
Code for Detrending and Retrending......Page all_10073_to_00502.cpc0068.djvu
Seasonal Variation......Page all_10073_to_00502.cpc0071.djvu
Scaling......Page all_10073_to_00502.cpc0073.djvu
Multiple Prediction......Page all_10073_to_00502.cpc0074.djvu
Multiple Predictors......Page all_10073_to_00502.cpc0075.djvu
Measuring Prediction Error......Page all_10073_to_00502.cpc0077.djvu
5. Function Approximation......Page all_10073_to_00502.cpc0080.djvu
Univariate Function Approximation......Page all_10073_to_00502.cpc0081.djvu
Inverse Modeling......Page all_10073_to_00502.cpc0085.djvu
Multiple Regression......Page all_10073_to_00502.cpc0087.djvu
6. Multilayer Feedforward Networks......Page all_10073_to_00502.cpc0090.djvu
Basic Architecture......Page all_10073_to_00502.cpc0091.djvu
Activation Functions......Page all_10073_to_00502.cpc0093.djvu
Example Network......Page all_10073_to_00502.cpc0095.djvu
Linear Output Neurons......Page all_10073_to_00502.cpc0097.djvu
Theoretical Discussion......Page all_10073_to_00502.cpc0098.djvu
Bibliography of Feedforward Network Theory......Page all_10073_to_00502.cpc0101.djvu
Algorithms for Executing the Network......Page all_10073_to_00502.cpc0103.djvu
Training the Network......Page all_10073_to_00502.cpc0107.djvu
Training by Backpropagation of Errors......Page all_10073_to_00502.cpc0113.djvu
Training by Conjugate Gradients......Page all_10073_to_00502.cpc0118.djvu
Minimizing along a Direction......Page all_10073_to_00502.cpc0119.djvu
Choosing the Direction for Minimization......Page all_10073_to_00502.cpc0123.djvu
Eluding Local Minima in Learning......Page all_10073_to_00502.cpc0124.djvu
Local Minima Happen Easily......Page all_10073_to_00502.cpc0125.djvu
Mistaken Minima......Page all_10073_to_00502.cpc0127.djvu
Other Means of Escape......Page all_10073_to_00502.cpc0128.djvu
When to Use a Multiple-Layer Feedforward Network......Page all_10073_to_00502.cpc0129.djvu
7. Eluding Local Minima I: Simulated Annealing......Page all_10073_to_00502.cpc0130.djvu
Overview......Page all_10073_to_00502.cpc0131.djvu
Choosing the Annealing Parameters......Page all_10073_to_00502.cpc0132.djvu
Implementation in Feedforward Network Learning......Page all_10073_to_00502.cpc0134.djvu
A Sample Program......Page all_10073_to_00502.cpc0135.djvu
A Sample Function......Page all_10073_to_00502.cpc0139.djvu
Random Number Generation......Page all_10073_to_00502.cpc0141.djvu
Going on from Here......Page all_10073_to_00502.cpc0145.djvu
8. Eluding Local Minima II: Genetic Optimization......Page all_10073_to_00502.cpc0148.djvu
Overview......Page all_10073_to_00502.cpc0149.djvu
Designing the Genetic Structure......Page all_10073_to_00502.cpc0151.djvu
Evaluation......Page all_10073_to_00502.cpc0153.djvu
Parent Selection......Page all_10073_to_00502.cpc0157.djvu
Reproduction......Page all_10073_to_00502.cpc0160.djvu
Mutation......Page all_10073_to_00502.cpc0161.djvu
A Genetic Minimization Subroutine......Page all_10073_to_00502.cpc0162.djvu
Some Functions for Genetic Optimization......Page all_10073_to_00502.cpc0168.djvu
Gray Codes......Page all_10073_to_00502.cpc0170.djvu
Two-Point Crossover......Page all_10073_to_00502.cpc0172.djvu
9. Regression and Neural Networks......Page all_10073_to_00502.cpc0178.djvu
Overview......Page all_10073_to_00502.cpc0179.djvu
Singular-Value Decomposition......Page all_10073_to_00502.cpc0180.djvu
Regression in Neural Networks......Page all_10073_to_00502.cpc0182.djvu
10. Designing Feedforward Network Architectures......Page all_10073_to_00502.cpc0186.djvu
How Many Hidden Layers?......Page all_10073_to_00502.cpc0187.djvu
How Many Hidden Neurons?......Page all_10073_to_00502.cpc0189.djvu
How Long Do I Train This Thing???......Page all_10073_to_00502.cpc0193.djvu
11. Interpreting Weights: How Does This Thing Work?......Page all_10073_to_00502.cpc0199.djvu
Features Used by Networks in General......Page all_10073_to_00502.cpc0202.djvu
Examination of Weight Vectors......Page all_10073_to_00502.cpc0203.djvu
Hinton Diagrams......Page all_10073_to_00502.cpc0204.djvu
Clustering......Page all_10073_to_00502.cpc0206.djvu
Sensitivity Analysis......Page all_10073_to_00502.cpc0207.djvu
Stereotypical Inputs......Page all_10073_to_00502.cpc0209.djvu
12. Probabilistic Neural Networks......Page all_10073_to_00502.cpc0213.djvu
Overview......Page all_10073_to_00502.cpc0214.djvu
Computational Aspects......Page all_10073_to_00502.cpc0220.djvu
Optimizing Sigma......Page all_10073_to_00502.cpc0221.djvu
Related Models......Page all_10073_to_00502.cpc0222.djvu
A Sample Program......Page all_10073_to_00502.cpc0223.djvu
Optimizing Sigma......Page all_10073_to_00502.cpc0225.djvu
Other Optimization Criteria......Page all_10073_to_00502.cpc0230.djvu
Bayesian Confidence Measures......Page all_10073_to_00502.cpc0231.djvu
Autoassociative Versions......Page all_10073_to_00502.cpc0232.djvu
When to Use a Probabilistic Neural Network......Page all_10073_to_00502.cpc0233.djvu
13. Functional Link Networks......Page all_10073_to_00502.cpc0235.djvu
Application to Nonlinear Approximation......Page all_10073_to_00502.cpc0238.djvu
Mathematics of the Functional Link Network......Page all_10073_to_00502.cpc0239.djvu
When to Use a Functional Link Network......Page all_10073_to_00502.cpc0241.djvu
14. Hybrid Networks......Page all_10073_to_00502.cpc0243.djvu
Functional Link Net as a Hidden Layer......Page all_10073_to_00502.cpc0244.djvu
Fast Bayesian Confidences......Page all_10073_to_00502.cpc0247.djvu
Training......Page all_10073_to_00502.cpc0250.djvu
Attention-based Processing......Page all_10073_to_00502.cpc0251.djvu
Factorable Problems......Page all_10073_to_00502.cpc0254.djvu
Training the Data Reduction Networks......Page all_10073_to_00502.cpc0255.djvu
Splitting Is Not Always Effective......Page all_10073_to_00502.cpc0256.djvu
15. Designing the Training Set......Page all_10073_to_00502.cpc0257.djvu
Number of Samples......Page all_10073_to_00502.cpc0258.djvu
Overfitting......Page all_10073_to_00502.cpc0259.djvu
Network Size Affects Training Set Size......Page all_10073_to_00502.cpc0260.djvu
Borderline Cases......Page all_10073_to_00502.cpc0261.djvu
Hidden Bias......Page all_10073_to_00502.cpc0262.djvu
Fudging Cases......Page all_10073_to_00502.cpc0263.djvu
16. Preparing Input Data......Page all_10073_to_00502.cpc0265.djvu
General Considerations......Page all_10073_to_00502.cpc0266.djvu
Nominal Variables......Page all_10073_to_00502.cpc0267.djvu
Ordinal Variables......Page all_10073_to_00502.cpc0271.djvu
Interval Variables......Page all_10073_to_00502.cpc0274.djvu
Is Scaling Always Necessary?......Page all_10073_to_00502.cpc0278.djvu
Transformations......Page all_10073_to_00502.cpc0279.djvu
Circular Discontinuity......Page all_10073_to_00502.cpc0282.djvu
View Angles......Page all_10073_to_00502.cpc0283.djvu
Hue......Page all_10073_to_00502.cpc0284.djvu
Outliers......Page all_10073_to_00502.cpc0286.djvu
Discarding Data......Page all_10073_to_00502.cpc0287.djvu
Missing Data......Page all_10073_to_00502.cpc0288.djvu
17. Fuzzy Data and Processing......Page all_10073_to_00502.cpc0291.djvu
Treating Fuzzy Values as Nominal and Ordinal......Page all_10073_to_00502.cpc0293.djvu
Advantages of Fuzzy Set Processing......Page all_10073_to_00502.cpc0294.djvu
The Neural Network - Fuzzy Set Interface......Page all_10073_to_00502.cpc0295.djvu
Membership Functions......Page all_10073_to_00502.cpc0296.djvu
Continuous Variables......Page all_10073_to_00502.cpc0299.djvu
Multivariate Domains......Page all_10073_to_00502.cpc0300.djvu
Hedges......Page all_10073_to_00502.cpc0301.djvu
Negation, Conjunction, and Disjunction......Page all_10073_to_00502.cpc0302.djvu
Modus Ponens......Page all_10073_to_00502.cpc0304.djvu
Combining Operations......Page all_10073_to_00502.cpc0307.djvu
Defuzzification......Page all_10073_to_00502.cpc0311.djvu
Maximum Height Method......Page all_10073_to_00502.cpc0312.djvu
Centroid Method......Page all_10073_to_00502.cpc0313.djvu
Constructors......Page all_10073_to_00502.cpc0315.djvu
Negation and Scaling......Page all_10073_to_00502.cpc0319.djvu
Conjunction and Disjunction......Page all_10073_to_00502.cpc0320.djvu
Centroid......Page all_10073_to_00502.cpc0326.djvu
Simplifying Interactions......Page all_10073_to_00502.cpc0328.djvu
Fuzzy One-of-n Coding......Page all_10073_to_00502.cpc0329.djvu
Simple Membership Output......Page all_10073_to_00502.cpc0331.djvu
Postprocessing with Defuzzification......Page all_10073_to_00502.cpc0332.djvu
18. Unsupervised Training......Page all_10073_to_00502.cpc0339.djvu
Input Normalization......Page all_10073_to_00502.cpc0342.djvu
Z-Axis Normalization......Page all_10073_to_00502.cpc0343.djvu
Training the Kohonen Network......Page all_10073_to_00502.cpc0344.djvu
Updating the Weights......Page all_10073_to_00502.cpc0346.djvu
Learning Rate......Page all_10073_to_00502.cpc0348.djvu
Measuring Network Error......Page all_10073_to_00502.cpc0349.djvu
Determining Convergence......Page all_10073_to_00502.cpc0350.djvu
Neurons That Refuse to Learn......Page all_10073_to_00502.cpc0351.djvu
Self-Organization......Page all_10073_to_00502.cpc0352.djvu
19. Evaluating Performance of Neural Networks......Page all_10073_to_00502.cpc0354.djvu
Mean Square Error......Page all_10073_to_00502.cpc0355.djvu
Problems with Mean Square Error......Page all_10073_to_00502.cpc0356.djvu
Relatives of Mean Square Error......Page all_10073_to_00502.cpc0357.djvu
Cost Functions......Page all_10073_to_00502.cpc0358.djvu
Confusion Matrix......Page all_10073_to_00502.cpc0359.djvu
ROC (Receiver Operating Characteristic) Curves......Page all_10073_to_00502.cpc0362.djvu
Computing the ROC Curve Area......Page all_10073_to_00502.cpc0365.djvu
Cost Functions and ROC Curves......Page all_10073_to_00502.cpc0368.djvu
Signal-to-Noise Ratio......Page all_10073_to_00502.cpc0370.djvu
20. Confidence Measures......Page all_10073_to_00502.cpc0372.djvu
Testing Individual Hypotheses......Page all_10073_to_00502.cpc0373.djvu
Computing Confidence......Page all_10073_to_00502.cpc0378.djvu
Confidence in the Null Hypothesis......Page all_10073_to_00502.cpc0379.djvu
Multiple Classes......Page all_10073_to_00502.cpc0380.djvu
Confidence in the Confidence......Page all_10073_to_00502.cpc0381.djvu
Example Programs......Page all_10073_to_00502.cpc0382.djvu
Sorting......Page all_10073_to_00502.cpc0383.djvu
Estimating the Distribution......Page all_10073_to_00502.cpc0384.djvu
Estimating Confidences......Page all_10073_to_00502.cpc0385.djvu
Bayesian Methods......Page all_10073_to_00502.cpc0387.djvu
Example Program......Page all_10073_to_00502.cpc0392.djvu
Multiple Classes......Page all_10073_to_00502.cpc0393.djvu
Hypothesis Testing versus Bayes' Method......Page all_10073_to_00502.cpc0395.djvu
21. Optimizing the Decision Threshold......Page all_10073_to_00502.cpc0400.djvu
22. Using the NEURAL Program......Page all_10073_to_00502.cpc0413.djvu
GENERAL Model......Page all_10073_to_00502.cpc0415.djvu
The LAYER Network Model......Page all_10073_to_00502.cpc0416.djvu
Initialization by Genetic Optimization......Page all_10073_to_00502.cpc0417.djvu
Learning......Page all_10073_to_00502.cpc0418.djvu
The KOHONEN Network Model......Page all_10073_to_00502.cpc0419.djvu
Initialization and Learning......Page all_10073_to_00502.cpc0420.djvu
Saving Weights and Execution Results......Page all_10073_to_00502.cpc0422.djvu
Alphabetical Glossary of Commands......Page all_10073_to_00502.cpc0423.djvu
Verification of Program Operation......Page all_10073_to_00502.cpc0427.djvu
Appendix......Page all_10073_to_00502.cpc0433.djvu
Bibliography......Page all_10073_to_00502.cpc0488.djvu
Index......Page all_10073_to_00502.cpc0500.djvu
