Artificial Intelligence in the Age of Neural Networks and Brain Computing

Cover......Page 1
Artificial Intelligence in the Age of Neural Networks and Brain Computing......Page 2
Copyright......Page 3
List of Contributors......Page 4
14 -
Meaning Versus Information, Prediction Versus Memory, and Question Versus Answer......Page 253
Introduction......Page 9
1. Introduction......Page 14
2. ADALINE and the LMS Algorithm, From the 1950s......Page 16
3. Unsupervised Learning With Adaline, From the 1960s......Page 18
4. Robert Lucky's Adaptive Equalization, From the 1960s......Page 20
5. Bootstrap Learning With a Sigmoidal Neuron......Page 23
6. Bootstrap Learning With a More “Biologically Correct” Sigmoidal Neuron......Page 26
6.1 Training a Network of Hebbian-LMS Neurons......Page 30
7.1 K-Means Clustering......Page 33
8. A General Hebbian-LMS Algorithm......Page 34
9. The Synapse......Page 35
10. Postulates of Synaptic Plasticity......Page 38
12. Nature's Hebbian-LMS Algorithm......Page 39
Appendix: Trainable Neural Network Incorporating Hebbian-LMS Learning......Page 40
References......Page 42
2 -
A Half Century of Progress Toward a Unified Neural Theory of Mind and Brain With Applications to Autonomous Adaptive Agents .........Page 44
1. Towards a Unified Theory of Mind and Brain......Page 45
2. The Neural Network Approach......Page 229
3. Revolutionary Brain Paradigms: Complementary Computing and Laminar Computing......Page 48
4. Electrophysiological Time-Series......Page 49
5. Adaptive Resonance Theory......Page 50
7. Homologous Laminar Cortical Circuits for All Biological Intelligence: Beyond Bayes......Page 53
8. Why a Unified Theory Is Possible: Equations, Modules, and Architectures......Page 58
10. The Varieties of Brain Resonances and the Conscious Experiences That They Support......Page 59
11. Why Does Resonance Trigger Consciousness?......Page 60
12. Towards Autonomous Adaptive Intelligent Agents and Clinical Therapies in Society......Page 61
References......Page 62
1. Introduction......Page 65
2. Third Gen AI......Page 68
1.1 Aristotle's Logic and Rule-Based Systems for Knowledge Representation and Reasoning......Page 73
2.2 The Inverse Is Convolution Neural Networks......Page 75
4.1 Passive Adaptation Modality......Page 259
3. Neural Networks Enter Mainstream Science......Page 82
4. Need for New Directions in Understanding Brain and Mind......Page 86
References......Page 89
6. Discussion and Future Work......Page 314
1. Dichotomies......Page 91
1.1 The Brain-Mind Problem......Page 92
1.2 The Brain-Computer Analogy/Disanalogy......Page 93
1.3 The Computational Theory of Mind......Page 95
2. Hermeneutics......Page 96
2.3 The Brain as a Hermeneutic Device......Page 97
2.4 Neural Hermeneutics......Page 98
3.1 Use of Data......Page 159
3.1 Hermeneutics, Cognitive Science, Schizophrenia......Page 99
4.1 Automated Diagnosis Support Tools for Neurodegenerative Disorders......Page 280
4.1 Understanding Situations: Needs Hermeneutic Interpretation......Page 100
References......Page 101
Further Reading......Page 104
1. Early History......Page 105
2. Ephapsis......Page 107
3. Embodied Cognition......Page 109
4. Wearable Personal Assistants......Page 117
References......Page 121
6 -
Evolving and Spiking Connectionist Systems for Brain-Inspired Artificial Intelligence......Page 122
1. From Aristotle's Logic to Artificial Neural Networks and Hybrid Systems......Page 123
1.2 Fuzzy Logic and Fuzzy Rule–Based Systems......Page 124
1.3 Classical Artificial Neural Networks (ANN)......Page 125
1.4 Integrating ANN With Rule-Based Systems: Hybrid Connectionist Systems......Page 126
1.5 Evolutionary Computation (EC): Learning Parameter Values of ANN Through Evolution of Individual Models as Part of Populatio .........Page 127
2.1 Principles of ECOS......Page 128
2.2 ECOS Realizations and AI Applications......Page 129
3.1 Main Principles, Methods, and Examples of SNN and Evolving SNN (eSNN)......Page 132
3.2 Applications and Implementations of SNN for AI......Page 135
4.1 Brain-Like AI Systems. NeuCube......Page 136
4.2 Deep Learning and Deep Knowledge Representation in NeuCube SNN Models: Methods and AI Applications [6]......Page 138
4.2.1 Supervised Learning for Classification of Learned Patterns in a SNN Model......Page 139
4.2.2 Semisupervised Learning......Page 140
5. Conclusion......Page 141
References......Page 142
1. Introduction......Page 272
1. Introduction......Page 150
2.1 Our Data Are Crap......Page 153
2.2 Our Algorithm Is Crap......Page 157
3.2 Performance Measures......Page 160
4.2 Evolving DNNs in the Language Modeling Benchmark......Page 308
4. Variability and Bias in Our Performance Estimates......Page 165
5. Conclusion......Page 168
ACKNOWLEDGMENT......Page 169
8 -
The New AI: Basic Concepts, and Urgent Risks and Opportunities in the Internet of Things......Page 171
1.2 The Deep Learning Cultural Revolution and New Opportunities......Page 172
1.3 Need and Opportunity for a Deep Learning Revolution in Neuroscience......Page 173
2.1 Overview of the Current Landscape......Page 174
2.2 How the Deep Revolution Actually Happened......Page 177
2.3 Backpropagation: The Foundation Which Made This Possible......Page 178
2.4 CoNNs, ﹥3 Layers, and Autoencoders: The Three Main Tools of Today's Deep Learning......Page 180
3.1 Two Types of Recurrent Neural Network......Page 182
3.2 Deep Versus Broad: A Few Practical Issues......Page 185
3.4 Emerging New Hardware to Enhance Capability by Orders of Magnitude......Page 186
4.1 Toward a Cultural Revolution in Hard Neuroscience......Page 189
4.2 From Mouse Brain to Human Mind: Personal Views of the Larger Picture......Page 192
5. Information Technology (IT) for Human Survival: An Urgent Unmet Challenge......Page 194
5.2 Cyber and EMP Threats to the Power Grid......Page 195
5.3 Threats From Underemployment of Humans......Page 196
5.4 Preliminary Vision of the Overall Problem, and of the Way out......Page 197
References......Page 198
9 -
Theory of the Brain and Mind: Visions and History......Page 201
2. Emergence of Some Neural Network Principles......Page 203
4. Is Computational Neuroscience Separate From Neural Network Theory?......Page 206
5. Deep Learning Models for EEG Signal Processing......Page 208
References......Page 209
1. Introduction......Page 214
2. AI Approaches......Page 217
3. Metastability in Cognition and in Brain Dynamics......Page 219
4. Multistability in Physics and Biology......Page 220
References......Page 225
11 - Deep Learning Approaches to Electrophysiological Multivariate Time-Series Analysis......Page 228
3. Deep Architectures and Learning......Page 231
3.1 Deep Belief Networks......Page 232
3.3 Convolutional Neural Networks......Page 233
4.2 Electroencephalography (EEG)......Page 235
4.3 High-Density Electroencephalography......Page 237
5.1 Stacked Autoencoders......Page 241
5.2 Summary of the Proposed Method for EEG Classification......Page 244
6. Future Directions of Research......Page 245
6.2 Advanced Learning Approaches in DL......Page 247
7. Conclusions......Page 248
References......Page 249
Further Reading......Page 252
3. Energy Harvesting and Management......Page 255
3.1 Energy Harvesting......Page 256
3.2 Energy Management and Research Challenges......Page 257
4. Learning in Nonstationary Environments......Page 258
4.2 Active Adaptation Modality......Page 260
4.3 Research Challenges......Page 261
5.1 Model-Free Fault Diagnosis Systems......Page 262
5.2 Research Challenges......Page 264
6.1 How Can CPS and IoT Be Protected From Cyberattacks?......Page 265
6.2 Case Study: Darknet Analysis to Capture Malicious Cyberattack Behaviors......Page 266
References......Page 269
2. Multiview Learning......Page 273
2.1 Integration Stage......Page 274
2.2 Type of Data......Page 275
3.1 Patient Subtyping......Page 276
3.2 Drug Repositioning......Page 279
5. Deep Multimodal Feature Learning......Page 282
5.2 Multimodal Neuroimaging Feature Learning With Deep Learning......Page 283
6. Conclusions......Page 284
References......Page 285
1. Introduction......Page 288
2. Meaning Versus Information......Page 289
3. Prediction Versus Memory......Page 291
4. Question Versus Answer......Page 294
5. Discussion......Page 295
6. Conclusion......Page 297
References......Page 298
1. Introduction......Page 300
2. Background and Related Work......Page 301
3.1 Extending NEAT to Deep Networks......Page 303
3.2 Cooperative Coevolution of Modules and Blueprints......Page 304
3.3 Evolving DNNs in the CIFAR-10 Benchmark......Page 305
4.1 Extending CoDeepNEAT to LSTMs......Page 307
5. Application Case Study: Image Captioning for the Blind......Page 309
5.1 Evolving DNNs for Image Captioning......Page 310
5.2 Building the Application......Page 311
5.3 Image Captioning Results......Page 312
References......Page 316
A......Page 320
C......Page 321
D......Page 323
F......Page 324
I......Page 325
M......Page 326
N......Page 327
P......Page 328
S......Page 329
T......Page 330
X......Page 331
Back Cover......Page 332