Artificial intelligence (AI) is a concept, whose meaning and perception has changed considerably over the last decades. Starting off with individual and purely theoretical research efforts in the 1950s, AI has grown into a fully developed research field of modern times and may arguably emerge as one of the most important technological advancements of mankind. Despite these rapid technological advancements, some key questions revolving around the matter of transparency, interpretability and explainability of an AI’s decision-making remain unanswered. Thus, a young research field coined with the general term Explainable AI (XAI) has emerged from increasingly strict requirements for AI to be used in safety critical or ethically sensitive domains. An important research branch of XAI is to develop methods that help to facilitate a deeper understanding for the learned knowledge of artificial neural systems. In this book, a series of scientific studies are presented that shed light on how to adopt an empirical neuroscience inspired approach to investigate a neural network’s learned representation in the same spirit as neuroscientific studies of the brain.
Author(s): Richard Meyes
Publisher: Springer Vieweg
Year: 2022
Language: English
Pages: 232
City: Wiesbaden
Contents
List of Figures
List of Tables
1 Introduction
1.1 Object of Investigation
1.2 Research Questions
1.3 Structure of the Thesis
2 Background & Foundations
2.1 A Short History of AI Research
2.1.1 The Early Years
2.1.2 The Golden Ages
2.1.3 The AI Winter
2.1.4 The AI Renaissance
2.2 The Modern Era of AI Research
2.2.1 Deep Learning for Computer Vision
2.2.2 Computer Vision Beyond the ILSVRC
2.2.3 From Supervised Learning to Reinforcement Learning
2.2.4 Deep Reinforcement Learning Breakthroughs for Video and Board Games
2.2.5 Tackling Games with Imperfect Information
2.3 Towards Research on Transparency & Interpretability
2.3.1 A Shift of Paradigm: From Optimizing to Understanding
2.3.2 Inspirations from Neuroscience Research
3 Methods and Terminology
3.1 Learned Representations
3.1.1 Investigating Learned Representations
3.1.2 Visualizing Structure of the Learned Representations
3.2 Delimitation of the Object of Investigation
3.2.1 Transparency for Computer Vision Models
3.2.2 Transparency for Motion Control Models
3.2.3 Transfer to an Industrial Application Scenario
4 Related Work
4.1 Relationship Between a Network’s Input Features and its Output
4.2 Visualization of Network Properties and Graphical User Interfaces
4.3 Investigating the Importance of Individual Network Components
4.3.1 Miscellaneous Contributions
4.3.2 Ablations Studies
4.3.3 Reverse Engineering of Neural Networks
5 Research Studies
5.1 Investigating Learned Representations in Computer Vision
5.1.1 Research Study 1: Characterizing Single Neurons in a Shallow MLP
5.1.2 Research Study 2: Network Ablations in a Deep Neural Network
5.1.3 Research Study 3: Functional Neuron Populations in Custom-made CNNs
5.2 Investigating Learned Representations in Motor Control
5.2.1 Research Study 4: Influence of Network Ablations on Activation Patterns
5.2.2 Research Study 5: Relation Between Neural Activations and Agent Behavior
6 Transfer Studies
6.1 Transfer Study 1: Network Ablations for Deep Drawing
6.1.1 Key Contributions of the Study
6.1.2 Methods and Experimental Design
6.1.3 Results
6.1.4 Summary and Contribution of the Results to the Research Questions
6.2 Transfer Study 2: Attention Mechanisms for Deep Drawing
6.2.1 Key Contributions of the Study
6.2.2 Methods and Experimental Design
6.2.3 Results
6.2.4 Summary and Contribution of the Results to the Research Questions
7 Critical Reflection & Outlook
7.1 Reflection of Results & Contribution to Research Questions
7.1.1 Research Question 1
7.1.2 Research Question 2
7.1.3 Research Question 3
7.1.4 Research Question 4
7.2 Future Research Directions
8 Summary
References
