This book presents recent advances in automated machine learning (AutoML) and automated algorithm design and indicates the future directions in this fast-developing area. Methods have been developed to automate the design of neural networks, heuristics and metaheuristics using techniques such as metaheuristics, statistical techniques, machine learning and hyper-heuristics. The book first defines the field of automated design, distinguishing it from the similar but different topics of automated algorithm configuration and automated algorithm selection. The chapters report on the current state of the art by experts in the field and include reviews of AutoML and automated design of search, theoretical analyses of automated algorithm design, automated design of control software for robot swarms, and overfitting as a benchmark and design tool. Also covered are automated generation of constructive and perturbative low-level heuristics, selection hyper-heuristics for automated design, automated design of deep-learning approaches using hyper-heuristics, genetic programming hyper-heuristics with transfer knowledge and automated design of classification algorithms. The book concludes by examining future research directions of this rapidly evolving field. The information presented here will especially interest researchers and practitioners in the fields of artificial intelligence, computational intelligence, evolutionary computation and optimisation.
Author(s): Nelishia Pillay, Rong Qu
Series: Natural Computing Series
Publisher: Springer
Year: 2021
Language: English
Pages: 190
City: Cham
Foreword
Preface
Acknowledgements
Contents
Contributors
Acronyms
1 Recent Developments of Automated Machine Learning and Search Techniques
1.1 Introduction
1.2 Automated Algorithm Design and Machine Learning
1.3 Challenges in Automated Design of Algorithms and Machine Learning
1.4 Conclusions
References
2 Automated Machine Learning—A Brief Review at the End of the Early Years
2.1 Introduction
2.2 Fundamentals of AutoML
2.2.1 Supervised Learning
2.2.2 Notions of AutoML
2.2.3 Disentangling AutoML Methods
2.3 AutoML Methodologies
2.3.1 First Wave: 2006–2010
2.3.2 Second Wave: 2011–2016
2.3.3 Third Wave: 2017 and On
2.4 AutoML Challenges
2.5 Open Issues and Research Opportunities
2.6 Conclusions
References
3 A General Model for Automated Algorithm Design
3.1 Introduction
3.2 Automated Algorithm Design
3.2.1 Automated Algorithm Configuration
3.2.2 Automated Algorithm Selection
3.2.3 Automated Algorithm Composition
3.3 The General Combinatorial Optimisation Problem
3.4 Search Algorithms Defined with the GCOP Model
3.5 Challenges in Automated Algorithm Design with the GCOP
3.6 Conclusions
References
4 Rigorous Performance Analysis of Hyper-heuristics
4.1 Introduction
4.2 Selection Hyper-heuristics
4.2.1 Heuristic Selection Methods
4.2.2 Move Acceptance Operators
4.3 Hyper-heuristics Are Necessary
4.4 Learning is Necessary
4.5 Hyper-heuristics Can Achieve Optimal Performance
4.5.1 The RandomGradient Hyper-heuristic has Optimal Performance for LeadingOnes
4.5.2 A Reinforcement Learning Hyper-heuristic has Optimal Performance for OneMax
4.6 Automatically Adapting the Learning Period is Necessary
4.7 Switching Between Move Acceptance Operators is Necessary
4.7.1 Local Optima with Small Basins of Attraction
4.7.2 Local Optima with Large Basins of Attraction
4.8 Conclusion
References
5 AutoMoDe: A Modular Approach to the Automatic Off-Line Design and Fine-Tuning of Control Software for Robot Swarms
5.1 An Introduction to Swarm Robotics and Its Design Problem
5.2 From Neuro-Evolutionary Robotics to AutoMoDe
5.3 The Specializations of AutoMoDe
5.4 Further Corroboration of Our Working Hypothesis
5.5 Discussion on Future Research Directions
References
6 A Cross-Domain Method for Generation of Constructive and Perturbative Heuristics
6.1 Introduction
6.2 Related Work
6.3 Evaluation Domains and Their Representation as Graphs
6.4 Using Grammatical Evolution to Evolve Low-Level Heuristics
6.4.1 Evolving Constructive Heuristics
6.4.2 Evolving Perturbative Heuristics
6.5 Methodology
6.5.1 Training Phase
6.5.2 Test Phase
6.6 Results
6.6.1 Constructive Heuristics
6.7 Results: Perturbative Heuristics
6.8 Discussion
6.9 Conclusion
References
7 Hyper-heuristics: Autonomous Problem Solvers
7.1 Introduction
7.2 Motivation and Claims
7.3 Background
7.3.1 Selection Hyper-heuristics
7.3.2 Generation Hyper-heuristics
7.4 Challenges, Research Gaps, and Poor Practices
7.5 Conclusion
References
8 Toward Real-Time Federated Evolutionary Neural Architecture Search
8.1 Introduction
8.2 Computationally Efficient Neural Architecture Search
8.3 Evolutionary Optimization of One-Shot Neural Architecture Search
8.3.1 Training the Supernet
8.3.2 Evolutionary Architecture Search
8.3.3 One-Shot NAS in Federated Learning
8.4 Real-Time Federated Evolutionary Neural Architecture Search
8.4.1 Real-Time Evolutionary Neural Architecture Search
8.4.2 Real-Time Federated Evolutionary Neural Architecture Search
8.5 Steady State Evolutionary Federated Neural Architecture Search
8.6 Illustrative Empirical Results
8.6.1 Experimental Settings
8.6.2 Dataset
8.6.3 Real-Time Performances Between NSGA-II and Steady-State NSGA-II
8.6.4 Real-Time Performances on Different Datasets
8.7 Conclusion
References
9 Knowledge Transfer in Genetic Programming Hyper-heuristics
9.1 Introduction
9.2 Background
9.2.1 Genetic Programming Hyper-heuristic
9.2.2 Transfer Learning and Optimisation
9.3 Knowledge Transfer in GPHH for UCARP
9.3.1 Existing GP Transfer Methods
9.3.2 New GPHH with Biased Subtree Transfer
9.3.3 New GPHH with Feature Importance Transfer
9.4 Experimental Studies
9.4.1 Transfer Scenarios
9.4.2 Effectiveness of Existing GP Transfer Methods
9.4.3 Effectiveness of GPHH with Biased Subtree Transfer
9.4.4 Effectiveness of GPHH with Feature Importance Transfer
9.4.5 Statistical Significance Test Results
9.5 Conclusions and Future Work
References
10 Automated Design of Classification Algorithms
10.1 Introduction
10.2 Design Decisions
10.3 Genetic Algorithm (autoGA)
10.3.1 Initial Population Generation
10.3.2 Fitness Evaluation
10.3.3 Selection
10.3.4 Regeneration
10.3.5 GA Parameter Values
10.4 Grammatical Evolution (autoGE)
10.4.1 Initial Population Generation
10.4.2 Fitness Evaluation
10.4.3 Selection
10.4.4 Regeneration
10.4.5 GE Parameter Values
10.5 Experimental Setup
10.5.1 Experiments
10.5.2 Datasets
10.5.3 Technical Specifications
10.6 Results Discussion
10.7 Conclusion
References
11 Automated Design (AutoDes): Current Trends and Future Research Directions
11.1 Introduction
11.1.1 Reusability in AutoDes
11.1.2 Explainable AutoDes
11.1.3 Computational Cost of Algorithm Automation
11.1.4 Theoretical Aspects
11.1.5 AutoDes Standardization
11.1.6 Semi-automated Design
Reference
