A remarkable facet of the human brain is its ability to manage multiple tasks with apparent simultaneity. Knowledge learned from one task can then be used to enhance problem-solving in other related tasks. In machine learning, the idea of leveraging relevant information across related tasks as inductive biases to enhance learning performance has attracted significant interest. In contrast, attempts to emulate the human brain’s ability to generalize in optimization – particularly in population-based evolutionary algorithms – have received little attention to date. Recently, a novel evolutionary search paradigm, Evolutionary Multi-Task (EMT) optimization, has been proposed in the realm of evolutionary computation. In contrast to traditional evolutionary searches, which solve a single task in a single run, evolutionary multi-tasking algorithm conducts searches concurrently on multiple search spaces corresponding to different tasks or optimization problems, each possessing a unique function landscape. By exploiting the latent synergies among distinct problems, the superior search performance of EMT optimization in terms of solution quality and convergence speed has been demonstrated in a variety of continuous, discrete, and hybrid (mixture of continuous and discrete) tasks.
This book discusses the foundations and methodologies of developing evolutionary multi-tasking algorithms for complex optimization, including in domains characterized by factors such as multiple objectives of interest, high-dimensional search spaces and NP-hardness.
Author(s): Liang Feng, Abhishek Gupta, Kay Chen Tan, Yew Soon Ong
Series: Machine Learning: Foundations, Methodologies, and Applications
Publisher: Springer
Year: 2023
Language: English
Pages: 219
City: Singapore
Preface
Contents
Part I Background
1 Introduction
1.1 Optimization
1.2 Evolutionary Optimization
1.3 Evolutionary Multi-Task Optimization
1.4 Organization of the Book
2 Overview and Application-Driven Motivations of Evolutionary Multitasking
2.1 An Overview of EMT Algorithms
2.2 EMT in Real-World Problems
2.2.1 Category 1: EMT in Data Science Pipelines
2.2.2 Category 2: EMT in Evolving Embodied Intelligence
2.2.3 Category 3: EMT in Unmanned Systems Planning
2.2.4 Category 4: EMT in Complex Design
2.2.5 Category 5: EMT in Manufacturing, Operations Research
2.2.6 Category 6: EMT in Software and Services Computing
Part II Evolutionary Multi-Task Optimization for Solving Continuous Optimization Problems
3 The Multi-Factorial Evolutionary Algorithm
3.1 Algorithm Design and Details
3.1.1 Multi-Factorial Optimization
3.1.2 Similarity and Difference Between Multi-factorial Optimization and Multi-Objective Optimization
3.1.3 The Multi-Factorial Evolutionary Algorithm
3.1.3.1 Population Initialization
3.1.3.2 Genetic Mechanisms
3.1.3.3 Selective Evaluation
3.1.3.4 Selection Operation
3.1.3.5 Summarizing the Salient Features of the MFEA
3.2 Empirical Study
3.2.1 Multitasking Across Functions with Intersecting Optima
3.2.2 Multitasking Across Functions with Separated Optima
3.2.3 Discussions
3.3 Summary
4 Multi-Factorial Evolutionary Algorithm with Adaptive Knowledge Transfer
4.1 Algorithm Design and Details
4.1.1 Representative Crossover Operators for Continuous Optimization
4.1.2 Knowledge Transfer via Different Crossover Operators in MFEA
4.1.3 MFEA with Adaptive Knowledge Transfer
4.1.3.1 Adaptive Assortative Mating and Adaptive Vertical Cultural Transmission
4.1.3.2 Adaptation of Transfer Crossover Indicators
4.2 Empirical Study
4.2.1 Experimental Setup
4.2.2 Performance Metric
4.2.3 Results and Discussions
4.2.3.1 Common Multi-Task Benchmarks
4.2.3.2 Complex Multi-Task Problems
4.2.4 Other Issues
4.3 Summary
5 Explicit Evolutionary Multi-Task Optimization Algorithm
5.1 Algorithm Design and Details
5.1.1 Denoising Autoencoder
5.1.2 The Explicit EMT Paradigm
5.1.2.1 Learning of Task Mapping
5.1.2.2 Explicit Genetic Transfer Across Tasks
5.2 Empirical Study
5.2.1 Single-Objective Multi-Task Optimization
5.2.1.1 Experiment Setup
5.2.1.2 Results and Discussions
5.2.2 Multi-Objective Multi-Task Optimization
5.2.2.1 Experiment Setup
5.2.2.2 Results and Discussions
5.3 Summary
Part III Evolutionary Multi-Task Optimization for Solving Combinatorial Optimization Problems
6 Evolutionary Multi-Task Optimization for Generalized Vehicle Routing Problem with Occasional Drivers
6.1 Vehicle Routing Problem with Heterogeneous Capacity, Time Window and Occasional Driver (VRPHTO)
6.1.1 Variants of the Vehicle Routing Problem
6.1.2 Mathematical Formulation for VRPHTO
6.2 Algorithm Design and Details
6.2.1 Evolutionary Multitasking for VRPHTO
6.2.2 Permutation-Based Unified Representation Scheme and Decoding Exemplar
6.2.3 Extended Split Procedure
6.2.4 Routing Information Exchange Across Instances in Evolutionary Multitasking
6.2.5 Chromosome Evaluation in Evolutionary Multitasking
6.3 Empirical Study
6.3.1 Benchmark Generation
6.3.1.1 Generation of VRPHTO Benchmark
6.3.1.2 Generation of Multitasking VRPHTO Benchmark
6.3.2 Experiment Setup
6.3.3 Results and Discussion
6.3.3.1 HVRPTW vs VRPHTO
6.3.3.2 EMA vs SEA
6.4 Summary
7 Explicit Evolutionary Multi-Task Optimization for Capacitated Vehicle Routing Problem
7.1 Capacitated Vehicle Routing Problem (CVPR)
7.2 Algorithm Design and Details
7.2.1 Learning of Mapping Across CVRPs
7.2.2 Knowledge Transfer Across CVRPs
7.2.2.1 Solution Selection
7.2.2.2 Knowledge Learning
7.2.2.3 Knowledge Transfer
7.3 Empirical Study
7.3.1 Experiment Setup
7.3.2 Results and Discussions
7.3.2.1 Solution Quality
7.3.2.2 Search Efficiency: Convergence Trends
7.3.3 Real-World Routing Application: The Package Delivery Problem
7.4 Summary
Part IV Evolutionary Multi-Task Optimization for Solving Large-Scale Optimization Problems
8 Multi-Space Evolutionary Search for Large-Scale Single-Objective Optimization
8.1 Existing Approaches for Simplifying Search Space of Large-Scale Single-Objective Optimization Problems
8.2 Algorithm Design and Details
8.2.1 Construction of the Simplified Problem Space
8.2.2 Learning of Mapping Across Problem Spaces
8.2.3 Knowledge Transfer Across Problem Spaces
8.2.4 Reconstruction of the Simplified Space
8.2.5 Summary of the Multi-Space Evolutionary Search
8.3 Empirical Study
8.3.1 Experimental Setup
8.3.2 Results and Discussion
8.3.2.1 Solution Quality
8.3.2.2 Search Efficiency
8.3.2.3 Sensitivity Study
8.3.3 AI Application of Recommender System
8.4 Summary
9 Multi-Space Evolutionary Search for Large-Scale Multi-Objective Optimization
9.1 Existing Approaches for Large-Scale Evolutionary Multi-Objective Optimization
9.2 Algorithm Design and Details
9.2.1 Outline of the Algorithm
9.2.2 Problem Variation
9.2.2.1 Task Construction
9.2.2.2 Dynamic Task Replacement
9.2.3 Multi-Factorial Evolutionary Search
9.2.3.1 Knowledge Transfer Decision
9.2.3.2 Selective Inheritance
9.3 Empirical Study
9.3.1 Experimental Settings
9.3.1.1 Baseline Methods and Benchmark Problems
9.3.1.2 Parameter Settings
9.3.1.3 Performance Indicator
9.3.2 Performance Comparisons with the State-of-the-Arts
9.3.2.1 IGD Values
9.3.2.2 Convergence Trace
9.3.2.3 Final Non-dominated Solutions
9.3.2.4 Computation Efficiency
9.3.3 Effectiveness of Knowledge Transfer
9.3.4 Parameter Sensitivity Analysis
9.3.4.1 Random Mating Probability (rmp)
9.3.4.2 Interval of Task Replacement (t)
9.3.5 Real-World Application of Neural Network Training Problem
9.4 Summary
References
