Focusing on comprehensive comparisons of the performance of stochastic optimization algorithms, this book provides an overview of the current approaches used to analyze algorithm performance in a range of common scenarios, while also addressing issues that are often overlooked. In turn, it shows how these issues can be easily avoided by applying the principles that have produced Deep Statistical Comparison and its variants. The focus is on statistical analyses performed using single-objective and multi-objective optimization data. At the end of the book, examples from a recently developed web-service-based e-learning tool (DSCTool) are presented. The tool provides users with all the functionalities needed to make robust statistical comparison analyses in various statistical scenarios.The book is intended for newcomers to the field and experienced researchers alike. For newcomers, it covers the basics of optimization and statistical analysis, familiarizing them with the subject matter before introducing the Deep Statistical Comparison approach. Experienced researchers can quickly move on to the content on new statistical approaches. The book is divided into three parts:
Part I: Introduction to optimization, benchmarking, and statistical analysis – Chapters 2-4.
Part II: Deep Statistical Comparison of meta-heuristic stochastic optimization algorithms – Chapters 5-7.
Part III: Implementation and application of Deep Statistical Comparison – Chapter 8.
Author(s): Tome Eftimov, Peter Korošec
Series: Natural Computing Series
Publisher: Springer
Year: 2022
Language: English
Pages: 140
City: Cham
Foreword
Preface
Book Content
Intended Audience
Expected Background Knowledge
Book Outline
Acknowledgments
Contents
Acronyms
1 Introduction
1.1 Motivation
1.2 Meta-heuristic Stochastic Optimization
1.3 Benchmarking Theory
1.4 Introduction to Statistical Analysis
1.5 Approaches to Statistical Comparisons Used for Stochastic Optimization Algorithms
1.6 The Deep Statistical Comparison in Single-Objective Optimization
1.7 The Deep Statistical Comparison in Multi-objective Optimization
1.8 DSCTool—A Web-Service-Based e-Learning Tool
2 Meta-heuristic Stochastic Optimization
2.1 Optimization
2.1.1 Combinatorial Optimization
2.1.2 Numerical Optimization
2.2 Optimization Problem
2.2.1 Single-Objective Optimization Problem
2.2.2 Multi-objective Optimization Problem
2.3 Heuristics
2.3.1 Exact Heuristics
2.3.2 Stochastic Heuristics
2.4 Meta-heuristics
2.5 Summary
3 Benchmarking Theory
3.1 Benchmarking
3.2 Objectives of Benchmarking
3.3 Problem Selection
3.3.1 Real-World Problems
3.3.2 Artificial Problems
3.4 Algorithm Selection
3.5 Experimental Design
3.6 Statistical Analysis
3.7 Summary
4 Introduction to Statistical Analysis
4.1 Statistical Analysis
4.2 Exploratory Data Analysis
4.3 Hypothesis Testing
4.3.1 Parametric Versus Non-parametric Statistical Tests
4.3.2 Statistical Scenarios
4.4 Benchmarking Scenarios
4.5 Guidelines to Select a Relevant Omnibus Statistical Test
4.6 Summary
5 Approaches to Statistical Comparisons Used for Stochastic Optimization Algorithms
5.1 Most Commonly Used Approach
5.1.1 Single-Problem Scenario
5.1.2 Multiple-problem Scenario
5.2 Deep Statistical Comparison Approach
5.2.1 Deep Statistical Comparison Ranking Scheme for One-Dimensional Data
5.2.2 Single-Problem Analysis
5.2.3 Multiple-problem Analysis
5.3 Summary
6 Deep Statistical Comparison in Single-Objective Optimization
6.1 Statistical Significance
6.1.1 Deep Statistical Comparison Ranking Scheme
6.1.2 Examples
6.2 Practical Significance
6.2.1 Practical Deep Statistical Comparison Ranking Scheme
6.2.2 Examples
6.3 Extended Deep Statistical Comparison
6.3.1 Extended Deep Statistical Comparison Ranking Scheme
6.3.2 Examples
6.4 Summary
7 Deep Statistical Comparison in Multi-Objective Optimization
7.1 Single-Quality-Indicator Analysis
7.1.1 Examples
7.2 Ensemble of Quality Indicator Analysis
7.2.1 Average Ensemble
7.2.2 Hierarchical Majority Vote Ensemble
7.2.3 Data-Driven Ensemble
7.2.4 Examples
7.3 Multi-objective Deep Statistical Comparison
7.3.1 Multi-Objective Deep Statistical Comparison Ranking Scheme
7.3.2 Sensitivity Analysis of the Multivariate mathcalE-test
7.3.3 Examples
7.4 Summary
8 DSCTool—A Web-Service-Based e-Learning Tool
8.1 DSCTool
8.2 REST Web Services
8.2.1 Registration Service
8.2.2 Ranking Service
8.2.3 Multivariate Service
8.2.4 Ensemble Service
8.2.5 Multi-objective Service
8.2.6 Omnibus Service
8.2.7 Post-Hoc Service
8.3 Clients
8.3.1 REST API Clients
8.3.2 API Implementations in R
8.4 Examples
8.5 Summary
Appendix References
Index
