This book introduces readers to the fundamentals of and recent advances in federated learning, focusing on reducing communication costs, improving computational efficiency, and enhancing the security level. Federated learning is a distributed machine learning paradigm which enables model training on a large body of decentralized data. Its goal is to make full use of data across organizations or devices while meeting regulatory, privacy, and security requirements.
The book starts with a self-contained introduction to artificial neural networks, deep learning models, supervised learning algorithms, evolutionary algorithms, and evolutionary learning. Concise information is then presented on multi-party secure computation, differential privacy, and homomorphic encryption, followed by a detailed description of federated learning. In turn, the book addresses the latest advances in federate learning research, especially from the perspectives of communication efficiency, evolutionary learning, and privacy preservation. The book is particularly well suited for graduate students, academic researchers, and industrial practitioners in the field of machine learning and artificial intelligence. It can also be used as a self-learning resource for readers with a science or engineering background, or as a reference text for graduate courses.
Author(s): Yaochu Jin, Hangyu Zhu, Jinjin Xu, Yang Chen
Series: Machine Learning: Foundations, Methodologies, and Applications
Publisher: Springer
Year: 2022
Language: English
Pages: 226
City: Singapore
Preface
Contents
1 Introduction
1.1 Artificial Neural Networks and Deep Learning
1.1.1 A Brief History of Artificial Intelligence
1.1.2 Multi-layer Perceptrons
1.1.3 Convolutional Neural Networks
1.1.4 Long Short-Term Memory
1.1.5 Decision Trees
1.1.6 Gradient-Based Methods
1.2 Evolutionary Optimization and Learning
1.2.1 Optimization and Learning
1.2.2 Genetic Algorithms
1.2.3 Genetic Programming
1.2.4 Evolutionary Multi-objective Optimization
1.2.5 Evolutionary Multi-objective Learning
1.2.6 Evolutionary Neural Architecture Search
1.3 Privacy-Preserving Computation
1.3.1 Secure Multi-party Computation
1.3.2 Differential Privacy
1.3.3 Homomorphic Encryption
1.4 Federated Learning
1.4.1 Horizontal and Vertical Federated Learning
1.4.2 Federated Averaging
1.4.3 Federated Transfer Learning
1.4.4 Federated Learning Over Non-IID Data
1.5 Summary
References
2 Communication Efficient Federated Learning
2.1 Communication Cost in Federated Learning
2.2 Main Methodologies
2.2.1 Non-IID/IID Data and Dataset Shift
2.2.2 Non-identical Client Distributions
2.2.3 Violations of Independence
2.3 Temporally Weighted Averaging and Layer-Wise Weight Update
2.3.1 Temporally Weighted Averaging
2.3.2 Layer-Wise Asynchronous Weight Update
2.3.3 Empirical Studies
2.4 Trained Ternary Compression for Federated Learning
2.4.1 Binary and Ternary Compression
2.4.2 Trained Ternary Compression
2.4.3 Trained Ternary Compression for Federated Learning
2.4.4 Theoretical Analysis
2.4.5 Empirical Studies
2.5 Summary
References
3 Evolutionary Multi-objective Federated Learning
3.1 Motivations and Challenges
3.2 Offline Evolutionary Multi-objective Federated Learning
3.2.1 Sparse Network Encoding with a Random Graph
3.2.2 Evolutionary Multi-objective Neural Architecture Search
3.2.3 Overall Framework
3.2.4 Empirical Results
3.3 Real-Time Evolutionary Federated Neural Architecture Search
3.3.1 Network Architecture Encoding Based On Supernet
3.3.2 Network Sampling and Client Sampling
3.3.3 Overall Framework
3.3.4 Empirical Studies
3.4 Summary
References
4 Secure Federated Learning
4.1 Threats to Federated Learning
4.2 Distributed Encryption for Horizontal Federated Learning
4.2.1 Distributed Data Encryption
4.2.2 Federated Encryption and Decryption
4.2.3 Ternary Quantization and Approximate Aggregation
4.2.4 Overall Framework
4.2.5 Empirical Studies
4.3 Secure Vertical Federated Learning
4.3.1 Vertical Federated Learning with XGBoost
4.3.2 Secure Node Split and Construction
4.3.3 Partial Differential Privacy
4.3.4 Security Analysis
4.3.5 Empirical Studies
4.4 Summary
References
5 Summary and Outlook
5.1 Summary
5.2 Future Directions
Index
