Multiblock Data Fusion in Statistics and Machine Learning: Applications in the Natural and Life Sciences

Multiblock Data Fusion in Statistics and Machine Learning
Contents
Foreword
Preface
List of Figures
List of Tables
Part I Introductory Concepts and Theory
chapnumcolor1 Introduction
1.1 Scope of the Book
1.2 Potential Audience
1.3 Types of Data and Analyses
1.3.1 Supervised and Unsupervised Analyses
1.3.2 High-, Mid- and Low-level Fusion
1.3.3 Dimension Reduction
1.3.4 Indirect Versus Direct Data
1.3.5 Heterogeneous Fusion
1.4 Examples
1.4.1 Metabolomics
1.4.2 Genomics
1.4.3 Systems Biology
1.4.4 Chemistry
1.4.5 Sensory Science
1.5 Goals of Analyses
1.6 Some History
1.7 Fundamental Choices
1.8 Common and Distinct Components
1.9 Overview and Links
1.10 Notation and Terminology
1.11 Abbreviations
chapnumcolor2 Basic Theory and Concepts
2.i General Introduction
2.1 Component Models
2.1.1 General Idea of Component Models
2.1.2 Principal Component Analysis
2.1.3 Sparse PCA
2.1.4 Principal Component Regression
2.1.5 Partial Least Squares
2.1.6 Sparse PLS
2.1.7 Principal Covariates Regression
2.1.8 Redundancy Analysis
2.1.9 Comparing PLS, PCovR and RDA
2.1.10 Generalised Canonical Correlation Analysis
2.1.11 Simultaneous Component Analysis
2.2 Properties of Data
2.2.1 Data Theory
2.2.2 Scale-types
2.3 Estimation Methods
2.3.1 Least-squares Estimation
2.3.2 Maximum-likelihood Estimation
2.3.3 Eigenvalue Decomposition-based Methods
2.3.4 Covariance or Correlation-based Estimation Methods
2.3.5 Sequential Versus Simultaneous Methods
2.3.6 Homogeneous Versus Heterogeneous Fusion
2.4 Within- and Between-block Variation
2.4.1 Definition and Example
2.4.2 MAXBET Solution
2.4.3 MAXNEAR Solution
2.4.4 PLS2 Solution
2.4.5 CCA Solution
2.4.6 Comparing the Solutions
2.4.7 PLS, RDA and CCA Revisited
2.5 Framework for Common and Distinct Components
2.6 Preprocessing
2.7 Validation
2.7.1 Outliers
2.7.1.1 Residuals
2.7.1.2 Leverage
2.7.2 Model Fit
2.7.3 Bias-variance Trade-off
2.7.4 Test Set Validation
2.7.5 Cross-validation
2.7.6 Permutation Testing
2.7.7 Jackknife and Bootstrap
2.7.8 Hyper-parameters and Penalties
2.8 Appendix
chapnumcolor3 Structure of Multiblock Data
3.i General Introduction
3.1 Taxonomy
3.2 Skeleton of a Multiblock Data Set
3.2.1 Shared Sample Mode
3.2.2 Shared Variable Mode
3.2.3 Shared Variable or Sample Mode
3.2.4 Shared Variable and Sample Mode
3.3 Topology of a Multiblock Data Set
3.3.1 Unsupervised Analysis
3.3.2 Supervised Analysis
3.4 Linking Structures
3.4.1 Linking Structure for Unsupervised Analysis
3.4.2 Linking Structures for Supervised Analysis
3.5 Summary
chapnumcolor4 Matrix Correlations
4.i General Introduction
4.1 Definition
4.2 Most Used Matrix Correlations
4.2.1 Inner Product Correlation
4.2.2 GCD coefficient
4.2.3 RV-coefficient
4.2.4 SMI-coefficient
4.3 Generic Framework of Matrix Correlations
4.4 Generalised Matrix Correlations
4.4.1 Generalised RV-coefficient
4.4.2 Generalised Association Coefficient
4.5 Partial Matrix Correlations
4.6 Conclusions and Recommendations
4.7 Open Issues
Part II Selected Methods for Unsupervised and Supervised Topologies
chapnumcolor5 Unsupervised Methods
5.i General Introduction
5.ii Relations to the General Framework
5.1 Shared Variable Mode
5.1.1 Only Common Variation
5.1.1.1 Simultaneous Component Analysis
5.1.1.2 Clustering and SCA
5.1.1.3 Multigroup Data Analysis
5.1.2 Common, Local, and Distinct Variation
5.1.2.1 Distinct and Common Components
5.1.2.2 Multivariate Curve Resolution
5.2 Shared Sample Mode
5.2.1 Only Common Variation
5.2.1.1 SUM-PCA
5.2.1.2 Multiple Factor Analysis and STATIS
5.2.1.3 Generalised Canonical Analysis
5.2.1.4 Regularised Generalised Canonical Correlation Analysis
5.2.1.5 Exponential Family SCA
5.2.1.6 Optimal-scaling
5.2.2 Common, Local, and Distinct Variation
5.2.2.1 Joint and Individual Variation Explained
5.2.2.2 Distinct and Common Components
5.2.2.3 PCA-GCA
5.2.2.4 Advanced Coupled Matrix and Tensor Factorisation
5.2.2.5 Penalised-ESCA
5.2.2.6 Multivariate Curve Resolution
5.3 Generic Framework
5.3.1 Framework for Simultaneous Unsupervised Methods
5.3.1.1 Description of the Framework
5.3.1.2 Framework Applied to Simultaneous Unsupervised Data Analysis Methods
5.3.1.3 Framework of Common/Distinct Applied to Simultaneous Unsupervised Multiblock Data Analysis Methods
5.4 Conclusions and Recommendations
5.5 Open Issues
chapnumcolor6 ASCA and Extensions
6.i General Introduction
6.ii Relations to the General Framework
6.1 ANOVA-Simultaneous Component Analysis
6.1.1 The ASCA Method
6.1.2 Validation of ASCA
6.1.2.1 Permutation Testing
6.1.2.2 Back-projection
6.1.2.3 Confidence Ellipsoids
6.1.3 The ASCA+ and LiMM-PCA Methods
6.2 Multilevel-SCA
6.3 Penalised-ASCA
6.4 Conclusions and Recommendations
6.5 Open Issues
chapnumcolor7 Supervised Methods
7.i General Introduction
7.ii Relations to the General Framework
7.1 Multiblock Regression: General Perspectives
7.1.1 Model and Assumptions
7.1.2 Different Challenges and Aims
7.2 Multiblock PLS Regression
7.2.1 Standard Multiblock PLS Regression
7.2.2 MB-PLS Used for Classification
7.2.3 Sparse Multiblock PLS Regression (sMB-PLS)
7.3 The Family of SO-PLS Regression Methods (Sequential and Orthogonalised PLS Regression)
7.3.1 The SO-PLS Method
7.3.2 Order of Blocks
7.3.3 Interpretation Tools
7.3.4 Restricted PLS Components and their Application in SO-PLS
7.3.5 Validation and Component Selection
7.3.6 Relations to ANOVA
7.3.7 Extensions of SO-PLS to Handle Interactions Between Blocks
7.3.8 Further Applications of SO-PLS
7.3.9 Relations Between SO-PLS and ASCA
7.4 Parallel and Orthogonalised PLS (PO-PLS) Regression
7.5 Response Oriented Sequential Alternation
7.5.1 The ROSA Method
7.5.2 Validation
7.5.3 Interpretation
7.6 Conclusions and Recommendations
7.7 Open Issues
Part III Methods for Complex Multiblock Structures
chapnumcolor8 Complex Block Structures; with Focus on L-Shape Relations
8.i General Introduction
8.ii Relations to the General Framework
8.1 Analysis of L-shape Data: General Perspectives
8.2 Sequential Procedures for L-shape Data Based on PLS/PCR and ANOVA
8.2.1 Interpretation of X1, Quantitative X2-data, Horizontal Axis First
8.2.2 Interpretation of X1, Categorical X2-data, Horizontal Axis First
8.2.3 Analysis of Segments/Clusters of X1 Data
8.3 The L-PLS Method for Joint Estimation of Blocks in L-shape Data
8.3.1 The Original L-PLS Method, Endo-L-PLS
8.3.2 Exo- Versus Endo-L-PLS
8.4 Modifications of the Original L-PLS Idea
8.4.1 Weighting Information from X3 and X1 in L-PLS Using a Parameter "α
8.4.2 Three-blocks Bifocal PLS
8.5 Alternative L-shape Data Analysis Methods
8.5.1 Principal Component Analysis with External Information
8.5.2 A Simple PCA Based Procedure for Using Unlabelled Data in Calibration
8.5.3 Multivariate Curve Resolution for Incomplete Data
8.5.4 An Alternative Approach in Consumer Science Based on Correlations Between X3 and X1
8.6 Domino PLS and More Complex Data Structures
8.7 Conclusions and Recommendations
8.8 Open Issues
Part IV Alternative Methods for Unsupervised and Supervised Topologies
chapnumcolor9 Alternative Unsupervised Methods
9.i General Introduction
9.ii Relationship to the General Framework
9.1 Shared Variable Mode
9.2 Shared Sample Mode
9.2.1 Only Common Variation
9.2.1.1 DIABLO
9.2.1.2 Generalised Coupled Tensor Factorisation
9.2.1.3 Representation Matrices
9.2.1.4 Extended PCA
9.2.2 Common, Local, and Distinct Variation
9.2.2.1 Generalised SVD
9.2.2.2 Structural Learning and Integrative Decomposition
9.2.2.3 Bayesian Inter-battery Factor Analysis
9.2.2.4 Group Factor Analysis
9.2.2.5 OnPLS
9.2.2.6 Generalised Association Study
9.2.2.7 Multi-Omics Factor Analysis
9.3 Two Shared Modes and Only Common Variation
9.3.1 Generalised Procrustes Analysis
9.3.2 Three-way Methods
9.4 Conclusions and Recommendations
9.4.1 Open Issues
chapnumcolor10 Alternative Supervised Methods
10.i General Introduction
10.ii Relations to the General Framework
10.1 Model and Focus
10.2 Extension of PCovR
10.2.1 Sparse Multiblock Principal Covariates Regression, Sparse PCovR
10.2.2 Multiway Multiblock Covariates Regression
10.3 Multiblock Redundancy Analysis
10.3.1 Standard Multiblock Redundancy Analysis
10.3.2 Sparse Multiblock Redundancy Analysis
10.4 Miscellaneous Multiblock Regression Methods
10.4.1 Multiblock Variance Partitioning
10.4.2 Network Induced Supervised Learning
10.4.3 Common Dimensions for Multiblock Regression
10.5 Modifications and Extensions of the SO-PLS Method
10.5.1 Extensions of SO-PLS to Three-Way Data
10.5.2 Variable Selection for SO-PLS
10.5.3 More Complicated Error Structure for SO-PLS
10.5.4 SO-PLS Used for Path Modelling
10.6 Methods for Data Sets Split Along the Sample Mode, Multigroup Methods
10.6.1 Multigroup PLS Regression
10.6.2 Clustering of Observations in Multiblock Regression
10.6.3 Domain-Invariant PLS, DI-PLS
10.7 Conclusions and Recommendations
10.8 Open Issues
Part V Software
chapnumcolor11 Algorithms and Software
11.1 Multiblock Software
11.2 R package multiblock
11.3 Installing and Starting the Package
11.4 Data Handling
11.4.1 Read From File
11.4.2 Data Pre-processing
11.4.3 Re-coding Categorical Data
11.4.4 Data Structures for Multiblock Analysis
11.4.4.1 Create List of Blocks
11.4.4.2 Create data.frame of Blocks
11.5 Basic Methods
11.5.1 Prepare Data
11.5.2 Modelling
11.5.3 Common Output Elements Across Methods
11.5.4 Scores and Loadings
11.6 Unsupervised Methods
11.6.1 Formatting Data for Unsupervised Data Analysis
11.6.2 Method Interfaces
11.6.3 Shared Sample Mode Analyses
11.6.4 Shared Variable Mode
11.6.5 Common Output Elements Across Methods
11.6.6 Scores and Loadings
11.6.7 Plot From Imported Package
11.7 ANOVA Simultaneous Component Analysis
11.7.1 Formula Interface
11.7.2 Simulated Data
11.7.3 ASCA Modelling
11.7.4 ASCA Scores
11.7.5 ASCA Loadings
11.8 Supervised Methods
11.8.1 Formatting Data for Supervised Analyses
11.8.2 Multiblock Partial Least Squares
11.8.2.1 MB-PLS Modelling
11.8.2.2 MB-PLS Summaries and Plotting
11.8.3 Sparse Multiblock Partial Least Squares
11.8.3.1 Sparse MB-PLS Modelling
11.8.3.2 Sparse MB-PLS Plotting
11.8.4 Sequential and Orthogonalised Partial Least Squares
11.8.4.1 SO-PLS Modelling
11.8.4.2 Mge Plot
11.8.4.3 SO-PLS Loadings
11.8.4.4 SO-PLS Scores
11.8.4.5 SO-PLS Prediction
11.8.4.6 SO-PLS Validation
11.8.4.7 Principal Components of Predictions
11.8.4.8 CVANOVA
11.8.5 Parallel and Orthogonalised Partial Least Squares
11.8.5.1 PO-PLS Modelling
11.8.5.2 PO-PLS Scores and Loadings
11.8.6 Response Optimal Sequential Alternation
11.8.6.1 ROSA Modelling
11.8.6.2 ROSA Loadings
11.8.6.3 ROSA Scores
11.8.6.4 ROSA Prediction
11.8.6.5 ROSA Validation
11.8.6.6 ROSA Image Plots
11.8.7 Multiblock Redundancy Analysis
11.8.7.1 MB-RDA Modelling
11.8.7.2 MB-RDA Loadings and Scores
11.9 Complex Data Structures
11.9.1 L-PLS
11.9.1.1 Simulated L-shaped Data
11.9.1.2 Exo-L-PLS
11.9.1.3 Endo-L-PLS
11.9.1.4 L-PLS Cross-validation
11.9.2 SO-PLS-PM
11.9.2.1 Single SO-PLS-PM Model
11.9.2.2 Multiple Paths in an SO-PLS-PM Model
11.10 Software Packages
11.10.1 R Packages
11.10.2 MATLAB Toolboxes
11.10.3 Python
11.10.4 Commercial Software
References
Index
EULA