Statistical Modeling in Machine Learning: Concepts and Applications

Front Cover
Statistical Modeling in Machine Learning
Statistical Modeling in Machine Learning
Copyright
Dedication
Contents
Contributors
Editors' biographies
Preface
Acknowledgments
1 - Introduction to statistical modeling in machine learning: a case study
1.1 Introduction
1.1.1 Machine-learning research in early age
1.1.2 Ensemble machine-learning technique
1.2 Classification of algorithms in machine learning
1.3 Regression algorithms in machine learning
1.4 Case study: prison crowding prediction
1.4.1 Methods and material
1.4.2 Data collection
1.4.3 Data preprocessing
1.4.4 Proposed regression-based prison overcrowding prediction model (RBPOPM)
1.5 Result and discussion
1.6 Conclusion
References
2 - A technique of data collection: web scraping with python
2.1 Introduction
2.2 Basics of web scraping
2.2.1 Definition
2.2.2 Why do we need to scrape data?
2.2.3 Choice of programming language
2.2.4 Ethics behind web scraping
2.3 Elements of web scraping
2.3.1 Architecture of web scraping
2.3.2 Components of web scraping
2.4 An implementation walkthrough
2.4.1 Libraries for web scraping
2.4.1.1 Scrapy
2.4.1.2 Selenium
2.4.1.3 Beautiful Soup
2.4.2 Importing required libraries
2.4.3 Accessing website data
2.4.3.1 Output
2.4.4 Web crawling
2.4.5 Data extraction
2.4.5.1 Output
2.4.6 Data framing
2.4.6.1 Output
2.4.7 Stages of data transformation
2.5 Web scraping in reality
2.6 Conclusion
References
3 - Analysis of Covid-19 using machine learning techniques
3.1 Introduction
3.2 Literature survey
3.3 Study of algorithms
3.3.1 Multiple regression
3.3.1.1 Ordinary least squares (OLS)
3.3.1.2 Multiple regression analysis has two main uses
3.3.1.3 Collinearity
3.3.1.4 Hypothesis test
3.3.1.5 Feature selection
3.3.1.6 Coefficient of determination
3.3.2 Logistic regression
3.3.3 Support vector machine
3.3.4 Building a decision tree
3.3.5 Random forest regressor
3.4 Experimental analysis and results
3.4.1 Analysis for multiple linear regression []
3.4.1.1 Select the input variable and output variable
3.4.1.2 Splitting the data set
3.4.1.3 Build and train the predictor
3.4.1.4 Evaluate performance
3.4.1.5 Results of the hypothesis test
3.4.1.6 Understanding the outputs of the model from the above OLS regression results: is this statistically significant?
3.4.1.6.1 What is the significance of p-value?
3.4.1.6.2 R squared and adjusted R squared?
3.4.1.6.3 Checking for errors
3.4.1.7 Forward selection
3.4.1.8 Visualization plots for each symptom against the target variable
3.4.1.8.1 Difficulty in breathing symptom versus infected
3.4.1.8.2 High fever symptom versus infected
3.4.1.8.3 Dry Cough versus infected
3.4.1.8.4 Sore throat versus infected
3.4.2 Evaluation of machine learning algorithms
3.5 Conclusion and future study
References
Further reading
4 - Discriminative dictionary learning based on statistical methods
4.1 Introduction
4.1.1 Regularization and dimension reduction
4.1.2 Sparse representation (SR)
4.2 Notation
4.3 Sparse coding methods
4.3.1 Importance of statistical concepts in sparse coding methods
4.4 Dictionary learning
4.4.1 Orthogonal dictionary learning
4.4.2 Overcomplete dictionary learning
4.4.3 Structured dictionary learning
4.4.4 Unsupervised DL algorithms
4.4.5 Supervised DL algorithms
4.5 Statistical concepts in dictionary learning
4.5.1 Histogram of oriented gradients (HoG)
4.5.2 Use of correlation analysis in dictionary learning
4.6 Parametric approaches to estimation of dictionary parameters
4.6.1 Hidden Markov model (HMM): discriminative dictionary learning
4.7 Nonparametric approaches to discriminative DL
4.7.1 UHTelPCC
4.7.2 MNIST
4.8 Conclusion
References
5 - Artificial intelligence–based uncertainty quantification technique for external flow computational fluid dynamic (CFD) ...
5.1 Introduction
5.2 Formulation
5.2.1 Governing equations and model for compressible flow over missile
5.2.2 Evolutionary neural architecture Search strategy
5.2.3 Determination of sample size for training the ANN
5.3 Results and discussions
5.4 Conclusions
Acknowledgments
References
6 - Contrast between simple and complex classification algorithms
6.1 Introduction
6.2 Data preprocessing and feature extraction
6.2.1 Data preprocessing
6.2.2 Feature study
6.2.2.1 Chroma
6.2.2.2 Root-mean-square (RMS)
6.2.2.3 Centroid
6.2.2.4 Bandwidth
6.2.2.5 Zero-crossing rate
6.2.2.6 Roll-off
6.2.2.7 Tempo
6.2.2.8 Mel-frequency cepstral coefficients
6.2.2.9 Harmony
6.3 Data modeling
6.3.1 Fitting linear discriminant analysis
6.3.2 Fitting quadratic discriminant analysis
6.3.3 Fitting k-nearest neighbors
6.3.4 Feedforward neural networks
6.3.5 Fitting feedforward neural networks
6.3.5.1 Gradient descent
6.3.5.2 Activation functions
6.3.5.3 Regularization
6.3.5.4 Model building
6.4 Conclusion
References
7 - Classification model of machine learning for medical data analysis
7.1 Introduction
7.2 Machine learning techniques for diseases detection
7.2.1 Logistic regression (LR)
7.2.2 Decision tree
7.2.2.1 Build our decision tree
7.2.2.2 Advantage
7.2.2.3 Disadvantage
7.2.3 Random forest
7.2.4 Support vector machine
7.2.5 Radial basis function neural network (RBFNN)
7.2.6 Deep learning
7.3 Disease detected by machine learning techniques
7.3.1 Glaucoma and diabetic retinopathy
7.3.2 Brain tumor
7.3.3 Breast cancer
7.3.4 Heart disease
7.3.5 Multimodal classification
7.4 Challenges in ML based classification for medical data
7.4.1 Data
7.4.2 Selection of algorithm
7.4.3 Overfitting
7.4.4 Underfitting
7.5 Conclusion
References
8 - Regression tasks for machine learning
8.1 Introduction
8.2 Steps in statistical modeling
8.2.1 Regression models and classification models
8.2.2 Regression model
8.2.3 Classification model
8.3 General linear regression model
8.4 Simple linear regression (SLR)
8.4.1 Estimate the regression parameters
8.5 Authentication of the simple linear regression model
8.5.1 Squared R
8.5.2 Interpretation of squared R
8.5.3 Hypothesis tests to the regression coefficients and p values
8.5.4 Inclusion/exclusion of explanatory variable decision
8.6 Multiple linear regression
8.7 Polynomial regression
8.8 Implementation using R programming
8.9 Conclusion
References
9 - Model selection and regularization
9.1 Introduction
9.2 Subset selection
9.2.1 Best subset selection
9.2.2 Stepwise selection
9.3 Regularization
9.4 Shrinkage methods
9.4.1 Ridge Regression
9.4.1.1 Bias, mean square error, and L2-Risk of RRE
9.4.1.2 Graphical representation of RRE
9.4.2 Lasso Regression
9.4.2.1 Computation of the Lasso solution
9.5 Dimensional reduction
9.5.1 How many principal components?
9.6 Implementation of Ridge and Lasso Regression
9.7 Conclusion
References
10 - Data clustering using unsupervised machine learning
10.1 Introduction
10.2 Techniques in unsupervised learning
10.2.1 Issues with unsupervised learning
10.2.2 Why is unsupervised learning needed despite these issues?
10.2.3 Applications of unsupervised learning
10.3 Unsupervised clustering
10.3.1 Hierarchical clustering
10.3.2 Partitional clustering
10.3.3 Latent variable models for clustering
10.3.4 Dimensionality reduction
10.3.5 The search-based clustering approaches
10.3.6 Bayesian clustering
10.3.7 Spectral clustering
10.4 Taxonomy of neural network-based deep clustering
10.4.1 Autoencoder (AE) based deep clustering
10.4.2 CDNN based deep clustering
10.4.3 Variational AE-based deep clustering
10.4.4 GAN-based deep clustering
10.5 Cluster evolution criteria
10.5.1 Similarity measurements
10.5.1.1 Lp and L1 based distance measurements
10.5.2 Internal quality criteria
10.5.2.1 Sum of squared error (SSE)
10.5.2.2 Scatter criteria
10.5.2.3 Condorcet's criterion
10.5.3 External quality standards
10.5.4 Clustering loss
10.5.5 Nonclustering loss
10.6 Applications of clustering
10.7 Feature selection with ML for clustering
10.7.1 Unsupervised filter model
10.7.2 Unsupervised wrapper model
10.7.3 Challenges
10.8 Classification in ML: challenges and research issues
10.9 Key findings and open challenges
10.10 Conclusion
References
11 - Emotion-based classification through fuzzy entropy-enhanced FCM clustering
11.1 Introduction
11.2 Related work
11.3 Emotion-based models
11.3.1 Ekman's emotions
11.3.2 The Plutchik's main emotions
11.3.3 The Hourglass of emotion
11.4 Theoretical background
11.4.1 Fuzzy entropy and fuzziness
11.4.2 FCM and weighted FCM algorithms
11.4.3 Entropy-based FCM algorithm
11.5 Logical design model
11.6 Experimental results
11.7 Conclusion
References
12 - Fundamental optimization methods for machine learning
12.1 Introduction
12.2 First-order optimization methods
12.2.1 Gradient descent
12.2.2 Gradient descent variants
12.2.2.1 Batch gradient descent
12.2.2.2 Stochastic gradient descent
12.2.2.3 Mini-batch gradient descent
12.2.3 Gradient descent optimization algorithms
12.2.3.1 Momentum
12.2.3.2 Nesterov accelerated gradient
12.2.3.3 AdaGrad
12.2.3.4 AdaDelta
12.2.3.5 Adaptive moment estimation
12.2.3.6 AdaMax
12.2.3.7 Nesterov accelerated adaptive moment estimation
12.3 High-order optimization method
12.3.1 Hessian-free Newton method
12.3.2 Quasi-Newton method
12.3.3 Gauss-Newton method
12.3.4 Natural gradient method
12.4 Derivative-free optimization methods
12.4.1 Methods for convex objective
12.4.2 Methods for stochastic optimization
12.5 Optimization methods challenges and issues in machine learning
12.6 Conclusion
References
13 - Stochastic optimization of industrial grinding operation through data-driven robust optimization
13.1 Introduction
13.2 Optimization under uncertainty
13.2.1 Brief overview of robust optimization
13.2.2 Uncertainty set: a paramount element during calculations of statistical terms in RO
13.2.3 Description of FCM: unsupervised ML approach
13.2.4 Issues in conventional FCM approach
13.3 DDRO: data-driven robust optimization for grinding model
13.3.1 Industrial grinding model: description
13.3.2 Formulation of optimization problem under uncertainty problem: grinding model
13.3.3 ANN assisted fuzzy C-means clustering technique
13.3.4 Generative modeling framework in the identified clusters
13.4 Results and discussions
13.5 Conclusion
Acknowledgments
References
14 - Dimensionality reduction using PCAs in feature partitioning framework
14.1 Introduction
14.2 Principal component analysis (PCA)
14.3 PCAs in feature partitioning framework
14.3.1 What is feature partitioning framework?
14.3.2 Subpattern principal component analysis (SpPCA)
14.3.3 Cross-correlation subpattern principal component analysis (SubXPCA)
14.3.4 Hybrid principal component analysis (HPCA)
14.3.4.1 Procedure to extract hybrid PCA features from 2D face images
Local pattern feature extraction
Global pattern feature extraction
Hybrid pattern feature extraction using ESpPCA
Hybrid pattern feature extraction using ESubXPCA
14.3.5 Pattern reconstruction
14.3.6 Theoretical analysis
14.3.6.1 Summarization of variance
14.3.6.2 Time complexity analysis
14.3.6.3 Space complexity analysis
14.4 Summary
Acknowledgments
References
15 - Impact of Midday Meal Scheme in primary schools in India using exploratory data analysis and data visualization
15.1 Introduction and background
15.2 Nutrition in primary schools in rural India
15.2.1 Malnutrition
15.3 Midday Meal Scheme
15.3.1Objectives
15.4 Exploratory data analysis and visualization methodology
15.4.1 Stacked-bar plot
15.4.1.1 Mean percent of anemia in non-MDM versus MDM states
15.4.1.2 Mean percent of Stunt-Growth in non-MDM versus MDM states
15.4.1.3 Mean percent of enrollment in non-MDM and MDM states
15.4.2 Box plot
15.4.2.1 Quartiles and outliers of anemia in non-MDM and MDM states
15.4.2.2 Quartiles and outliers of Stunt-Growth in non-MDM and MDM states
15.4.2.3 Quartiles and outliers of enrollment in non-MDM and MDM states
15.4.3 Violin plot
15.4.3.1 Density plot distribution of anemia in non-MDM and MDM states
15.4.3.2 Density plot distribution of Stunt-Growth in non-MDM and MDM states
15.4.3.3 Density plot distribution of enrollment in non-MDM and MDM states
15.4.4 Scatter plot
15.4.4.1 Prevalence of anemia for all MDM states in the year 2005–2006 (in India)
15.4.4.2 Prevalence of anemia for all MDM states in 2015–2016 (in India)
15.4.4.3 Percentage of Stunt-Growth for all MDM states in the year 2016–2018 (in India)
15.4.4.4 Percentage of enrollment for all MDM states in the year 2005–2006 (in India)
15.4.4.5 Percentage of enrollment for all MDM states in 2015–2016 (in India)
15.4.5 Heatmap
15.5 Data visualization insights on impact of MDM
15.6 Conclusion
References
16 - Nonlinear system identification of environmental pollutants using recurrent neural networks and Global Sensitivity Ana ...
16.1 Introduction
16.2 Formulation
16.2.1 Environmental pollutants data
16.2.2 Algorithm for design of RNNs
16.2.3 Global Sensitivity Analysis
16.3 Results and discussions
16.4 Conclusions
Acknowledgments
References
17 - Comparative study of automated deep learning techniques for wind time-series forecasting
17.1 Introduction
17.2 Formulation
17.2.1 Data description and analysis
17.2.2 Techniques for modeling the data
17.2.2.1 ADAM optimizer
17.2.2.2 Recurrent Neural Networks
17.2.2.3 Long short-term memory networks
17.2.3 Design of optimal networks
17.3 Results
17.3.1 Time-series analysis and decomposition
17.3.2 Optimal design of RNNs and LSTMs
17.3.3 Significance of predictions in wind farm studies
17.4 Conclusions
Acknowledgments
References
Index
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
Back Cover