Bridge the gap between a high-level understanding of how an algorithm works and knowing the nuts and bolts to tune your models better. This book will give you the confidence and skills when developing all the major machine learning models. In Pro Machine Learning Algorithms, you will first develop the algorithm in Excel so that you get a practical understanding of all the levers that can be tuned in a model, before implementing the models in Python/R.
You will cover all the major algorithms: supervised and unsupervised learning, which include linear/logistic regression; k-means clustering; PCA; recommender system; decision tree; random forest; GBM; and neural networks. You will also be exposed to the latest in deep learning through CNNs, RNNs, and word2vec for text mining. You will be learning not only the algorithms, but also the concepts of feature engineering to maximize the performance of a model. You will see the theory along with case studies, such as sentiment classification, fraud detection, recommender systems, and image recognition, so that you get the best of both theory and practice for the vast majority of the machine learning algorithms used in industry. Along with learning the algorithms, you will also be exposed to running machine-learning models on all the major cloud service providers.
You are expected to have minimal knowledge of statistics/software programming and by the end of this book you should be able to work on a machine learning project with confidence.
What You Will Learn
Get an in-depth understanding of all the major machine learning and deep learning algorithms
Fully appreciate the pitfalls to avoid while building models
Implement machine learning algorithms in the cloud
Follow a hands-on approach through case studies for each algorithm
Gain the tricks of ensemble learning to build more accurate models
Discover the basics of programming in R/Python and the Keras framework for deep learning
Who This Book Is For
Business analysts/ IT professionals who want to transition into data science roles. Data scientists who want to solidify their knowledge in machine learning.
Author(s): Kishore Ayyadevara
Publisher: Apress
Year: 2018
Language: English
Pages: 372
Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Introduction
Chapter 1: Basics of Machine Learning
Regression and Classification
Training and Testing Data
The Need for Validation Dataset
Measures of Accuracy
Absolute Error
Root Mean Square Error
Confusion Matrix
AUC Value and ROC Curve
Unsupervised Learning
Typical Approach Towards Building a Model
Where Is the Data Fetched From?
Which Data Needs to Be Fetched?
Pre-processing the Data
Feature Interaction
Feature Generation
Building the Models
Productionalizing the Models
Build, Deploy, Test, and Iterate
Summary
Chapter 2: Linear Regression
Introducing Linear Regression
Variables: Dependent and Independent
Correlation
Causation
Simple vs. Multivariate Linear Regression
Formalizing Simple Linear Regression
The Bias Term
The Slope
Solving a Simple Linear Regression
More General Way of Solving a Simple Linear Regression
Minimizing the Overall Sum of Squared Error
Solving the Formula
Working Details of Simple Linear Regression
Complicating Simple Linear Regression a Little
Arriving at Optimal Coefficient Values
Introducing Root Mean Squared Error
Running a Simple Linear Regression in R
Residuals
Coefficients
SSE of Residuals (Residual Deviance)
Null Deviance
R Squared
F-statistic
Running a Simple Linear Regression in Python
Common Pitfalls of Simple Linear Regression
Multivariate Linear Regression
Working details of Multivariate Linear Regression
Multivariate Linear Regression in R
Multivariate Linear Regression in Python
Issue of Having a Non-significant Variable in the Model
Issue of Multicollinearity
Mathematical Intuition of Multicollinearity
Further Points to Consider in Multivariate Linear Regression
Assumptions of Linear Regression
Summary
Chapter 3: Logistic Regression
Why Does Linear Regression Fail for Discrete Outcomes?
A More General Solution: Sigmoid Curve
Formalizing the Sigmoid Curve (Sigmoid Activation)
From Sigmoid Curve to Logistic Regression
Interpreting the Logistic Regression
Working Details of Logistic Regression
Estimating Error
Scenario 1
Scenario 2
Least Squares Method and Assumption of Linearity
Running a Logistic Regression in R
Running a Logistic Regression in Python
Identifying the Measure of Interest
Common Pitfalls
Time Between Prediction and the Event Happening
Outliers in Independent variables
Summary
Chapter 4: Decision Tree
Components of a Decision Tree
Classification Decision Tree When There Are Multiple Discrete Independent Variables
Information Gain
Calculating Uncertainty: Entropy
Calculating Information Gain
Uncertainty in the Original Dataset
Measuring the Improvement in Uncertainty
Which Distinct Values Go to the Left and Right Nodes
Gini Impurity
Splitting Sub-nodes Further
When Does the Splitting Process Stop?
Classification Decision Tree for Continuous Independent Variables
Classification Decision Tree When There Are Multiple Independent Variables
Classification Decision Tree When There Are Continuous and Discrete Independent Variables
What If the Response Variable Is Continuous?
Continuous Dependent Variable and Multiple Continuous Independent Variables
Continuous Dependent Variable and Discrete Independent Variable
Continuous Dependent Variable and Discrete, Continuous Independent Variables
Implementing a Decision Tree in R
Implementing a Decision Tree in Python
Common Techniques in Tree Building
Visualizing a Tree Build
Impact of Outliers on Decision Trees
Summary
Chapter 5: Random Forest
A Random Forest Scenario
Bagging
Working Details of a Random Forest
Implementing a Random Forest in R
Parameters to Tune in a Random Forest
Variation of AUC by Depth of Tree
Implementing a Random Forest in Python
Summary
Chapter 6: Gradient Boosting Machine
Gradient Boosting Machine
Working details of GBM
Shrinkage
AdaBoost
Theory of AdaBoost
Working Details of AdaBoost
Additional Functionality for GBM
Implementing GBM in Python
Implementing GBM in R
Summary
Chapter 7: Artificial Neural Network
Structure of a Neural Network
Working Details of Training a Neural Network
Forward Propagation
Applying the Activation Function
Back Propagation
Working Out Back Propagation
Stochastic Gradient Descent
Diving Deep into Gradient Descent
Why Have a Learning Rate?
Batch Training
The Concept of Softmax
Different Loss Optimization Functions
Scaling a Dataset
Scenario Without Scaling the Input
Scenario with Input Scaling
Implementing Neural Network in Python
Avoiding Over-fitting using Regularization
Assigning Weightage to Regularization term
Implementing Neural Network in R
Summary
Chapter 8: Word2vec
Hand-Building a Word Vector
Methods of Building a Word Vector
Issues to Watch For in a Word2vec Model
Frequent Words
Negative Sampling
Implementing Word2vec in Python
Summary
Chapter 9: Convolutional Neural Network
The Problem with Traditional NN
Scenario 1
Scenario 2
Scenario 3
Scenario 4
Understanding the Convolutional in CNN
From Convolution to Activation
From Convolution Activation to Pooling
How Do Convolution and Pooling Help?
Creating CNNs with Code
Working Details of CNN
Deep Diving into Convolutions/Kernels
From Convolution and Pooling to Flattening: Fully Connected Layer
From One Fully Connected Layer to Another
From Fully Connected Layer to Output Layer
Connecting the Dots: Feed Forward Network
Other Details of CNN
Backward Propagation in CNN
Putting It All Together
Data Augmentation
Implementing CNN in R
Summary
Chapter 10: Recurrent Neural Network
Understanding the Architecture
Interpreting an RNN
Working Details of RNN
Time Step 1
Time Step 2
Time Step 3
Implementing RNN: SimpleRNN
Compiling a Model
Verifying the Output of RNN
Implementing RNN: Text Generation
Embedding Layer in RNN
Issues with Traditional RNN
The Problem of Vanishing Gradient
The Problem of Exploding Gradients
LSTM
Implementing Basic LSTM in keras
Implementing LSTM for Sentiment Classification
Implementing RNN in R
Summary
Chapter 11: Clustering
Intuition of clustering
Building Store Clusters for Performance Comparison
Ideal Clustering
Striking a Balance Between No Clustering and Too Much Clustering: K-means Clustering
The Process of Clustering
Working Details of K-means Clustering Algorithm
Applying the K-means Algorithm on a Dataset
Properties of the K-means Clustering Algorithm
Totss (Total Sum of Squares)
Cluster Centers
Tot.withinss
Betweenss
Implementing K-means Clustering in R
Implementing K-means Clustering in Python
Significance of the Major Metrics
Identifying the Optimal K
Top-Down Vs. Bottom-Up Clustering
Hierarchical Clustering
Major Drawback of Hierarchical Clustering
Industry Use-Case of K-means Clustering
Summary
Chapter 12: Principal Component Analysis
Intuition of PCA
Working Details of PCA
Scaling Data in PCA
Extending PCA to Multiple Variables
Implementing PCA in R
Implementing PCA in Python
Applying PCA to MNIST
Summary
Chapter 13: Recommender Systems
Understanding k-nearest Neighbors
Working Details of User-Based Collaborative Filtering
Euclidian Distance
Normalizing for a User
Issue with Considering a Single User
Cosine Similarity
Weighted Average Rating Calculation
Choosing the Right Approach
Calculating the Error
Issues with UBCF
Item-Based Collaborative Filtering
Implementing Collaborative Filtering in R
Implementing Collaborative Filtering in Python
Working Details of Matrix Factorization
Implementing Matrix Factorization in Python
Implementing Matrix Factorization in R
Summary
Chapter 14: Implementing Algorithms in the Cloud
Google Cloud Platform
Microsoft Azure Cloud Platform
Amazon Web Services
Transferring Files to the Cloud Instance
Running Instance Jupyter Notebooks from Your Local Machine
Installing R on the Instance
Summary
Appendix: Basics of Excel, R, and Python
Basics of Excel
Basics of R
Downloading R
Installing and Configuring RStudio
Getting Started with RStudio
Basics of Python
Downloading and installing Python
Basic operations in Python
Numpy
Number generation using Numpy
Slicing and indexing
Pandas
Indexing and slicing using Pandas
Summarizing data
Index
