Regularization is an infallible way to produce accurate results with unseen data, however, applying regularization is challenging as it is available in multiple forms and applying the appropriate technique to every model is a must. The Regularization Cookbook provides you with the appropriate tools and methods to handle any case, with ready-to-use working codes as well as theoretical explanations.
After an introduction to regularization and methods to diagnose when to use it, you’ll start implementing regularization techniques on linear models, such as linear and logistic regression, and tree-based models, such as random forest and gradient boosting. You’ll then be introduced to specific regularization methods based on data, high cardinality features, and imbalanced datasets. In the last five chapters, you’ll discover regularization for deep learning models. After reviewing general methods that apply to any type of neural network, you’ll dive into more NLP-specific methods for RNNs and transformers, as well as using BERT or GPT-3. By the end, you’ll explore regularization for computer vision, covering CNN specifics, along with the use of generative models such as stable diffusion and Dall-E.
By the end of this book, you’ll be armed with different regularization techniques to apply to your ML and DL models.
Author(s): Vincent Vandenbussche
Edition: 1
Publisher: Packt Publishing Ltd.
Year: 2023
Language: English
Pages: 905
The Regularization Cookbook
Foreword
Contributors
About the author
About the reviewer
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Conventions used
Sections
Getting ready
How to do it…
How it works…
There’s more…
See also
Get in touch
Share Your Thoughts
Download a free PDF copy of this book
1
An Overview of Regularization
Technical requirements
Introducing regularization
Examples of models that did not pass the deployment test
Intuition about regularization
Key concepts of regularization
Bias and variance
Underfitting and overfitting
Regularization – from overfitting to underfitting
Unavoidable bias
Diagnosing bias and variance
Regularization – a multi-dimensional problem
Summary
2
Machine Learning Refresher
Technical requirements
Loading data
Getting ready
How to do it…
There’s more…
See also
Splitting data
Getting ready
How to do it…
See also
Preparing quantitative data
Getting ready
How to do it…
There’s more…
See also
Preparing qualitative data
Getting ready
How to do it…
There’s more…
See also
Training a model
Getting ready
How to do it…
See also
Evaluating a model
Getting ready
How to do it…
See also
Performing hyperparameter optimization
Getting ready
How to do it…
3
Regularization with Linear Models
Technical requirements
Training a linear regression model with scikit-learn
Getting ready
How to do it…
There’s more…
See also
Regularizing with ridge regression
Getting ready
How to do it…
There’s more…
See also
Regularizing with lasso regression
Getting ready
How to do it…
There’s more…
See also
Regularizing with elastic net regression
Getting ready
How to do it…
See also
Training a logistic regression model
Getting ready
How to do it…
Regularizing a logistic regression model
Getting ready
How to do it…
There’s more…
Choosing the right regularization
Getting ready
How to do it…
See also
4
Regularization with Tree-Based Models
Technical requirements
Building a classification tree
Disorder measurement
Loss function
Getting ready
How to do it…
There’s more…
See also
Building regression trees
Getting ready
How to do it…
See also
Regularizing a decision tree
Getting ready
How to do it…
How it works…
There’s more…
See also
Training the Random Forest algorithm
Getting ready
How to do it…
See also
Regularization of Random Forest
Getting started
How to do it…
Training a boosting model with XGBoost
Getting ready
How to do it…
See also
Regularization with XGBoost
Getting ready
How to do it…
There’s more…
5
Regularization with Data
Technical requirements
Hashing high cardinality features
Getting started
How to do it...
See also
Aggregating features
Getting ready
How to do it...
There’s more...
Undersampling an imbalanced dataset
Getting ready
How to do it...
There’s more...
See also
Oversampling an imbalanced dataset
Getting ready
How to do it...
There’s more...
See also
Resampling imbalanced data with SMOTE
Getting ready
How to do it...
There’s more...
See also
6
Deep Learning Reminders
Technical requirements
Training a perceptron
Getting started
How to do it…
There’s more…
See also
Training a neural network for regression
Getting started
How to do it…
There’s more…
See also
Training a neural network for binary classification
Getting ready
How to do it…
There’s more…
See also
Training a multiclass classification neural network
Getting ready
How to do it…
There’s more…
See also
7
Deep Learning Regularization
Technical requirements
Regularizing a neural network with L2 regularization
Getting ready
How to do it...
There’s more...
See also
Regularizing a neural network with early stopping
Getting ready
How to do it...
There’s more...
Regularization with network architecture
Getting ready
How to do it...
There’s more...
Regularizing with dropout
Getting ready
How to do it...
There’s more...
See also
8
Regularization with Recurrent Neural Networks
Technical requirements
Training an RNN
Getting started
How to do it…
There’s more…
See also
Training a GRU
Getting started
How to do it…
There’s more…
See also
Regularizing with dropout
Getting ready
How to do it…
There’s more…
Regularizing with the maximum sequence length
Getting ready
How to do it…
There’s more…
9
Advanced Regularization in Natural Language Processing
Technical requirements
Regularization using a word2vec embedding
Getting ready
How to do it…
There’s more…
See also
Data augmentation using word2vec
Getting ready
How to do it…
There’s more…
See also
Zero-shot inference with pre-trained models
Getting ready
How to do it…
There’s more…
See also
Regularization with BERT embeddings
Getting ready
How to do it…
There’s more…
See also
Data augmentation using GPT-3
Getting ready
How to do it…
There’s more…
See also
10
Regularization in Computer Vision
Technical requirements
Training a CNN
Getting started
How to do it…
There’s more…
See also
Regularizing a CNN with vanilla NN methods
Getting started
How to do it…
There’s more…
See also
Regularizing a CNN with transfer learning for object detection
Object detection
Mean average precision
COCO dataset
Getting started
How to do it…
There’s more…
See also
Semantic segmentation using transfer learning
Getting started
How to do it…
There’s more…
See also
11
Regularization in Computer Vision – Synthetic Image Generation
Technical requirements
Applying image augmentation with Albumentations
Spatial-level augmentation
Pixel-level augmentation
Albumentations
Getting started
How to do it…
There’s more…
See also
Creating synthetic images for object detection
Getting started
How to do it…
There’s more…
See also
Implementing real-time style transfer
Stable Diffusion
Perceptual loss
Getting started
How to do it…
There’s more…
See also
Index
Why subscribe?
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts
Download a free PDF copy of this book
