Learn how to use PyTorch to build neural network models using code snippets updated for this second edition. This book includes new chapters covering topics such as distributed PyTorch modeling, deploying PyTorch models in production, and developments around PyTorch with updated code.
You'll start by learning how to use tensors to develop and fine-tune neural network models and implement deep learning models such as LSTMs, and RNNs. Next, you'll explore probability distribution concepts using PyTorch, as well as supervised and unsupervised algorithms with PyTorch. This is followed by a deep dive on building models with convolutional neural networks, deep neural networks, and recurrent neural networks using PyTorch.
This new edition covers also topics such as Scorch, a compatible module equivalent to the Scikit machine learning library, model quantization to reduce parameter size, and preparing a model for deployment within a production system. Distributed parallel processing for balancing PyTorch workloads, using PyTorch for image processing, audio analysis, and model interpretation are also covered in detail. Each chapter includes recipe code snippets to perform specific activities.
By the end of this book, you will be able to confidently build neural network models using PyTorch.
PyTorch is a recent entrant to the league of graph computation tools/programming languages. Addressing the limitations of previous frameworks, PyTorch promises a better user experience in the deployment of deep learning models and the creation of advanced models using a combination of convolutional neural networks, recurrent neural networks, LSTMs, and deep neural networks.
What You Will Learn
Utilize new code snippets and models to train machine learning models using PyTorch
Train deep learning models with fewer and smarter implementations
Explore the PyTorch framework for model explainability and to bring transparency to model interpretation
Build, train, and deploy neural network models designed to scale with PyTorch
Understand best practices for evaluating and fine-tuning models using PyTorch
Use advanced torch features in training deep neural networks
Explore various neural network models using PyTorch
Discover functions compatible with sci-kit learn compatible models
Perform distributed PyTorch training and execution
Who This Book Is For
Machine learning engineers, data scientists and Python programmers and software developers interested in learning the PyTorch framework.
Author(s): Pradeepta Mishra
Edition: 2
Publisher: Apress
Year: 2023
Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Introduction
Chapter 1: Introduction to PyTorch, Tensors, and Tensor Operations
What Is PyTorch?
PyTorch Installation
Recipe 1-1. Using Tensors
Problem
Solution
How It Works
Conclusion
Chapter 2: Probability Distributions Using PyTorch
Recipe 2-1. Sampling Tensors
Problem
Solution
How It Works
Recipe 2-2. Variable Tensors
Problem
Solution
How It Works
Recipe 2-3. Basic Statistics
Problem
Solution
How It Works
Recipe 2-4. Gradient Computation
Problem
Solution
How It Works
Recipe 2-5. Tensor Operations
Problem
Solution
How It Works
Recipe 2-6. Tensor Operations
Problem
Solution
How It Works
Recipe 2-7. Distributions
Problem
Solution
How It Works
Conclusion
Chapter 3: CNN and RNN Using PyTorch
Recipe 3-1. Setting Up a Loss Function
Problem
Solution
How It Works
Recipe 3-2. Estimating the Derivative of the Loss Function
Problem
Solution
How It Works
Recipe 3-3. Fine-Tuning a Model
Problem
Solution
How It Works
Recipe 3-4. Selecting an Optimization Function
Problem
Solution
How It Works
Recipe 3-5. Further Optimizing the Function
Problem
Solution
How It Works
Recipe 3-6. Implementing a Convolutional Neural Network (CNN)
Problem
Solution
How It Works
Recipe 3-7. Reloading a Model
Problem
Solution
How It Works
Recipe 3-8. Implementing a Recurrent Neural Network
Problem
Solution
How It Works
Recipe 3-9. Implementing a RNN for Regression Problems
Problem
Solution
How It Works
Recipe 3-10. Using PyTorch’s Built-In Functions
Problem
Solution
How It Works
Recipe 3-11. Working with Autoencoders
Problem
Solution
How It Works
Recipe 3-12. Fine-Tuning Results Using Autoencoder
Problem
Solution
How It Works
Recipe 3-13. Restricting Model Overfitting
Problem
Solution
How It Works
Recipe 3-14. Visualizing the Model Overfit
Problem
Solution
How It Works
Recipe 3-15. Initializing Weights in the Dropout Rate
Problem
Solution
How It Works
Recipe 3-16. Adding Math Operations
Problem
Solution
How It Works
Recipe 3-17. Embedding Layers in RNN
Problem
Solution
How It Works
Conclusion
Chapter 4: Introduction to Neural Networks Using PyTorch
Recipe 4-1. Working with Activation Functions
Problem
Solution
How It Works
Linear Function
Bilinear Function
Sigmoid Function
Hyperbolic Tangent Function
Log Sigmoid Transfer Function
ReLU Function
Leaky ReLU
Recipe 4-2. Visualizing the Shape of Activation Functions
Problem
Solution
How It Works
Recipe 4-3. Basic Neural Network Model
Problem
Solution
How It Works
Recipe 4-4. Tensor Differentiation
Problem
Solution
How It Works
Conclusion
Chapter 5: Supervised Learning Using PyTorch
Introduction to Linear Regression
Recipe 5-1. Data Preparation for a Supervised Model
Problem
Solution
How It Works
Recipe 5-2. Forward and Backward PropagationNeural network
Problem
Solution
How It Works
Recipe 5-3. Optimization and Gradient Computation
Problem
Solution
How It Works
Recipe 5-4. Viewing Predictions
Problem
Solution
How It Works
Recipe 5-5. Supervised Model Logistic Regression
Problem
Solution
How It Works
Conclusion
Chapter 6: Fine-Tuning Deep Learning Models Using PyTorch
Recipe 6-1. Building Sequential Neural Networks
Problem
Solution
How It Works
Recipe 6-2. Deciding the Batch Size
Problem
Solution
How It Works
Recipe 6-3. Deciding the Learning Rate
Problem
Solution
How It Works
Recipe 6-4. Performing Parallel Training
Problem
Solution
How It Works
Conclusion
Chapter 7: Natural Language Processing Using PyTorch
Recipe 7-1. Word Embedding
Problem
Solution
How It Works
Recipe 7-2. CBOW Model in PyTorch
Problem
Solution
How It Works
Recipe 7-3. LSTM Model
Problem
Solution
How It Works
Summary
Chapter 8: Distributed PyTorch Modelling, Model Optimization, and Deployment
Recipe 8-1. Distributed Torch Architecture
Problem
Solution
How It Works
Recipe 8-2. Components of Torch Distributed
Problem
Solution
How It Works
Recipe 8-3. Setting Up Distributed PyTorch
Problem
Solution
How It Works
Recipe 8-4. Loading Data to Distributed PyTorch
Problem
Solution
How It Works
Recipe 8-5. Quantization of Models in PyTorch
Problem
Solution
How It Works
Recipe 8-6. Quantization Observer Application
Problem
Solution
How It Works
Recipe 8-7. Quantization Application Using the MNIST Dataset
Problem
Solution
How It Works
Summary
Chapter 9: Data Augmentation, Feature Engineering, and Extractions for Image and Audio
Recipe 9-1. Spectogram for Audio Processing
Problem
Solution
How It Works
Recipe 9-2. Installation of Torchaudio
Problem
Solution
How It Works
Recipe 9-3. Loading Audio Files into PyTorch
Problem
Solution
How It Works
Recipe 9-4. Installation of Librosa for Audio
Problem
Solution
How It Works
Recipe 9-5. Spectogram Transformation
Problem
Solution
How It Works
Recipe 9-6. Griffin-Lim Transformation
Problem
Solution
How It Works
Recipe 9-7. Mel Scale Transformation Using a Filter Bank
Problem
Solution
How It Works
Recipe 9-8. Librosa Mel Scale Conversion vs. the Torchaudio Version
Problem
Solution
How It Works
Recipe 9-9. MFCC and LFCC Using Librosa and Torchaudio
Problem
Solution
How It Works
Recipe 9-10. Data Augmentation for Images
Problem
Solution
How It Works
Conclusion
Chapter 10: PyTorch Model Interpretability and Interface to Sklearn
Recipe 10-1. Installation of Captum
Problem
Solution
How It Works
Recipe 10-2. Primary Attribution Feature Importance of a Deep Learning Model
Problem
Solution
How It Works
Recipe 10-3. Neuron Importance of a Deep Learning Model
Problem
Solution
How It Works
Recipe 10-4. Installation of Skorch
Problem
Solution
How It Works
Recipe 10-5. Skorch Components for a Neuralnet Classifier
Problem
Solution
How It Works
Recipe 10-6. Skorch Neuralnet Regressor
Problem
Solution
How It Works
Recipe 10-7. Skorch Model Save and Load
Problem
Solution
How It Works
Recipe 10-8. Skorch Model Pipeline Creation
Problem
Solution
How It Works
Recipe 10-9. Skorch Model Epoch Scoring
Problem
Solution
How It Works
Recipe 10-10. Grid Search for Best Hyper Parameter
Problem
Solution
How It Works
Conclusion
Index
