Zero to Deep Learning with Keras and Tensorflow

Preface
Introduction
Why you should care
Who this book is for
Acknowledgements
About the author
How to use this book
How to approach exercises
Notation
Prerequisites - Is this book right for me?
Sum
Partial derivatives
Dot product
Python
Our development environment
Miniconda Python
Conda Environment
GPU enabled environment
Tensorflow 2.0
Jupyter notebook
Environment check
Python 3.7
Jupyter
Other packages
Troubleshooting installation
Updating Conda
Getting Started
Deep Learning in the real world
First Deep Learning Model
Numpy
Matplotlib
Scikit-Learn
Keras
Exercises
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Data Manipulation
Many types of data
Tabular Data
Data Exploration with Pandas
Indexing
Selections
Unique Values
Sorting
Aggregations
Merge
Pivot Tables
Correlations
Visual data exploration
Line Plot
Scatter plot
Histograms
Cumulative Distribution
Box plot
Subplots
Pie charts
Hexbin plot
Unstructured data
Images
Sound
Text data
Feature Engineering
Exercises
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Exercise 5
Machine Learning
The purpose of Machine Learning
Different types of learning
Supervised Learning
Configuration File
Linear Regression
Let's draw some examples.
Cost Function
Finding the best model
Linear Regression with Keras
Evaluating Model Performance
Train / Test split
Classification
Linear regression fail
Logistic Regression
Train/Test split
Overfitting
How to avoid overfitting
Cross-Validation
Confusion Matrix
Precision
Recall
F1 Score
Feature Preprocessing
Categorical Features
Feature Transformations
Exercises
Exercise 1
Exercise 2
Deep Learning
Beyond linear models
Neural Network Diagrams
Linear regression
Logistic regression
Perceptron
Deeper Networks
Activation functions
Tanh
ReLU
Softplus
SeLU
Binary classification
Logistic Regression
Deep model
Multiclass classification
Tags
Mutually exclusive classes and Softmax
The Iris dataset
Conclusion
Exercises
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Deep Learning Internals
This is a special chapter
Derivatives
Finite differences
Partial derivatives and the gradient
Backpropagation intuition
Learning Rate
Gradient descent
Gradient calculation in Neural Networks
The math of backpropagation
Forward Pass
Weight updates
Fully Connected Backpropagation
Forward Pass
Backpropagation
Matrix Notation
Forward Pass
Backpropagation
Gradient descent
Random Forest
Logistic Regression Model
Learning Rates
Batch Sizes
Optimizers
Stochastic Gradient Descent (or Simply Go Down) and its variations
Initialization
Inner layer representation
Exercises
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Convolutional Neural Networks
Intro
Machine Learning on images with pixels
MNIST
Pixels as features
Multiclass output
Fully connected on images
Beyond pixels as features
Using local information
Images as tensors
Colored images
Convolutional Neural Networks
Convolutional Layers
Pooling layers
Final architecture
Convolutional network on images
Beyond images
Conclusion
Exercises
Exercise 1
Exercise 2
Time Series and Recurrent Neural Networks
Time Series
Time series classification
Fully connected networks
Fully connected networks with feature engineering
Fully connected networks with 1D Convolution
Sequence Problems
1-to-1
1-to-many
many-to-1
asynchronous many-to-many
synchronous many-to-many
RNN allow graphs with cycles
Time series forecasting
Fully connected network
Recurrent Neural Networks
Recurrent Neural Network Maths
Long Short-Term Memory Networks (LSTM)
LSTM forecasting
Improving forecasting
Conclusion
Exercises
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Natural Language Processing and Text Data
Use cases
Text Data
Loading text data
Feature extraction from text
Bag of Words features
Sentiment classification
Text as a sequence
Sequence generation and language modeling
Character sequences
Recurrent Model
Sampling from the model
Sequence to sequence models and language translation
Exercises
Exercise 1
Exercise 2
Training with GPUs
Graphical Processing Units
Cloud GPU providers
Google Colab
Pipeline AI
Floydhub
Paperspace
AWS EC2 Deep Learning AMI
AWS Sagemaker
Google Cloud and Microsoft Azure
The DIY solution (on Ubuntu)
GPU VS CPU training
Tensorflow 2.0 compatibility
Convolutional model comparison
Multiple GPUs
Distribution strategies
Data Parallelization using Tensorflow
Data Parallelization using Keras
Data Parallelization using Horovod
Supercomputing with Tensorflow Mesh
Conclusion
Exercises
Exercise 1
Exercise 2
Performance Improvement
Learning curves
Reducing Overfitting
Model Regularization
Dropout
Batch Normalization
Data augmentation
Tensorflow Data API
Hyperparameter optimization
Hyper-parameter tuning in Tensorboard
Weights and Biases
Hyperopt and Hyperas
Cloud based tools
Exercises
Exercise 1
Pretrained Models for Images
Recognizing sports from images
Keras applications
Predict class with pre-trained Xception
Transfer Learning
Data augmentation
Bottleneck features
Train a fully connected on bottlenecks
Image search
Exercises
Exercise 1
Exercise 2
Exercise 3
Pretrained Embeddings for Text
Unsupervised-supervised learning
GloVe embeddings
Loading pre-trained embeddings in Keras
Gensim
Word Analogies
Visualization
Other pre-trained embeddings
Word2Vec
FastText
Exercises
Exercise 1
Exercise 2
Serving Deep Learning Models
The model development cycle
Data Collection
Labels
Data Processing
Model Development
Model Evaluation
Model Exporting
Model Deployment
Model Monitoring
Deploy a model to predict indoor location
Data exploration
Model definintion and training
Export the model with Keras
A simple deployment with Flask
Full script
Run the script
Get Predictions from the API
Deployment with Tensorflow Serving
Saving a model for Tensorflow Serving
Inference with Tensorflow Serving using Docker and the Rest API
The gRPC API
Exercises
Exercise 1
Exercise 2
Conclusions and Next Steps
Where to go next
Online resources
Bootcamp
Appendix
Matrix multiplication
Chain rule
Univariate functions
Multivariate functions
Exponentially Weighted Moving Average (EWMA)
Tensors
Tensor Dot Product
Convolutions
1D Convolution & Correlation
2D Convolution
Image filters with convolutions
Backpropagation for Recurrent Networks
Getting Started Exercises Solutions
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Data Manipulation Exercises Solutions
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Exercise 5
Machine Learning Exercises Solutions
Exercise 1
Exercise 2
Deep Learning Exercises Solutions
Exercise 1
Exercise 2
Exercise 3
Deep Learning Internals Exercises Solutions
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Convolutional Neural Networks Exercises Solutions
Exercise 1
Exercise 2
Time Series and Recurrent Neural Networks Exercises Solutions
Exercise 1
Exercise 2
Exercise 3
Exercise 4
Natural Language Processing and Text Data Exercises Solutions
Exercise 1
Exercise 2
Training with GPUs Exercises Solutions
Exercise 1
Exercise 2
Performance Improvement Exercises Solutions
Exercise 1
Pretrained Models for Images Exercises Solutions
Exercise 1
Exercise 2
Exercise 3
Pretrained Embeddings for Text Exercises Solutions
Exercise 1
Exercise 2
Serving Deep Learning Models Exercises Solutions
Exercise 1
Exercise 2