TensorFlow in Action

TensorFlow in Action
brief contents
contents
preface
acknowledgments
about this book
Who should read this book?
How this book is organized: A roadmap
About the code
liveBook discussion forum
about the author
about the cover illustration
Part 1—Foundations of TensorFlow 2 and deep learning
1 The amazing world of TensorFlow
1.1 What is TensorFlow?
1.1.1 An overview of popular components of TensorFlow
1.1.2 Building and deploying a machine learning model
1.2 GPU vs. CPU
1.3 When and when not to use TensorFlow
1.3.1 When to use TensorFlow
1.3.2 When not to use TensorFlow
1.4 What will this book teach you?
1.4.1 TensorFlow fundamentals
1.4.2 Deep learning algorithms
1.4.3 Monitoring and optimization
1.5 Who is this book for?
1.6 Should we really care about Python and TensorFlow 2?
Summary
2 TensorFlow 2
2.1 First steps with TensorFlow 2
2.1.1 How does TensorFlow operate under the hood?
2.2 TensorFlow building blocks
2.2.1 Understanding tf.Variable
2.2.2 Understanding tf.Tensor
2.2.3 Understanding tf.Operation
2.3 Neural network–related computations in TensorFlow
2.3.1 Matrix multiplication
2.3.2 Convolution operation
2.3.3 Pooling operation
Summary
Answers to exercises
3 Keras and data retrieval in TensorFlow 2
3.1 Keras model-building APIs
3.1.1 Introducing the data set
3.1.2 The Sequential API
3.1.3 The functional API
3.1.4 The sub-classing API
3.2 Retrieving data for TensorFlow/Keras models
3.2.1 tf.data API
3.2.2 Keras DataGenerators
3.2.3 tensorflow-datasets package
Summary
Answers to exercises
4 Dipping toes in deep learning
4.1 Fully connected networks
4.1.1 Understanding the data
4.1.2 Autoencoder model
4.2 Convolutional neural networks
4.2.1 Understanding the data
4.2.2 Implementing the network
4.3 One step at a time: Recurrent neural networks (RNNs)
4.3.1 Understanding the data
4.3.2 Implementing the model
4.3.3 Predicting future CO2 values with the trained model
Summary
Answers to exercises
5 State-of-the-art in deep learning: Transformers
5.1 Representing text as numbers
5.2 Understanding the Transformer model
5.2.1 The encoder-decoder view of the Transformer
5.2.2 Diving deeper
5.2.3 Self-attention layer
5.2.4 Understanding self-attention using scalars
5.2.5 Self-attention as a cooking competition
5.2.6 Masked self-attention layers
5.2.7 Multi-head attention
5.2.8 Fully connected layer
5.2.9 Putting everything together
Summary
Answers to exercises
Part 2—Look ma, no hands! Deep networks in the real world
6 Teaching machines to see: Image classification with CNNs
6.1 Putting the data under the microscope: Exploratory data analysis
6.1.1 The folder/file structure
6.1.2 Understanding the classes in the data set
6.1.3 Computing simple statistics on the data set
6.2 Creating data pipelines using the Keras ImageDataGenerator
6.3 Inception net: Implementing a state-of-the-art image classifier
6.3.1 Recap on CNNs
6.3.2 Inception net v1
6.3.3 Putting everything together
6.3.4 Other Inception models
6.4 Training the model and evaluating performance
Summary
Answers to exercises
7 Teaching machines to see better: Improving CNNs and making them confess
7.1 Techniques for reducing overfitting
7.1.1 Image data augmentation with Keras
7.1.2 Dropout: Randomly switching off parts of your network to improve generalizability
7.1.3 Early stopping: Halting the training process if the network starts to underperform
7.2 Toward minimalism: Minception instead of Inception
7.2.1 Implementing the stem
7.2.2 Implementing Inception-ResNet type A block
7.2.3 Implementing the Inception-ResNet type B block
7.2.4 Implementing the reduction block
7.2.5 Putting everything together
7.2.6 Training Minception
7.3 If you can't beat them, join ‘em: Using pretrained networks for enhancing performance
7.3.1 Transfer learning: Reusing existing knowledge in deep neural networks
7.4 Grad-CAM: Making CNNs confess
Summary
Answers to exercises
8 Telling things apart: Image segmentation
8.1 Understanding the data
8.2 Getting serious: Defining a TensorFlow data pipeline
8.2.1 Optimizing tf.data pipelines
8.2.2 The final tf.data pipeline
8.3 DeepLabv3: Using pretrained networks to segment images
8.3.1 A quick overview of the ResNet-50 model
8.3.2 Atrous convolution: Increasing the receptive field of convolution layers with holes
8.3.3 Implementing DeepLab v3 using the Keras functional API
8.3.4 Implementing the atrous spatial pyramid pooling module
8.3.5 Putting everything together
8.4 Compiling the model: Loss functions and evaluation metrics in image segmentation
8.4.1 Loss functions
8.4.2 Evaluation metrics
8.5 Training the model
8.6 Evaluating the model
Summary
Answers to exercises
9 Natural language processing with TensorFlow: Sentiment analysis
9.1 What the text? Exploring and processing text
9.2 Getting text ready for the model
9.2.1 Splitting training/validation and testing data
9.2.2 Analyze the vocabulary
9.2.3 Analyzing the sequence length
9.2.4 Text to words and then to numbers with Keras
9.3 Defining an end-to-end NLP pipeline with TensorFlow
9.4 Happy reviews mean happy customers: Sentiment analysis
9.4.1 LSTM Networks
9.4.2 Defining the final model
9.5 Training and evaluating the model
9.6 Injecting semantics with word vectors
9.6.1 Word embeddings
9.6.2 Defining the final model with word embeddings
9.6.3 Training and evaluating the model
Summary
Answers to exercises
10 Natural language processing with TensorFlow: Language modeling
10.1 Processing the data
10.1.1 What is language modeling?
10.1.2 Downloading and playing with data
10.1.3 Too large vocabulary? N-grams to the rescue
10.1.4 Tokenizing text
10.1.5 Defining a tf.data pipeline
10.2 GRUs in Wonderland: Generating text with deep learning
10.3 Measuring the quality of the generated text
10.4 Training and evaluating the language model
10.5 Generating new text from the language model: Greedy decoding
10.6 Beam search: Enhancing the predictive power of sequential models
Summary
Answers to exercises
Part 3—Advanced deep networks for complex problems
11 Sequence-to-sequence learning: Part 1
11.1 Understanding the machine translation data
11.2 Writing an English-German seq2seq machine translator
11.2.1 The TextVectorization layer
11.2.2 Defining the TextVectorization layers for the seq2seq model
11.2.3 Defining the encoder
11.2.4 Defining the decoder and the final model
11.2.5 Compiling the model
11.3 Training and evaluating the model
11.4 From training to inference: Defining the inference model
Summary
Answers to exercises
12 Sequence-to-sequence learning: Part 2
12.1 Eyeballing the past: Improving our model with attention
12.1.1 Implementing Bahdanau attention in TensorFlow
12.1.2 Defining the final model
12.1.3 Training the model
12.2 Visualizing the attention
Summary
Answers to exercises
13 Transformers
13.1 Transformers in more detail
13.1.1 Revisiting the basic components of the Transformer
13.1.2 Embeddings in the Transformer
13.1.3 Residuals and normalization
13.2 Using pretrained BERT for spam classification
13.2.1 Understanding BERT
13.2.2 Classifying spam with BERT in TensorFlow
13.3 Question answering with Hugging Face’s Transformers
13.3.1 Understanding the data
13.3.2 Processing data
13.3.3 Defining the DistilBERT model
13.3.4 Training the model
13.3.5 Ask BERT a question
Summary
Answers to exercises
14 TensorBoard: Big brother of TensorFlow
14.1 Visualize data with TensorBoard
14.2 Tracking and monitoring models with TensorBoard
14.3 Using tf.summary to write custom metrics during model training
14.4 Profiling models to detect performance bottlenecks
14.4.1 Optimizing the input pipeline
14.4.2 Mixed precision training
14.5 Visualizing word vectors with the TensorBoard
Summary
Answers to exercises
15 TFX: MLOps and deploying models with TensorFlow
15.1 Writing a data pipeline with TFX
15.1.1 Loading data from CSV files
15.1.2 Generating basic statistics from the data
15.1.3 Inferring the schema from data
15.1.4 Converting data to features
15.2 Training a simple regression neural network: TFX Trainer API
15.2.1 Defining a Keras model
15.2.2 Defining the model training
15.2.3 SignatureDefs: Defining how models are used outside TensorFlow
15.2.4 Training the Keras model with TFX Trainer
15.3 Setting up Docker to serve a trained model
15.4 Deploying the model and serving it through an API
15.4.1 Validating the infrastructure
15.4.2 Resolving the correct model
15.4.3 Evaluating the model
15.4.4 Pushing the final model
15.4.5 Predicting with the TensorFlow serving API
Summary
Answers to exercises
Appendix A—Setting up the environment
A.1 In a Unix-based environment
A.1.1 Creating a virtual Python environment with Anaconda distribution (Ubuntu)
A.1.2 Prerequisites for GPU support (Ubuntu)
A.1.3 Notes on MacOS
A.2 In Windows Environments
A.2.1 Creating a Virtual Python Environment (Anaconda)
A.2.2 Prerequisites for GPU support
A.3 Activating and deactivating the conda environment
A.4 Running the Jupyter Notebook server and creating notebooks
A.5 Miscellaneous notes
Appendix B—Computer vision
B.1 Grad-CAM: Interpreting computer vision models
B.2 Image segmentation: U-Net model
B.2.1 Understanding and defining the U-Net model
B.2.2 What’s better than an encoder? A pretrained encoder
Appendix C—Natural language processing
C.1 Touring around the zoo: Meeting other Transformer models
C.1.1 Generative pre-training (GPT) model (2018)
C.1.2 DistilBERT (2019)
C.1.3 RoBERT/ToBERT (2019)
C.1.4 BART (2019)
C.1.5 XLNet (2020)
C.1.6 Albert (2020)
C.1.7 Reformer (2020)
index
Numerics
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Y
Z