Deep Learning for Natural Language Processing: Develop Deep Learning Models for your Natural Language Problems

Copyright
Contents
Preface
I Introductions
Welcome
Who Is This Book For?
About Your Outcomes
How to Read This Book
About the Book Structure
About Python Code Examples
About Further Reading
About Getting Help
Summary
II Foundations
Natural Language Processing
Natural Language
Challenge of Natural Language
From Linguistics to Natural Language Processing
Natural Language Processing
Further Reading
Summary
Deep Learning
Deep Learning is Large Neural Networks
Deep Learning is Hierarchical Feature Learning
Deep Learning as Scalable Learning Across Domains
Further Reading
Summary
Promise of Deep Learning for Natural Language
Promise of Deep Learning
Promise of Drop-in Replacement Models
Promise of New NLP Models
Promise of Feature Learning
Promise of Continued Improvement
Promise of End-to-End Models
Further Reading
Summary
How to Develop Deep Learning Models With Keras
Keras Model Life-Cycle
Keras Functional Models
Standard Network Models
Further Reading
Summary
III Data Preparation
How to Clean Text Manually and with NLTK
Tutorial Overview
Metamorphosis by Franz Kafka
Text Cleaning Is Task Specific
Manual Tokenization
Tokenization and Cleaning with NLTK
Additional Text Cleaning Considerations
Further Reading
Summary
How to Prepare Text Data with scikit-learn
The Bag-of-Words Model
Word Counts with CountVectorizer
Word Frequencies with TfidfVectorizer
Hashing with HashingVectorizer
Further Reading
Summary
How to Prepare Text Data With Keras
Tutorial Overview
Split Words with text_to_word_sequence
Encoding with one_hot
Hash Encoding with hashing_trick
Tokenizer API
Further Reading
Summary
IV Bag-of-Words
The Bag-of-Words Model
Tutorial Overview
The Problem with Text
What is a Bag-of-Words?
Example of the Bag-of-Words Model
Managing Vocabulary
Scoring Words
Limitations of Bag-of-Words
Further Reading
Summary
How to Prepare Movie Review Data for Sentiment Analysis
Tutorial Overview
Movie Review Dataset
Load Text Data
Clean Text Data
Develop Vocabulary
Save Prepared Data
Further Reading
Summary
Project: Develop a Neural Bag-of-Words Model for Sentiment Analysis
Tutorial Overview
Movie Review Dataset
Data Preparation
Bag-of-Words Representation
Sentiment Analysis Models
Comparing Word Scoring Methods
Predicting Sentiment for New Reviews
Extensions
Further Reading
Summary
V Word Embeddings
The Word Embedding Model
Overview
What Are Word Embeddings?
Word Embedding Algorithms
Using Word Embeddings
Further Reading
Summary
How to Develop Word Embeddings with Gensim
Tutorial Overview
Word Embeddings
Gensim Python Library
Develop Word2Vec Embedding
Visualize Word Embedding
Load Google's Word2Vec Embedding
Load Stanford's GloVe Embedding
Further Reading
Summary
How to Learn and Load Word Embeddings in Keras
Tutorial Overview
Word Embedding
Keras Embedding Layer
Example of Learning an Embedding
Example of Using Pre-Trained GloVe Embedding
Tips for Cleaning Text for Word Embedding
Further Reading
Summary
VI Text Classification
Neural Models for Document Classification
Overview
Word Embeddings + CNN = Text Classification
Use a Single Layer CNN Architecture
Dial in CNN Hyperparameters
Consider Character-Level CNNs
Consider Deeper CNNs for Classification
Further Reading
Summary
Project: Develop an Embedding + CNN Model for Sentiment Analysis
Tutorial Overview
Movie Review Dataset
Data Preparation
Train CNN With Embedding Layer
Evaluate Model
Extensions
Further Reading
Summary
Project: Develop an n-gram CNN Model for Sentiment Analysis
Tutorial Overview
Movie Review Dataset
Data Preparation
Develop Multichannel Model
Evaluate Model
Extensions
Further Reading
Summary
VII Language Modeling
Neural Language Modeling
Overview
Problem of Modeling Language
Statistical Language Modeling
Neural Language Models
Further Reading
Summary
How to Develop a Character-Based Neural Language Model
Tutorial Overview
Sing a Song of Sixpence
Data Preparation
Train Language Model
Generate Text
Further Reading
Summary
How to Develop a Word-Based Neural Language Model
Tutorial Overview
Framing Language Modeling
Jack and Jill Nursery Rhyme
Model 1: One-Word-In, One-Word-Out Sequences
Model 2: Line-by-Line Sequence
Model 3: Two-Words-In, One-Word-Out Sequence
Further Reading
Summary
Project: Develop a Neural Language Model for Text Generation
Tutorial Overview
The Republic by Plato
Data Preparation
Train Language Model
Use Language Model
Extensions
Further Reading
Summary
VIII Image Captioning
Neural Image Caption Generation
Overview
Describing an Image with Text
Neural Captioning Model
Encoder-Decoder Architecture
Further Reading
Summary
Neural Network Models for Caption Generation
Image Caption Generation
Inject Model
Merge Model
More on the Merge Model
Further Reading
Summary
How to Load and Use a Pre-Trained Object Recognition Model
Tutorial Overview
ImageNet
The Oxford VGG Models
Load the VGG Model in Keras
Develop a Simple Photo Classifier
Further Reading
Summary
How to Evaluate Generated Text With the BLEU Score
Tutorial Overview
Bilingual Evaluation Understudy Score
Calculate BLEU Scores
Cumulative and Individual BLEU Scores
Worked Examples
Further Reading
Summary
How to Prepare a Photo Caption Dataset For Modeling
Tutorial Overview
Download the Flickr8K Dataset
How to Load Photographs
Pre-Calculate Photo Features
How to Load Descriptions
Prepare Description Text
Whole Description Sequence Model
Word-By-Word Model
Progressive Loading
Further Reading
Summary
Project: Develop a Neural Image Caption Generation Model
Tutorial Overview
Photo and Caption Dataset
Prepare Photo Data
Prepare Text Data
Develop Deep Learning Model
Evaluate Model
Generate New Captions
Extensions
Further Reading
Summary
IX Machine Translation
Neural Machine Translation
What is Machine Translation?
What is Statistical Machine Translation?
What is Neural Machine Translation?
Further Reading
Summary
What are Encoder-Decoder Models for Neural Machine Translation
Encoder-Decoder Architecture for NMT
Sutskever NMT Model
Cho NMT Model
Further Reading
Summary
How to Configure Encoder-Decoder Models for Machine Translation
Encoder-Decoder Model for Neural Machine Translation
Baseline Model
Word Embedding Size
RNN Cell Type
Encoder-Decoder Depth
Direction of Encoder Input
Attention Mechanism
Inference
Final Model
Further Reading
Summary
Project: Develop a Neural Machine Translation Model
Tutorial Overview
German to English Translation Dataset
Preparing the Text Data
Train Neural Translation Model
Evaluate Neural Translation Model
Extensions
Further Reading
Summary
X Appendix
Getting Help
Official Keras Destinations
Where to Get Help with Keras
Where to Get Help with Natural Language
How to Ask Questions
Contact the Author
How to Setup a Workstation for Deep Learning
Overview
Download Anaconda
Install Anaconda
Start and Update Anaconda
Install Deep Learning Libraries
Further Reading
Summary
How to Use Deep Learning in the Cloud
Overview
Setup Your AWS Account
Launch Your Server Instance
Login, Configure and Run
Build and Run Models on AWS
Close Your EC2 Instance
Tips and Tricks for Using Keras on AWS
Further Reading
Summary
XI Conclusions
How Far You Have Come