Blueprints for Text Analytics Using Python: Machine Learning-Based Solutions for Common Real World (NLP) Applications

Cover
Copyright
Table of Contents
Preface
Approach of the Book
Prerequisites
Some Important Libraries to Know
Books to Read
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
Chapter 1. Gaining Early Insights from Textual Data
What You’ll Learn and What We’ll Build
Exploratory Data Analysis
Introducing the Dataset
Blueprint: Getting an Overview of the Data with Pandas
Calculating Summary Statistics for Columns
Checking for Missing Data
Plotting Value Distributions
Comparing Value Distributions Across Categories
Visualizing Developments Over Time
Blueprint: Building a Simple Text Preprocessing Pipeline
Performing Tokenization with Regular Expressions
Treating Stop Words
Processing a Pipeline with One Line of Code
Blueprints for Word Frequency Analysis
Blueprint: Counting Words with a Counter
Blueprint: Creating a Frequency Diagram
Blueprint: Creating Word Clouds
Blueprint: Ranking with TF-IDF
Blueprint: Finding a Keyword-in-Context
Blueprint: Analyzing N-Grams
Blueprint: Comparing Frequencies Across Time Intervals and Categories
Creating Frequency Timelines
Creating Frequency Heatmaps
Closing Remarks
Chapter 2. Extracting Textual Insights with APIs
What You’ll Learn and What We’ll Build
Application Programming Interfaces
Blueprint: Extracting Data from an API Using the Requests Module
Pagination
Rate Limiting
Blueprint: Extracting Twitter Data with Tweepy
Obtaining Credentials
Installing and Configuring Tweepy
Extracting Data from the Search API
Extracting Data from a User’s Timeline
Extracting Data from the Streaming API
Closing Remarks
Chapter 3. Scraping Websites and Extracting Data
What You’ll Learn and What We’ll Build
Scraping and Data Extraction
Introducing the Reuters News Archive
URL Generation
Blueprint: Downloading and Interpreting robots.txt
Blueprint: Finding URLs from sitemap.xml
Blueprint: Finding URLs from RSS
Downloading Data
Blueprint: Downloading HTML Pages with Python
Blueprint: Downloading HTML Pages with wget
Extracting Semistructured Data
Blueprint: Extracting Data with Regular Expressions
Blueprint: Using an HTML Parser for Extraction
Blueprint: Spidering
Introducing the Use Case
Error Handling and Production-Quality Software
Density-Based Text Extraction
Extracting Reuters Content with Readability
Summary Density-Based Text Extraction
All-in-One Approach
Blueprint: Scraping the Reuters Archive with Scrapy
Possible Problems with Scraping
Closing Remarks and Recommendation
Chapter 4. Preparing Textual Data for Statistics and Machine Learning
What You’ll Learn and What We’ll Build
A Data Preprocessing Pipeline
Introducing the Dataset: Reddit Self-Posts
Loading Data Into Pandas
Blueprint: Standardizing Attribute Names
Saving and Loading a DataFrame
Cleaning Text Data
Blueprint: Identify Noise with Regular Expressions
Blueprint: Removing Noise with Regular Expressions
Blueprint: Character Normalization with textacy
Blueprint: Pattern-Based Data Masking with textacy
Tokenization
Blueprint: Tokenization with Regular Expressions
Tokenization with NLTK
Recommendations for Tokenization
Linguistic Processing with spaCy
Instantiating a Pipeline
Processing Text
Blueprint: Customizing Tokenization
Blueprint: Working with Stop Words
Blueprint: Extracting Lemmas Based on Part of Speech
Blueprint: Extracting Noun Phrases
Blueprint: Extracting Named Entities
Feature Extraction on a Large Dataset
Blueprint: Creating One Function to Get It All
Blueprint: Using spaCy on a Large Dataset
Persisting the Result
A Note on Execution Time
There Is More
Language Detection
Spell-Checking
Token Normalization
Closing Remarks and Recommendations
Chapter 5. Feature Engineering and Syntactic Similarity
What You’ll Learn and What We’ll Build
A Toy Dataset for Experimentation
Blueprint: Building Your Own Vectorizer
Enumerating the Vocabulary
Vectorizing Documents
The Document-Term Matrix
The Similarity Matrix
Bag-of-Words Models
Blueprint: Using scikit-learn’s CountVectorizer
Blueprint: Calculating Similarities
TF-IDF Models
Optimized Document Vectors with TfidfTransformer
Introducing the ABC Dataset
Blueprint: Reducing Feature Dimensions
Blueprint: Improving Features by Making Them More Specific
Blueprint: Using Lemmas Instead of Words for Vectorizing Documents
Blueprint: Limit Word Types
Blueprint: Remove Most Common Words
Blueprint: Adding Context via N-Grams
Syntactic Similarity in the ABC Dataset
Blueprint: Finding Most Similar Headlines to a Made-up Headline
Blueprint: Finding the Two Most Similar Documents in a Large Corpus (Much More Difficult)
Blueprint: Finding Related Words
Tips for Long-Running Programs like Syntactic Similarity
Summary and Conclusion
Chapter 6. Text Classification Algorithms
What You’ll Learn and What We’ll Build
Introducing the Java Development Tools Bug Dataset
Blueprint: Building a Text Classification System
Step 1: Data Preparation
Step 2: Train-Test Split
Step 3: Training the Machine Learning Model
Step 4: Model Evaluation
Final Blueprint for Text Classification
Blueprint: Using Cross-Validation to Estimate Realistic Accuracy Metrics
Blueprint: Performing Hyperparameter Tuning with Grid Search
Blueprint Recap and Conclusion
Closing Remarks
Further Reading
Chapter 7. How to Explain a Text Classifier
What You’ll Learn and What We’ll Build
Blueprint: Determining Classification Confidence Using Prediction Probability
Blueprint: Measuring Feature Importance of Predictive Models
Blueprint: Using LIME to Explain the Classification Results
Blueprint: Using ELI5 to Explain the Classification Results
Blueprint: Using Anchor to Explain the Classification Results
Using the Distribution with Masked Words
Working with Real Words
Closing Remarks
Chapter 8. Unsupervised Methods: Topic Modeling and Clustering
What You’ll Learn and What We’ll Build
Our Dataset: UN General Debates
Checking Statistics of the Corpus
Preparations
Nonnegative Matrix Factorization (NMF)
Blueprint: Creating a Topic Model Using NMF for Documents
Blueprint: Creating a Topic Model for Paragraphs Using NMF
Latent Semantic Analysis/Indexing
Blueprint: Creating a Topic Model for Paragraphs with SVD
Latent Dirichlet Allocation
Blueprint: Creating a Topic Model for Paragraphs with LDA
Blueprint: Visualizing LDA Results
Blueprint: Using Word Clouds to Display and Compare Topic Models
Blueprint: Calculating Topic Distribution of Documents and Time Evolution
Using Gensim for Topic Modeling
Blueprint: Preparing Data for Gensim
Blueprint: Performing Nonnegative Matrix Factorization with Gensim
Blueprint: Using LDA with Gensim
Blueprint: Calculating Coherence Scores
Blueprint: Finding the Optimal Number of Topics
Blueprint: Creating a Hierarchical Dirichlet Process with Gensim
Blueprint: Using Clustering to Uncover the Structure of Text Data
Further Ideas
Summary and Recommendation
Conclusion
Chapter 9. Text Summarization
What You’ll Learn and What We’ll Build
Text Summarization
Extractive Methods
Data Preprocessing
Blueprint: Summarizing Text Using Topic Representation
Identifying Important Words with TF-IDF Values
LSA Algorithm
Blueprint: Summarizing Text Using an Indicator Representation
Measuring the Performance of Text Summarization Methods
Blueprint: Summarizing Text Using Machine Learning
Step 1: Creating Target Labels
Step 2: Adding Features to Assist Model Prediction
Step 3: Build a Machine Learning Model
Closing Remarks
Further Reading
Chapter 10. Exploring Semantic Relationships with Word Embeddings
What You’ll Learn and What We’ll Build
The Case for Semantic Embeddings
Word Embeddings
Analogy Reasoning with Word Embeddings
Types of Embeddings
Blueprint: Using Similarity Queries on Pretrained Models
Loading a Pretrained Model
Similarity Queries
Blueprints for Training and Evaluating Your Own Embeddings
Data Preparation
Blueprint: Training Models with Gensim
Blueprint: Evaluating Different Models
Blueprints for Visualizing Embeddings
Blueprint: Applying Dimensionality Reduction
Blueprint: Using the TensorFlow Embedding Projector
Blueprint: Constructing a Similarity Tree
Closing Remarks
Further Reading
Chapter 11. Performing Sentiment Analysis on Text Data
What You’ll Learn and What We’ll Build
Sentiment Analysis
Introducing the Amazon Customer Reviews Dataset
Blueprint: Performing Sentiment Analysis Using Lexicon-Based Approaches
Bing Liu Lexicon
Disadvantages of a Lexicon-Based Approach
Supervised Learning Approaches
Preparing Data for a Supervised Learning Approach
Blueprint: Vectorizing Text Data and Applying a Supervised Machine Learning Algorithm
Step 1: Data Preparation
Step 2: Train-Test Split
Step 3: Text Vectorization
Step 4: Training the Machine Learning Model
Pretrained Language Models Using Deep Learning
Deep Learning and Transfer Learning
Blueprint: Using the Transfer Learning Technique and a Pretrained Language Model
Step 1: Loading Models and Tokenization
Step 2: Model Training
Step 3: Model Evaluation
Closing Remarks
Further Reading
Chapter 12. Building a Knowledge Graph
What You’ll Learn and What We’ll Build
Knowledge Graphs
Information Extraction
Introducing the Dataset
Named-Entity Recognition
Blueprint: Using Rule-Based Named-Entity Recognition
Blueprint: Normalizing Named Entities
Merging Entity Tokens
Coreference Resolution
Blueprint: Using spaCy’s Token Extensions
Blueprint: Performing Alias Resolution
Blueprint: Resolving Name Variations
Blueprint: Performing Anaphora Resolution with NeuralCoref
Name Normalization
Entity Linking
Blueprint: Creating a Co-Occurrence Graph
Extracting Co-Occurrences from a Document
Visualizing the Graph with Gephi
Relation Extraction
Blueprint: Extracting Relations Using Phrase Matching
Blueprint: Extracting Relations Using Dependency Trees
Creating the Knowledge Graph
Don’t Blindly Trust the Results
Closing Remarks
Further Reading
Chapter 13. Using Text Analytics in Production
What You’ll Learn and What We’ll Build
Blueprint: Using Conda to Create Reproducible Python Environments
Blueprint: Using Containers to Create Reproducible Environments
Blueprint: Creating a REST API for Your Text Analytics Model
Blueprint: Deploying and Scaling Your API Using a Cloud Provider
Blueprint: Automatically Versioning and Deploying Builds
Closing Remarks
Further Reading
Index
About the Authors
Colophon