Introduction to Python for Humanists

Cover
Half Title
Series Page
Title Page
Copyright Page
Contents
Preface
Acknowledgments
About the Author
Part I The Basics of Python
Chapter 1 Introduction to Python
1.1 Introduction to Python
1.1.1 Why Should Humanists Learn to Code?
1.1.1.1 The Timeline for Starting a Large Digital History Project
1.1.1.2 The Self-Reliant Digital Humanist
1.1.1.3 Other Benefits
1.1.2 What Is Python?
1.1.3 Why Python?
1.2 Installing Python
1.2.1 Trinket
1.2.2 Using Google Colab
1.2.3 Using Binder from JupyterBook
1.2.4 Using Jupyter Notebooks Online
1.2.5 Installing Python Locally
1.2.5.1 Download Anaconda Navigator
1.2.5.2 Using Anaconda Navigator
1.2.5.3 Installing JupyterLab
1.2.6 Conclusion
1.3 Coding Basics
1.3.1 The Print Function
1.3.2 Objects
1.3.3 Variables
1.3.4 Case Sensitivity
1.3.5 Reserved Words
1.3.6 Built-in Types
1.3.7 Type Function
1.3.8 Bugs
Chapter 2 Data and Data Structures
2.1 Introduction to Data
2.1.1 What Is Data?
2.1.2 Strings
2.1.3 Working with Strings as Data
2.1.3.1 Upper Method
2.1.3.2 Lower Method
2.1.3.3 Capitalize Method
2.1.3.4 Replace Method
2.1.3.5 Split Method
2.1.4 Numbers (Integers and Floats)
2.1.5 Working with Numbers as Data
2.1.6 Booleans
2.1.7 Conclusion
2.2 Introduction to Data Structures
2.2.1 Data Structures
2.2.2 Lists
2.2.2.1 Indexing a List
2.2.3 Tuples
2.2.4 Mutability vs Immutability
2.2.5 Sets (Bonus Data Structure)
2.2.6 Dictionaries
2.2.6.1 Indexing Dictionaries
Chapter 3 Loops and Logic
3.1 Introduction to Loops
3.1.1 What Are Loops?
3.1.2 For Loops
3.1.3 List Comprehension
3.1.4 Indexing a List with and without Enumerate
3.1.5 Operators
3.1.6 While Loops
3.2 Conditionals
3.2.1 If Statement
3.2.2 Else Statement
3.2.3 Elif and the ‘in’ Operator with Strings
3.2.4 Conditionals and Lists with ‘in’ and ‘not in’
3.2.5 Conclusion
Chapter 4 Formal Coding: Functions, Classes, and Libraries
4.1 Functions
4.1.1 Introduction
4.1.2 Functions in Action
4.1.3 Docstrings
4.1.4 Functions with Multiple Arguments
4.1.5 Keyword Argument
4.1.6 Keyword Arbitrary Arguments
4.1.7 Conclusion
4.1.8 Answer for Result
4.2 Classes
4.2.1 Introduction
4.2.2 Creating a Class
4.2.3 Adding Functions to a Class
4.3 Libraries in Python
4.3.1 Introduction
4.3.2 How to Install Python Libraries
4.3.3 How to Import a Library
4.3.4 Conclusion
Chapter 5 Working with External Data
5.1 Working with Textual Data
5.1.1 Introduction
5.1.2 The “With” Statement
5.1.3 How to Open a Text File
5.1.4 How toWrite Data to a Text File
5.2 Working with JSON Data
5.2.1 Introduction
5.2.2 Writing JSON Data with json.dump()
5.2.3 Reading JSON data with json.load()
5.3 Working with Multiple Files
5.3.1 Introduction
5.3.2 Working with Glob
5.3.3 Grabbing Multiple Nested Directories
5.3.4 Walking a Directory
5.3.5 Conclusion
Chapter 6 Working with Data on theWeb
6.1 Introduction to HTML
6.1.1 Introduction
6.1.2 Diving into HTML
6.1.3 Understanding Attributes
6.1.4 Parsing HTML with BeautifulSoup
6.1.5 How to Find aWebsite’s HTML
6.2 Scraping Web Pages with Requests and BeautifulSoup
6.2.1 Introduction
6.2.2 Requests
6.2.3 BeautifulSoup
Part II Data Analysis with Pandas
Chapter 7 Introduction to Pandas
7.1 Introduction to Pandas
7.1.1 What Is Pandas?
7.1.2 Why Use Pandas?
7.1.3 How to Install Pandas
7.1.4 How to Import Pandas
7.2 The Basics of Pandas
7.2.1 How to Create a DataFrame from a Dictionary
7.2.2 How to Display a DataFrame
7.2.3 How to Save DataFrame to CSV
7.2.4 How to Read DataFrame from CSV
7.2.5 How to Save DataFrame to JSON
7.2.6 How to Add a Column to the DataFrame
7.2.7 How to Grab a Specific Column
7.2.8 How to Convert a Column to a List
7.2.9 Isolating Unique Values in a Column
7.2.10 How to Grab a Specific Row of a DataFrame with iloc
7.2.11 Iterating over a DataFrame with df.iterrows()
7.2.12 Conclusion
Chapter 8 Working with Data in Pandas
8.1 Finding Data in DataFrame
8.1.1 About the Titanic Dataset
8.1.2 How to Find Column Data
8.1.3 How to Get a Quick Sense of the Dataset with df.head()
8.1.4 How to Grab a Specific Range of Rows with df.iloc[]
8.1.5 HowtoGet aQuickQuantitativeUnderstanding of theDatasetwith describe()
8.1.6 How to Find Specific Information in the Dataset with df.loc
8.1.7 How to Query with “OR” (|) on a DataFrame
8.2 Organizing the DataFrame
8.2.1 How to Sort Data By Single Column
8.2.2 How to Reverse Sort Data by Single Column
8.2.3 How to Sort Data by Multiple Columns
8.2.4 How to Sort Data byMultiple Columns with Different Values Organized Differently
8.3 Cleaning the DataFrame
8.3.1 How to Drop a Column in Pandas DataFrame
8.3.2 How to Remove Rows That Have NaN in Any Column
8.3.3 How to Remove Rows That Have NaN in a Specific Column
8.3.4 How to Convert DataFrame Data Types (from Float to Int)
8.3.5 Conclusion
Chapter 9 Searching for Data
9.1 Advanced Searching on Strings
9.1.1 Finding Features within a String
9.1.2 Finding Strings That Don’t Contain Feature
9.1.3 Using RegEx with Pandas
9.2 Filter and Querying
9.2.1 Introduction
9.2.2 The Filter Function
9.2.3 The Query Function
9.3 Grouping with groupby()
9.3.1 Introduction
9.3.2 groupby()
9.3.3 Quantitative Analysis with .count() and .sum()
9.3.4 Working with Multiple Groups
9.3.5 Groupings with Many Subsets
Chapter 10 Advanced Pandas
10.1 Plotting Data with Pandas
10.1.1 Importing the DataFrame
10.1.2 Bar and Barh Charts with Pandas
10.1.3 Pie Charts with Pandas
10.1.4 Scatter Plots with Pandas
10.2 Graphing Network Data with Pandas
10.2.1 Getting the Data from Pandas to NetworkX
10.2.2 Graphing the Data
10.2.3 Customize the Graph
10.3 Time Series Data
10.3.1 What Is Time Series Data
10.3.2 About the Dataset
10.3.3 Cleaning the Data from Float to Int
10.3.4 Convert to Time Series DateTime in Pandas
Part III Natural Language Processing with spaCy
Chapter 11 Introduction to Spacy
11.1 The Basics of spaCy
11.1.1 What Is spaCy?
11.1.2 How to Install spaCy
11.1.3 Containers
11.2 Getting Started with spaCy and Its Linguistic Annotations
11.2.1 Importing spaCy and Loading Data
11.2.2 Creating a Doc Container
11.2.3 Sentence Boundary Detection (SBD)
11.2.4 Token Attributes
11.2.4.1 Text
11.2.4.2 Head
11.2.4.3 Left Edge
11.2.4.4 Right Edge
11.2.4.5 Entity Type
11.2.4.6 Ent IOB
11.2.4.7 Lemma
11.2.4.8 Morph
11.2.4.9 Part of Speech
11.2.4.10 Syntactic Dependency
11.2.4.11 Language
11.2.5 Part-of-Speech (POS) Tagging
11.2.6 Named Entity Recognition (NER)
11.2.7 Conclusion
11.3 spaCy’s Pipelines
11.3.1 Standard Pipes (Components and Factories) Available from spaCy
11.3.1.1 Attribute Rulers
11.3.1.2 Matchers
11.3.2 How to Add Pipes
11.3.3 Examining a Pipeline
11.3.4 Conclusion
Chapter 12 Rules-Based spaCy
12.1 The EntityRuler
12.1.1 Introduction to spaCy’s EntityRuler
12.1.2 Demonstration of EntityRuler in Action
12.1.3 Introducing Complex Rules and Variance to the EntityRuler (Advanced)
12.2 The Matcher
12.2.1 Introduction
12.2.2 A Basic Example
12.2.3 Attributes Taken by Matcher
12.2.4 Applied Matcher
12.2.4.1 Grabbing all Proper Nouns
12.2.4.2 Improving it with Multi-Word Tokens
12.2.4.3 Greedy Keyword Argument
12.2.4.4 Adding in Sequences
12.3 The PhraseMatcher
12.3.1 Introduction
12.3.2 Setting a Custom Attribute
12.3.3 Adding a Function with on_match
12.4 Using RegEx with spaCy
12.4.1 What Is Regular Expressions (RegEx)?
12.4.2 The Strengths of RegEx
12.4.3 The Weaknesses of RegEx
12.4.4 How to Use RegEx in Python
12.4.5 How to Use RegEx in spaCy
12.5 Working with Multi-Word Token Entities and RegEx in spaCy 3x
12.5.1 Key Concepts in This Notebook
12.5.2 Problems with Multi-Word Tokens in spaCy as Entities
12.5.3 Extract Multi-Word Tokens
12.5.4 Reconstruct Spans
12.5.5 Inject the Spans into the doc.ents
12.5.6 Give Priority to Longer Spans
Chapter 13 Solving a Domain-Specific Problem: A Case Study with Holocaust NER
13.1 Cultivating Good Datasets for Entities
13.1.1 Introduction to Datasets
13.1.2 Acquiring the Data
13.1.3 United States Holocaust Memorial Museum
13.1.4 Normalizing Data
13.2 The Challenges of Holocaust NER
13.2.1 An Overview of the Problems
13.2.2 Ethical
13.2.3 Linguistic
13.2.4 Toponyms
13.3 Creating a Rules-Based Pipeline for Holocaust Documents
13.3.1 Creating a Blank spaCy Model
13.3.2 Creating EntityRulers
13.3.3 Creating Function for Matching RegEx
13.3.4 Add Pipe for Finding Streets
13.3.5 Creating a Pipe for Finding Ships
13.3.6 Create Pipe for Identifying a Military Personnel
13.3.7 Create Pipe for Identifying Spouses
13.3.8 Creating a Pipe for Finding Ghettos
13.3.9 Creating a Geography Pipe
13.3.10 Seeing the Pipes at Work
Chapter 14 Topic Modeling: Concepts and Theory
14.1 What Is Topic Modeling?
14.1.1 Rules-Based Methods
14.1.2 Machine Learning-Based Methods
14.1.3 Why Use Topic Modeling?
14.2 Topics and Clusters
14.2.1 What Are Topics?
14.2.2 What Are Clusters?
14.3 Bigrams and Trigrams
14.3.1 Textual Ambiguity
14.3.2 Bigrams
14.3.3 Trigrams
14.3.4 Why Are These Important?
14.4 LDA Topic Modeling
14.4.1 Process of Topic Modeling
14.4.2 Knowing the Total Number of Topics
14.4.3 Applying a Topic Model
14.4.4 Summary of Key Issues with LDA Topic Modeling
14.5 Creating LDA in Python
14.5.1 Importing the Required Libraries and Data
14.5.2 Cleaning Documents
14.5.3 Create ID-Word Index
14.5.4 Creating LDA Topic Model
14.5.5 Analyze a Document
14.5.6 Analyze the Topic Model
14.6 Transformer Models
14.6.1 Importing Libraries and Gathering Data
14.6.2 Embedding the Documents
14.6.3 Flattening the Data
14.6.4 Isolating Clusters with HDBScan
14.6.5 Analyzing the Labels
14.6.6 Outliers (Noise)
14.7 Top2Vec in Python
14.7.1 Creating a Top2Vec Model
14.7.2 Analyzing Our Topic Model
14.7.3 Working with Bigrams and Trigrams
14.7.4 Saving and Loading a Top2Vec Model
Chapter 15 Text Analysis with BookNLP
15.1 Introduction to BookNLP
15.1.1 What Is BookNLP?
15.1.2 Why Books and Larger Documents?
15.1.3 How to Install BookNLP
15.2 Getting Started With BookNLP
15.2.1 Importing BookNLP and Creating a Pipeline
15.2.2 Setting up the File and Directories
15.2.3 Running the Pipeline
15.3 The Output Files
15.3.1 The .tokens File
15.3.2 The .entities File
15.3.3 The .quotes File
15.3.4 The .supersense File
15.3.5 The .book File
15.3.6 The .book.html File
15.4 Character Analysis
15.4.1 Analyzing the Characters (From BookNLP Repo)
15.4.2 Parsing Verb Usage
15.5 Events Analysis
15.5.1 Exploring the Tokens File
15.5.2 Grabbing the Events
15.5.3 Analyzing Events Words and Lemmas
15.5.4 Grabbing Event Sentences
15.5.5 Bringing Everything Together
15.5.6 Creating an .events File
15.5.7 Conclusion
Chapter 16 Social Network Analysis
16.1 The Basic Concepts of Social Network Analysis
16.1.1 Basic Terminology
16.1.2 SNA Libraries in Python
16.2 Introduction to NetworkX
16.2.1 Adding All Edges at Once
16.2.2 Asymmetrical Networks
16.2.3 Calculating Shortest Distance
16.2.4 Calculating Connections
16.2.5 Identifying Major Actors in a Network
16.2.6 Limitations of Matplotlib and NetworkX
16.3 Producing Dynamic Graphs with PyVis
16.3.1 The Basics of PyVis
16.4 NetworkX and PyVis
16.5 Adding Color to Nodes
16.6 SNA on Humanities Data: Structuring the Data
16.6.1 Examining the Data
16.7 SNA on Humanities Data: Creating the Graph
16.7.1 Building the Network
16.7.2 Visualizing the Network
16.7.3 Adding Menus
16.7.4 Conclusion
Part IV Designing an Application with Streamlit
Chapter 17 Introduction to Streamlit
17.1 Creating Our First App
17.1.1 Options for Application Development in Python
17.1.2 Installing Streamlit
17.1.3 Creating a Home Page
17.2 Displaying Data in Streamlit
17.2.1 Displaying Text to Users
17.2.2 Displaying Python Data Structures
17.2.2.1 Data Structures with st.write()
17.2.2.2 Data Structures with st.json()
17.2.3 Displaying Tabular Data
17.2.3.1 Tabular Data with st.write()
17.2.3.2 Tabular Data with st.dataframe()
17.2.3.3 Tabular Data with st.table()
17.2.3.4 Tabular Data with st.markdown()
17.2.4 Displaying Multimedia in Streamlit
17.2.4.1 Images
17.2.4.2 Audio
17.2.4.3 Video
17.3 Streamlit Input Widgets
17.3.1 Text Input Widgets
17.3.1.1 st.text_input()
17.3.1.2 st.text_area()
17.3.2 Numerical Input Widgets
17.3.2.1 st.number_input()
17.3.2.2 st.slider()
17.3.3 Date and Time Input Widgets
17.3.3.1 st.date_input()
17.3.3.2 st.time_input()
17.3.4 Boolean Input Widgets
17.3.4.1 st.checkbox()
17.3.4.2 st.button()
17.3.5 Selection Widgets
17.3.5.1 st.radio()
17.3.5.2 st.selectbox()
17.3.5.3 st.multiselect()
Chapter 18 Advanced Streamlit Features
18.1 Data Visualization
18.1.1 Metrics
18.1.2 Plotting Basic Graphs with Streamlit
18.1.2.1 Line Charts with st.line_chart()
18.1.2.2 Bar Charts with st.bar_chart()
18.1.2.3 Area Charts with st.area_chart()
18.1.3 Map Charts
18.1.3.1 Creating Maps with st.map()
18.1.3.2 Third-Party Maps – An Example with PyDeck
18.2 Layout Design
18.2.1 Layout Widgets
18.2.1.1 Sidebar
18.2.1.2 Columns
18.2.1.3 Expander
18.2.1.4 Container
18.2.1.5 Tabs
18.2.1.6 Empty
18.3 Streamlit Cache and Session States
18.3.1 Caching Data with @st.cache_data
18.4 Storing Data with st.session_state
18.5 Custom HTML
18.6 Multi-Page Applications
Chapter 19 Building a Database Query Application
19.1 Building a Database Query Application
19.1.1 Importing the Libraries
19.1.2 Caching Data
19.1.3 Creating Our App Layout
19.1.4 Using User Inputs to Produce a New DataFrame
19.2 Deploying an App in the Cloud with Streamlit Share
19.2.1 Create a GitHub Account
19.2.2 Upload Application to GitHub
19.2.3 Connect Streamlit Share to your GitHub
19.2.4 Create a New App
19.2.5 Set Custom Subdomain
Part V Conclusion
Chapter 20 Conclusion
Index