Pro Python Best Practices: Debugging, Testing and Maintenance

Contents at a Glance
Contents
About the Author
About the Technical Reviewer
Acknowledgments
Preface
Chapter 1: Introduction
A Lesson in Humility
The Case for Best Practices in Python
The Origin of Best Practices
Hacking
Software Engineering
Agile
Software Craftsmanship
Who This Book Is For
What This Book Is About
Part 1: Debugging
Part 2: Automated Testing
Part 3: Maintenance
Further Benefits
The MazeRun Game
How to Use This Book?
Install Python 3
Install the Pygame Library
Install a Text Editor
Download the Source Code Examples
Part I: Debugging
Chapter 2: Exceptions in Python
Exceptions Are Defects We Know About
Reading the Code
SyntaxError
Best Practices for Debugging SyntaxErrors
Examining the Error Message
The Error Type
The Error Description
The Traceback
Deduction
Catching Exceptions
Best Practices for Debugging IOErrors
Errors and Defects
Where Do Defects Come From?
The Correct Code
Best Practices
Chapter 3: Semantic Errors in Python
Comparing Expected and Factual Output
Defects
Defects in Variable Assignments
Multiple Initialization
Accidental Assignment
Accidental Comparison
Wrong Variables in an Expression
Swapped Literals in Expression
Defects in Indexing
Creating Wrong Indices
Using Wrong Indices
Defects in Control Flow Statements
Defects in Boolean Expressions
Defects with Indentation
Defects in Using Functions
Omitting a Function Call
Missing Return Statement
Not Storing the Return Value
Error Propagation
Best Practices
Chapter 4: Debugging with the Scientific Method
Applying the Scientific Method
Reproducing the Error
Automating the Error
Isolating the Defect
The Strip-Down Strategy
The Binary Search Strategy
Getting Help
Taking a Break
Explain the Problem to Someone Else
Pair Programming
Code Reviews
Reading
Cleaning Up
The Scientific Method and Other Best Practices
Best Practices
Chapter 5: Debugging with print Statements
Diagnose Whether Code Was Executed
Print the Content of Variables
Pretty-Printing Data Structures
Simplify Input Data
Start with Minimal Input
Gradually Add More Input Data
Switching print Output On and Off
Complete Code
Pros and Cons of Using print Statements
Best Practices
Chapter 6: Debugging with Introspection Functions
Explorative Coding in IPython
Exploring Files and Directories
Overview of IPython Commands
Exploring Namespaces
Exploring Namespaces with dir()
Exploring Namespaces of Objects
Exploring Attributes in a Python Program
Alternatives to dir in IPython
Namespace Mechanics
Python Uses Namespaces for Its Own Functions
Modifying a Namespace
Namespaces and Local Scope
Namespaces Are a Core Feature of Python
Using Self-Documenting Objects
Accessing Docstrings with help()
Object Summaries in IPython
Analyzing Object Types
Checking Object Identity
Checking Instances and Subclasses
Practical Use of Introspection
Finding Typos with Introspection
Combining Introspection Functions
Introspection in Big and Small Programs
Best Practices
Chapter 7: Using an Interactive Debugger
The Interactive Debugger ipdb
Installing ipdb
Starting the Debugger
Running ipdb from the Command Line
Starting ipdb from a Program
Postmortem Debugging
Launching the Debugger on Exceptions
Fixing the Defect
Commands at the Debugger Prompt
Inspect Variables
Evaluate Python Expressions
Stepping Through Our Code
Start Over
Using Breakpoints
Viewing and Deleting Breakpoints
Conditional Breakpoints
Configuring ipdb
Example ipdb Session
Adding a Game Control Function
Stepping Through the Code
Fixing the Defect
It Is Working!
Is the Program Without Defects Now?
Other Debugging Tools
pdb, the Python Debugger
The PyCharm IDE
ipdbplugin
pudb
wdb
django-debug-toolbar
cProfile
Best Practices
Part II: Automated Testing
Chapter 8: Writing Automated Tests
Installing py.test
Writing a Test Function
Running Tests
Writing a Failing Test
Making the Test Pass
Passing Versus Failing Tests
Writing Separate Test Functions
Assertions Provide Helpful Output
Testing for Exceptions
Border Cases
Complex Border Cases
Benefits of Automated Testing
Other Test Frameworks in Python
unittest
nose
doctest
Writing a __main__ block
Best Practices
Chapter 9: Organizing Test Data
Using Fixtures
The scope Parameter
Test Parameterization
Multiple Parameters
Parametrized Fixtures
Mocking
Testing Output Files
Cleaning Up After Tests
Using Temporary Files
Comparing Output Files with Test Data
The filecmp Module
The difflib Module
Best Practices for Tests Involving Large Files
Generating Random Test Data
Where to Store Test Data?
Test Data Modules
Test Data Directories
Test Databases
Best Practices
Chapter 10: Writing a Test Suite
Test Modules
Test Classes
Refactoring Test Functions
Fixtures in Test Classes
How Do Tests Find the Tested Code?
Multiple Test Packages
Test Autodiscovery
Executing a Test Suite
Partial Execution
Executing Test Modules and Packages
Executing Test Classes
Executing Single Tests
Selecting Tests by Keywords
Examining Failures
Rerunning Tests
Calculating Test Coverage
A Test Suite Needs Maintenance
Best Practices
Chapter 11: Testing Best Practices
Types of Automated Tests
Unit Tests
Integration Tests
Acceptance Tests
Regression Tests
Performance Tests
Performance Optimization
The Test-First Approach
Writing Tests Against a Specification
Writing Tests Against Defects
Test-Driven Development
Advantages of Automated Testing
Testing Saves Time
Testing Adds Precision
Testing Makes Collaboration Easier
Limitations of Automated Testing
Testing Requires Testable Code
Testing Does Not Work Well for Projects Evolving Quickly
Testing Does Not Prove Correctness
Programs That Are Difficult to Test
Random Numbers
Graphical User Interfaces
Complex or Large Output
Concurrency
Situations Where Automated Tests Fail
Alternatives to Automated Testing
Prototyping
Code Reviews
Checklists
Processes Promoting Correctness
Conclusions
Best Practices
Part III: Maintenance
Chapter 12: Version Control
Getting Started with git
Creating a Repository
Adding Files to a Repository
Tracking Changes in Files
Moving and Deleting Files
Discarding Changes
Navigating the History of Our Code
Checking out Older Commits
Traveling Back to the Most Recent Commit
Publishing Code on GitHub
Starting a Project on GitHub
Using GitHub as a Single Contributor
Working on Projects Started by Other People
Projects with Multiple Contributors
Merging Changes by Two People
Pull Requests
Development with Branches
Merging Branches
Configuring git
Ignoring Files
Global Settings
Usage Examples
Twenty Characters: A Small Project with Low Traffic
Python: A Huge Project with Daily Commits
grep: A Long-Term Project
Other Version Control Systems
Mercurial
Subversion (SVN)
Concurrent Versions Software (CVS)
Bitbucket
Sourceforge
Best Practices
Chapter 13: Setting Up a Python Project
Creating a Project Structure with pyscaffold
Installing pyscaffold
Typical Directories in a Python Project
Directories Created by pyscaffold
The Main Python Package Directory
The tests/ Directory
The docs/ Directory
The .git / Directory
Directories Not Created by pyscaffold
The bin/ Directory
The Directories build/, dist/, and sdist/
The .hg/ Directory
Data Directories
Files
Files Created by pyscaffold
README.rst
setup.py
AUTHORS.rst
LICENSE.rst
MANIFEST.in
versioneer.py
requirements.txt
.coveragerc
.gitattributes and .gitignore
Files Not Created by pyscaffold
Setting the Version Number of Our Program
Managing a Python Project Environment with virtualenv
Installing virtualenv
Connecting a Project to virtualenv
Working with a virtualenv Project
Installing Packages in virtualenv
Leaving a virtualenv Session
Configuring virtualenv Startup and Deactivation
Setting the PYTHONPATH Variable
Installing Pygame with virtualenv
Best Practices
Chapter 14: Cleaning Up Code
Organized and Unorganized Code
Software Entropy: Causes of Unorganized Code
How to Recognize Unorganized Code?
Readability
Structural Weaknesses
Redundancy
Design Weaknesses
Cleaning Up Python Instructions
Place import Statements Together
Place Constants Together
Remove Unnecessary Lines
Choose Meaningful Variable Names
Idiomatic Python Code
Refactoring
Extract Functions
Create a Simple Command-Line Interface
Structuring Programs into Modules
The Cleaned Code
PEP8 and pylint
Warning Messages
Code Score
Make It Work, Make It Right, Make It Fast
Make It Work
Make It Right
Make It Fast
Examples of Well-Organized Code
Best Practices
Chapter 15: Decomposing Programming Tasks
Decomposing Programming Tasks Is Difficult
A Process to Decompose Programming Tasks
Write a User Story
Add Details to the Description
Acceptance Criteria
Use Case Descriptions
Check Nonfunctional Requirements
Identify Problems
Incomplete Information
Domain Expertise
Changing Existing Code
Anticipating Future Change
Decide on an Architecture
Identify Program Components
Implement
Other Planning Tools
The One-Page Project Plan
Issue Trackers
Kanban
Best Practices
Chapter 16: Static Typing in Python
Weaknesses of Dynamic Typing
Function Signatures
Value Boundaries
Semantic Meaning of Types
Composite Types
Is Stronger Typing Possible in Python?
Assertions
NumPy
Databases
Integrating C Code
Cython
Type Hints
mypy
Which Method of Type Control to Use?
Best Practices
Chapter 17: Documentation
Who Do We Write Documentation For?
Sphinx: A Documentation Tool for Python
Setting Up Sphinx
Files Created by Sphinx
Building the Documentation
Building HTML Documentation
Building PDF Documentation
Building EPUB Documentation
Writing Documentation
Directives
Organizing Documents
Code Examples
Generating Documentation from Docstrings
Doctests
Configuring Sphinx
Todo Entries
Creating a Todo-List
Conditional Building
Changing the Look and Feel
How to Write Good Documentation?
Text Sections in Technical Documentation
Summary
Prerequisites and Installation
Getting Started
Cookbook
Case Studies
Technical Reference
Design Documentation
Legal Aspects
Examples of Good Documentation
Other Documentation Tools
MkDocs
Jupyter Notebooks
Gitbook
Read the Docs
pydoc
S5
pygments
doctest
PyPDF2
pandoc
Best Practices
Index