High Performance Python: Practical Performant Programming for Humans

Cover
Copyright
Table of Contents
Foreword
Preface
Who This Book Is For
Who This Book Is Not For
What You’ll Learn
Python 3
Changes from Python 2.7
License
How to Make an Attribution
Errata and Feedback
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
Chapter 1. Understanding Performant Python
The Fundamental Computer System
Computing Units
Memory Units
Communications Layers
Putting the Fundamental Elements Together
Idealized Computing Versus the Python Virtual Machine
So Why Use Python?
How to Be a Highly Performant Programmer
Good Working Practices
Some Thoughts on Good Notebook Practice
Getting the Joy Back into Your Work
Chapter 2. Profiling to Find Bottlenecks
Profiling Efficiently
Introducing the Julia Set
Calculating the Full Julia Set
Simple Approaches to Timing—print and a Decorator
Simple Timing Using the Unix time Command
Using the cProfile Module
Visualizing cProfile Output with SnakeViz
Using line_profiler for Line-by-Line Measurements
Using memory_profiler to Diagnose Memory Usage
Introspecting an Existing Process with PySpy
Bytecode: Under the Hood
Using the dis Module to Examine CPython Bytecode
Different Approaches, Different Complexity
Unit Testing During Optimization to Maintain Correctness
No-op @profile Decorator
Strategies to Profile Your Code Successfully
Wrap-Up
Chapter 3. Lists and Tuples
A More Efficient Search
Lists Versus Tuples
Lists as Dynamic Arrays
Tuples as Static Arrays
Wrap-Up
Chapter 4. Dictionaries and Sets
How Do Dictionaries and Sets Work?
Inserting and Retrieving
Deletion
Resizing
Hash Functions and Entropy
Dictionaries and Namespaces
Wrap-Up
Chapter 5. Iterators and Generators
Iterators for Infinite Series
Lazy Generator Evaluation
Wrap-Up
Chapter 6. Matrix and Vector Computation
Introduction to the Problem
Aren’t Python Lists Good Enough?
Problems with Allocating Too Much
Memory Fragmentation
Understanding perf
Making Decisions with perf’s Output
Enter numpy
Applying numpy to the Diffusion Problem
Memory Allocations and In-Place Operations
Selective Optimizations: Finding What Needs to Be Fixed
numexpr: Making In-Place Operations Faster and Easier
A Cautionary Tale: Verify “Optimizations” (scipy)
Lessons from Matrix Optimizations
Pandas
Pandas’s Internal Model
Applying a Function to Many Rows of Data
Building DataFrames and Series from Partial Results Rather than Concatenating
There’s More Than One (and Possibly a Faster) Way to Do a Job
Advice for Effective Pandas Development
Wrap-Up
Chapter 7. Compiling to C
What Sort of Speed Gains Are Possible?
JIT Versus AOT Compilers
Why Does Type Information Help the Code Run Faster?
Using a C Compiler
Reviewing the Julia Set Example
Cython
Compiling a Pure Python Version Using Cython
pyximport
Cython Annotations to Analyze a Block of Code
Adding Some Type Annotations
Cython and numpy
Parallelizing the Solution with OpenMP on One Machine
Numba
Numba to Compile NumPy for Pandas
PyPy
Garbage Collection Differences
Running PyPy and Installing Modules
A Summary of Speed Improvements
When to Use Each Technology
Other Upcoming Projects
Graphics Processing Units (GPUs)
Dynamic Graphs: PyTorch
Basic GPU Profiling
Performance Considerations of GPUs
When to Use GPUs
Foreign Function Interfaces
ctypes
cffi
f2py
CPython Module
Wrap-Up
Chapter 8. Asynchronous I/O
Introduction to Asynchronous Programming
How Does async/await Work?
Serial Crawler
Gevent
tornado
aiohttp
Shared CPU–I/O Workload
Serial
Batched Results
Full Async
Wrap-Up
Chapter 9. The multiprocessing Module
An Overview of the multiprocessing Module
Estimating Pi Using the Monte Carlo Method
Estimating Pi Using Processes and Threads
Using Python Objects
Replacing multiprocessing with Joblib
Random Numbers in Parallel Systems
Using numpy
Finding Prime Numbers
Queues of Work
Verifying Primes Using Interprocess Communication
Serial Solution
Naive Pool Solution
A Less Naive Pool Solution
Using Manager.Value as a Flag
Using Redis as a Flag
Using RawValue as a Flag
Using mmap as a Flag
Using mmap as a Flag Redux
Sharing numpy Data with multiprocessing
Synchronizing File and Variable Access
File Locking
Locking a Value
Wrap-Up
Chapter 10. Clusters and Job Queues
Benefits of Clustering
Drawbacks of Clustering
$462 Million Wall Street Loss Through Poor Cluster Upgrade Strategy
Skype’s 24-Hour Global Outage
Common Cluster Designs
How to Start a Clustered Solution
Ways to Avoid Pain When Using Clusters
Two Clustering Solutions
Using IPython Parallel to Support Research
Parallel Pandas with Dask
NSQ for Robust Production Clustering
Queues
Pub/sub
Distributed Prime Calculation
Other Clustering Tools to Look At
Docker
Docker’s Performance
Advantages of Docker
Wrap-Up
Chapter 11. Using Less RAM
Objects for Primitives Are Expensive
The array Module Stores Many Primitive Objects Cheaply
Using Less RAM in NumPy with NumExpr
Understanding the RAM Used in a Collection
Bytes Versus Unicode
Efficiently Storing Lots of Text in RAM
Trying These Approaches on 11 Million Tokens
Modeling More Text with Scikit-Learn’s FeatureHasher
Introducing DictVectorizer and FeatureHasher
Comparing DictVectorizer and FeatureHasher on a Real Problem
SciPy’s Sparse Matrices
Tips for Using Less RAM
Probabilistic Data Structures
Very Approximate Counting with a 1-Byte Morris Counter
K-Minimum Values
Bloom Filters
LogLog Counter
Real-World Example
Chapter 12. Lessons from the Field
Streamlining Feature Engineering Pipelines with Feature-engine
Feature Engineering for Machine Learning
The Hard Task of Deploying Feature Engineering Pipelines
Leveraging the Power of Open Source Python Libraries
Feature-engine Smooths Building and Deployment of Feature Engineering Pipelines
Helping with the Adoption of a New Open Source Package
Developing, Maintaining, and Encouraging Contribution to Open Source Libraries
Highly Performant Data Science Teams
How Long Will It Take?
Discovery and Planning
Managing Expectations and Delivery
Numba
A Simple Example
Best Practices and Recommendations
Getting Help
Optimizing Versus Thinking
Adaptive Lab’s Social Media Analytics (2014)
Python at Adaptive Lab
SoMA’s Design
Our Development Methodology
Maintaining SoMA
Advice for Fellow Engineers
Making Deep Learning Fly with RadimRehurek.com (2014)
The Sweet Spot
Lessons in Optimizing
Conclusion
Large-Scale Productionized Machine Learning at Lyst.com (2014)
Cluster Design
Code Evolution in a Fast-Moving Start-Up
Building the Recommendation Engine
Reporting and Monitoring
Some Advice
Large-Scale Social Media Analysis at Smesh (2014)
Python’s Role at Smesh
The Platform
High Performance Real-Time String Matching
Reporting, Monitoring, Debugging, and Deployment
PyPy for Successful Web and Data Processing Systems (2014)
Prerequisites
The Database
The Web Application
OCR and Translation
Task Distribution and Workers
Conclusion
Task Queues at Lanyrd.com (2014)
Python’s Role at Lanyrd
Making the Task Queue Performant
Reporting, Monitoring, Debugging, and Deployment
Advice to a Fellow Developer
Index
About the Authors
Colophon