Python How-To: 63 techniques to improve your Python code

Python How-To
contents
preface
acknowledgments
about this book
Who should read this book
How this book is organized: A road map
About the appendices
About the code
liveBook discussion forum
Other online resources
about the author
about the cover illustration
Chapter 1: Developing a pragmatic learning strategy
1.1 Aiming at becoming a pragmatic programmer
1.1.1 Focusing on writing readable Python code
1.1.2 Considering maintainability even before you write any code
1.2 What Python can do well or as well as other languages
1.3 What Python can’t do or can’t do well
1.4 What you’ll learn in this book
1.4.1 Focusing on domain-independent knowledge
1.4.2 Solving problems through synthesis
1.4.3 Learning skills in context
Part 1: Using built-in data models
Chapter 2: Processing and formatting strings
2.1 How do I use f-strings for string interpolation and formatting?
2.1.1 Formatting strings before f-strings
2.1.2 Using f-strings to interpolate variables
2.1.3 Using f-strings to interpolate expressions
2.1.4 Applying specifiers to format f-strings
2.1.5 Discussion
2.1.6 Challenge
2.2 How do I convert strings to retrieve the represented data?
2.2.1 Checking whether strings represent alphanumeric values
2.2.2 Casting strings to numbers
2.2.3 Evaluating strings to derive their represented data
2.2.4 Discussion
2.2.5 Challenge
2.3 How do I join and split strings?
2.3.1 Joining strings with whitespaces
2.3.2 Joining strings with any delimiters
2.3.3 Splitting strings to create a list of strings
2.3.4 Discussion
2.3.5 Challenge
2.4 What are the essentials of regular expressions?
2.4.1 Using regular expressions in Python
2.4.2 Creating the pattern with a raw string
2.4.3 Understanding the essentials of a search pattern
2.4.4 Dissecting the matches
2.4.5 Knowing the common methods
2.4.6 Discussion
2.4.7 Challenge
2.5 How do I use regular expressions to process texts?
2.5.1 Creating a working pattern to find the matches
2.5.2 Extracting the needed data from the matches
2.5.3 Using named groups for text processing
2.5.4 Discussion
2.5.5 Challenge
Chapter 3: Using built-in data containers
3.1 How do I choose between lists and tuples?
3.1.1 Using tuples for immutability and using lists for mutability
3.1.2 Using tuples for heterogeneity and using lists for homogeneity
3.1.3 Discussion
3.1.4 Challenge
3.2 How do I sort lists of complicated data using custom functions?
3.2.1 Sorting lists using the default order
3.2.2 Using a built-in function as the sorting key
3.2.3 Using custom functions for more complicated sorting needs
3.2.4 Discussion
3.2.5 Challenge
3.3 How do I build a lightweight data model using named tuples?
3.3.1 Understanding alternative data models
3.3.2 Creating named tuples to hold data
3.3.3 Discussion
3.3.4 Challenge
3.4 How do I access dictionary keys, values, and items?
3.4.1 Using dynamic view objects (keys, values, and items) directly
3.4.2 Being cautious with the KeyError exception
3.4.3 Avoiding KeyError with a hygiene check first: The non-Pythonic way
3.4.4 Using the get method to access a dictionary item
3.4.5 Watching for the setdefault method’s side effect
3.4.6 Discussion
3.4.7 Challenge
3.5 When do I use dictionaries and sets instead of lists and tuples?
3.5.1 Taking advantage of the constant lookup efficiency
3.5.2 Understanding hashable and hashing
3.5.3 Discussion
3.5.4 Challenge
3.6 How do I use set operations to check the relationships between lists?
3.6.1 Checking whether a list contains all items of another list
3.6.2 Checking whether a list contains any element of another list
3.6.3 Dealing with multiple set objects
3.6.4 Discussion
3.6.5 Challenge
Chapter 4: Dealing with sequence data
4.1 How do I retrieve and manipulate subsequences with slice objects?
4.1.1 Taking advantage of the full features of slicing
4.1.2 Not confusing slices with ranges
4.1.3 Using named slice objects to process sequence data
4.1.4 Manipulating list items with slicing operations
4.1.5 Discussion
4.1.6 Challenge
4.2 How do I use positive and negative indexing to retrieve items?
4.2.1 Positive indexing starts from the beginning of the list
4.2.2 Negative indexing starts from the end of the list
4.2.3 Combining positive and negative indices as needed
4.2.4 Discussion
4.2.5 Challenge
4.3 How do I find items in a sequence?
4.3.1 Checking an item’s presence
4.3.2 Using the index method to locate the item
4.3.3 Finding substrings in a string
4.3.4 Finding an instance of custom classes in a list
4.3.5 Discussion
4.3.6 Challenge
4.4 How do I unpack a sequence? Beyond tuple unpacking
4.4.1 Unpacking short sequences with one-to-one correspondence
4.4.2 Retrieving consecutive items using the starred expression
4.4.3 Denoting unwanted items with underscores to remove distraction
4.4.4 Discussion
4.4.5 Challenge
4.5 When should I consider data models other than lists and tuples?
4.5.1 Using sets where membership is concerned
4.5.2 Using deques if you care about first-in-first-out
4.5.3 Processing multidimensional data with NumPy and Pandas
4.5.4 Discussion
4.5.5 Challenge
Chapter 5: Iterables and iterations
5.1 How do I create common data containers using iterables?
5.1.1 Getting to know iterables and iterators
5.1.2 Inspecting iterability
5.1.3 Using iterables to create built-in data containers
5.1.4 Discussion
5.1.5 Challenge
5.2 What are list, dictionary, and set comprehensions?
5.2.1 Creating lists from iterables using list comprehension
5.2.2 Creating dictionaries from iterables using dictionary comprehension
5.2.3 Creating sets from iterables using set comprehension
5.2.4 Applying a filtering condition
5.2.5 Using embedded for loops
5.2.6 Discussion
5.2.7 Challenge
5.3 How do I improve for-loop iterations with built-in functions?
5.3.1 Enumerating items with enumerate
5.3.2 Reversing items with reversed
5.3.3 Aligning iterables with zip
5.3.4 Chaining multiple iterables with chain
5.3.5 Filtering the iterable with filter
5.3.6 Discussion
5.3.7 Challenge
5.4 Using optional statements within for and while loops
5.4.1 Exiting the loops with the break statement
5.4.2 Skipping an iteration with the continue statement
5.4.3 Using else statements in the for and while loops
5.4.4 Discussion
5.4.5 Challenge
Part 2: Defining functions
Chapter 6: Defining user-friendly functions
6.1 How do I set default arguments to make function calls easier?
6.1.1 Calling functions with default arguments
6.1.2 Defining functions with default arguments
6.1.3 Avoiding the pitfall of setting default arguments for mutable parameters
6.1.4 Discussion
6.1.5 Challenge
6.2 How do I set and use the return value in function calls?
6.2.1 Returning a value implicitly or explicitly
6.2.2 Defining functions returning zero, one, or multiple values
6.2.3 Using multiple values returned from a function call
6.2.4 Discussion
6.2.5 Challenge
6.3 How do I use type hints to write understandable functions?
6.3.1 Providing type hinting to variables
6.3.2 Using type hinting in function definitions
6.3.3 Applying advanced type-hinting skills to function definitions
6.3.4 Discussion
6.3.5 Challenge
6.4 How do I increase function flexibility with *args and **kwargs?
6.4.1 Knowing positional and keyword arguments
6.4.2 Accepting a variable number of positional arguments
6.4.3 Accepting a variable number of keyword arguments
6.4.4 Discussion
6.4.5 Challenge
6.5 How do I write proper docstrings for a function?
6.5.1 Examining the basic structure of a function's docstring
6.5.2 Specifying the function's action as the summary
6.5.3 Documenting the parameters and the return value
6.5.4 Specifying any exceptions possibly raised
6.5.5 Discussion
6.5.6 Challenge
Chapter 7: Using functions beyond the basics
7.1 How do I use lambda functions for small jobs?
7.1.1 Creating a lambda function
7.1.2 Using lambdas to perform a small one-time job
7.1.3 Avoiding pitfalls when using lambda functions
7.1.4 Discussion
7.1.5 Challenge
7.2 What are the implications of functions as objects?
7.2.1 Storing functions in a data container
7.2.2 Sending functions as arguments to higher-order functions
7.2.3 Using functions as a return value
7.2.4 Discussion
7.2.5 Challenge
7.3 How do I check functions’ performance with decorators?
7.3.1 Decorating a function to show its performance
7.3.2 Dissecting the decorator function
7.3.3 Wrapping to carry over the decorated function’s metadata
7.3.4 Discussion
7.3.5 Challenge
7.4 How can I use generator functions as a memory- efficient data provider?
7.4.1 Creating a generator to yield perfect squares
7.4.2 Using generators for their memory efficiency
7.4.3 Using generator expressions where applicable
7.4.4 Discussion
7.4.5 Challenge
7.5 How do I create partial functions to make routine function calls easier?
7.5.1 “Localizing” shared functions to simplify function calls
7.5.2 Creating a partial function to localize a function
7.5.3 Discussion
7.5.4 Challenge
Part 3: Defining classes
Chapter 8: Defining user-friendly classes
8.1 How do I define the initialization method for a class?
8.1.1 Demystifying self: The first parameter in __init__
8.1.2 Setting proper arguments in __init__
8.1.3 Specifying all attributes in __init__
8.1.4 Defining class attributes outside the __init__ method
8.1.5 Discussion
8.1.6 Challenge
8.2 When do I define instance, static, and class methods?
8.2.1 Defining instance methods for manipulating individual instances
8.2.2 Defining static methods for utility functionalities
8.2.3 Defining class methods for accessing class-level attributes
8.2.4 Discussion
8.2.5 Challenge
8.3 How do I apply finer access control to a class?
8.3.1 Creating protected methods by using an underscore as the prefix
8.3.2 Creating private methods by using double underscores as the prefix
8.3.3 Creating read-only attributes with the property decorator
8.3.4 Verifying data integrity with a property setter
8.3.5 Discussion
8.3.6 Challenge
8.4 How do I customize string representation for a class?
8.4.1 Overriding __str__ to show meaningful information for an instance
8.4.2 Overriding __repr__ to provide instantiation information
8.4.3 Understanding the differences between __str__ and __repr__
8.4.4 Discussion
8.4.5 Challenge
8.5 Why and how do I create a superclass and subclasses?
8.5.1 Identifying the use scenario of subclasses
8.5.2 Inheriting the superclass's attributes and methods automatically
8.5.3 Overriding the superclass's methods to provide customized behaviors
8.5.4 Creating non-public methods of the superclass
8.5.5 Discussion
8.5.6 Challenge
Chapter 9: Using classes beyond the basics
9.1 How do I create enumerations?
9.1.1 Avoiding a regular class for enumerations
9.1.2 Creating an enumeration class
9.1.3 Using enumerations
9.1.4 Defining methods for the enumeration class
9.1.5 Discussion
9.1.6 Challenge
9.2 How do I use data classes to eliminate boilerplate code?
9.2.1 Creating a data class using the dataclass decorator
9.2.2 Setting default values for the fields
9.2.3 Making data classes immutable
9.2.4 Creating a subclass of an existing data class
9.2.5 Discussion
9.2.6 Challenge
9.3 How do I prepare and process JSON data?
9.3.1 Understanding JSON’s data structure
9.3.2 Mapping data types between JSON and Python
9.3.3 Deserializing JSON strings
9.3.4 Serializing Python data to JSON format
9.3.5 Discussion
9.3.6 Challenge
9.4 How do I create lazy attributes to improve performance?
9.4.1 Identifying the use scenario
9.4.2 Overriding the __getattr_ special method to implement lazy attributes
9.4.3 Implementing a property as a lazy attribute
9.4.4 Discussion
9.4.5 Challenge
9.5 How do I define classes to have distinct concerns?
9.5.1 Analyzing a class
9.5.2 Creating additional classes to isolate the concerns
9.5.3 Connecting related classes
9.5.4 Discussion
9.5.5 Challenge
Part 4: Manipulating objects and files
Chapter 10: Fundamentals of objects
10.1 How do I inspect an object’s type to improve code flexibility?
10.1.1 Checking an object’s type using type
10.1.2 Checking an object’s type using isinstance
10.1.3 Checking an object’s type generically
10.1.4 Discussion
10.1.5 Challenge
10.2 What’s the lifecycle of instance objects?
10.2.1 Instantiating an object
10.2.2 Being active in applicable namespaces
10.2.3 Tracking reference counts
10.2.4 Destructing the object
10.2.5 Discussion
10.2.6 Challenge
10.3 How do I copy an object?
10.3.1 Creating a (shallow) copy
10.3.2 Noting the potential problem of a shallow copy
10.3.3 Creating a deep copy
10.3.4 Discussion
10.3.5 Challenge
10.4 How do I access and change a variable in a different scope?
10.4.1 Accessing any variable: The LEGB rule for name lookup
10.4.2 Changing a global variable in a local scope
10.4.3 Changing an enclosing variable
10.4.4 Discussion
10.4.5 Challenge
10.5 What’s callability, and what does it imply?
10.5.1 Distinguishing classes from functions
10.5.2 Revisiting the higher-order function map
10.5.3 Using callable as the key argument
10.5.4 Creating decorators as classes
10.5.5 Discussion
10.5.6 Challenge
Chapter 11: Dealing with files
11.1 How do I read and write files using context management?
11.1.1 Opening and closing files: Context manager
11.1.2 Reading data from a file in different ways
11.1.3 Writing data to a file in different ways
11.1.4 Discussion
11.1.5 Challenge
11.2 How do I deal with tabulated data files?
11.2.1 Reading a CSV file using csv reader
11.2.2 Reading a CSV file that has a header
11.2.3 Writing data to a CSV file
11.2.4 Discussion
11.2.5 Challenge
11.3 How do I preserve data as files using pickling?
11.3.1 Pickling objects for data preservation
11.3.2 Restoring data by unpickling
11.3.3 Weighing the pros and cons of pickling
11.3.4 Discussion
11.3.5 Challenge
11.4 How do I manage files on my computer?
11.4.1 Creating a directory and files
11.4.2 Retrieving the list of files of a specific kind
11.4.3 Moving files to a different folder
11.4.4 Copying files to a different folder
11.4.5 Deleting a specific kind of files
11.4.6 Discussion
11.4.7 Challenge
11.5 How do I retrieve file metadata?
11.5.1 Retrieving the filename-related information
11.5.2 Retrieving the file's size and time information
11.5.3 Discussion
11.5.4 Challenge
Part 5: Safeguarding the codebase
Chapter 12: Logging and exception handling
12.1 How do I monitor my program with logging?
12.1.1 Creating the Logger object to log application events
12.1.2 Using files to store application events
12.1.3 Adding multiple handlers to the logger
12.1.4 Discussion
12.1.5 Challenge
12.2 How do I save log records properly?
12.2.1 Categorizing application events with levels
12.2.2 Setting a handler’s level
12.2.3 Setting formats to the handler
12.2.4 Discussion
12.2.5 Challenge
12.3 How do I handle exceptions?
12.3.1 Handling exceptions with try. . .except. . .
12.3.2 Specifying the exception in the except clause
12.3.3 Handling multiple exceptions
12.3.4 Showing more information about an exception
12.3.5 Discussion
12.3.6 Challenge
12.4 How do I use else and finally clauses in exception handling?
12.4.1 Using else to continue the logic of the code in the try clause
12.4.2 Cleaning up the exception handling with the finally clause
12.4.3 Discussion
12.4.4 Challenge
12.5 How do I raise informative exceptions with custom exception classes?
12.5.1 Raising exceptions with a custom message
12.5.2 Preferring built-in exception classes
12.5.3 Defining custom exception classes
12.5.4 Discussion
12.5.5 Challenge
Chapter 13: Debugging and testing
13.1 How do I spot problems with tracebacks?
13.1.1 Understanding how a traceback is generated
13.1.2 Analyzing a traceback when running code in a console
13.1.3 Analyzing a traceback when running a script
13.1.4 Focusing on the last call in a traceback
13.1.5 Discussion
13.1.6 Challenge
13.2 How do I debug my program interactively?
13.2.1 Activating the debugger with a breakpoint
13.2.2 Running code line by line
13.2.3 Stepping into another function
13.2.4 Inspecting pertinent variables
13.2.5 Discussion
13.2.6 Challenge
13.3 How do I test my functions automatically?
13.3.1 Understanding the basis for testing functions
13.3.2 Creating a TestCase subclass for testing functions
13.3.3 Setting up the test
13.3.4 Discussion
13.3.5 Challenge
13.4 How do I test a class automatically?
13.4.1 Creating a TestCase subclass for testing a class
13.4.2 Responding to test failures
13.4.3 Discussion
13.4.4 Challenge
Part 6: Building a web app
Chapter 14: Completing a real project
14.1 How do I use a virtual environment for my project?
14.1.1 Understanding the rationale for virtual environments
14.1.2 Creating a virtual environment for each project
14.1.3 Installing packages in the virtual environment
14.1.4 Using virtual environments in Visual Studio Code
14.1.5 Discussion
14.1.6 Challenge
14.2 How do I build the data models for my project?
14.2.1 Identifying the business needs
14.2.2 Creating helper classes and functions
14.2.3 Creating the Task class to address these needs
14.2.4 Discussion
14.2.5 Challenge
14.3 How do I use SQLite as my application’s database?
14.3.1 Creating the database
14.3.2 Retrieving records from the database
14.3.3 Saving records to the database
14.3.4 Updating a record in a database
14.3.5 Deleting a record from the database
14.3.6 Discussion
14.3.7 Challenge
14.4 How do I build a web app as the frontend?
14.4.1 Understanding the essential features of streamlit
14.4.2 Understanding the app’s interface
14.4.3 Tracking user activities using session state
14.4.4 Setting up the sidebar
14.4.5 Showing the tasks
14.4.6 Showing a task’s details
14.4.7 Creating a new task
14.4.8 Organizing your project
14.4.9 Running the app
14.4.10 Discussion
14.4.11 Challenge
solutions to the challenges
Chapter 1
Chapter 2
Section 2.1
Section 2.2
Section 2.3
Section 2.4
Section 2.5
Chapter 3
Section 3.1
Section 3.2
Section 3.3
Section 3.4
Section 3.5
Section 3.6
Chapter 4
Section 4.1
Section 4.2
Section 4.3
Section 4.4
Section 4.5
Chapter 5
Section 5.1
Section 5.2
Section 5.3
Section 5.4
Chapter 6
Section 6.1
Section 6.2
Section 6.3
Section 6.4
Section 6.5
Chapter 7
Section 7.1
Section 7.2
Section 7.3
Section 7.4
Section 7.5
Chapter 8
Section 8.1
Section 8.2
Section 8.3
Section 8.4
Section 8.5
Chapter 9
Section 9.1
Section 9.2
Section 9.3
Section 9.4
Section 9.5
Chapter 10
Section 10.1
Section 10.2
Section 10.3
Section 10.4
Section 10.5
Chapter 11
Section 11.1
Section 11.2
Section 11.3
Section 11.4
Section 11.5
Chapter 12
Section 12.1
Section 12.2
Section 12.3
Section 12.4
Section 12.5
Chapter 13
Section 13.1
Section 13.2
Section 13.3
Section 13.4
Chapter 14
Section 14.1
Section 14.2
Section 14.3
Section 14.4
index
Symbols
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
Y
Z