Attain a deep understanding of building, maintaining, packaging, and shipping robust Python applications
Key Features
• Discover the new features of Python, such as dictionary merge, the zoneinfo module, and structural pattern matching
• Create manageable code to run in various environments with different sets of dependencies
• Implement effective Python data structures and algorithms to write, test, and optimize code
Book Description
Python is used in a wide range of domains owing to its simple yet powerful nature. Although writing Python code is easy, making it readable, reusable, and easy to maintain can be challenging. Complete with best practices, useful tools, and standards implemented by professional Python developers, this fourth edition will help you in not only overcoming such challenges but also learning Python’s latest features and advanced concepts.
The book begins with a warm-up, where you will catch-up with the latest Python improvements, syntax elements, and interesting tools to boost your development efficiency. Further, the initial few chapters should allow experienced programmers coming from different languages to safely land in the Python ecosystem.
As you progress, you will explore common software design patterns and various programming methodologies, such as event-driven programming, concurrency, and metaprogramming. You will also go through complex code examples and try to solve meaningful problems by bridging Python with C and C++, writing extensions that benefit from the strengths of multiple languages. Finally, you will understand the complete lifetime of any application after it goes live.
By the end of the book, you should be proficient in writing efficient and maintainable Python code.
What you will learn
• Explore modern ways of setting up repeatable and consistent Python development environments
• Effectively package Python code for community and production use
• Learn modern syntax elements of Python programming, such as f-strings, enums, and lambda functions
• Demystify metaprogramming in Python with metaclasses
• Write concurrent code in Python
• Extend and integrate Python with code written in different languages
Who This Book Is For
The Python programming book is intended for expert programmers who want to learn Python’s advanced-level concepts and latest features.
Anyone who has basic Python skills should be able to follow the content of the book, although it might require some additional effort from less experienced programmers. It should also be a good introduction to Python 3.9 for those who are still a bit behind and continue to use other older versions.
Author(s): Michal Jaworski, Tarek Ziadé
Edition: 4
Publisher: Packt Publishing
Year: 2021
Language: English
Commentary: Vector PDF
Pages: 630
City: Birmingham, UK
Tags: Debugging; Multithreading; Python; Concurrency; Asynchronous Programming; Monitoring; Docker; Best Practices; Application Development; Object-Oriented Programming; Packages; Graph Algorithms; Callback Functions; Event-Driven Programming; PyTest; Testing; Multiprocessing; iPython; Metaprogramming; Code Optimization; Polymorphism; Poetry
Cover
Copyright
Contributors
Table of Contents
Preface
Chapter 1: Current Status of Python
Where are we now and where are we going?
What to do with Python 2
Keeping up to date
PEP documents
Active communities
Other resources
Summary
Chapter 2: Modern Python Development Environments
Technical requirements
Python's packaging ecosystem
Installing Python packages using pip
Isolating the runtime environment
Application-level isolation versus system-level isolation
Application-level environment isolation
Poetry as a dependency management system
System-level environment isolation
Containerization versus virtualization
Virtual environments using Docker
Writing your first Dockerfile
Running containers
Setting up complex environments
Useful Docker and Docker Compose recipes for Python
Virtual development environments using Vagrant
Popular productivity tools
Custom Python shells
Using IPython
Incorporating shells in your own scripts and programs
Interactive debuggers
Other productivity tools
Summary
Chapter 3: New Things in Python
Technical requirements
Recent language additions
Dictionary merge and update operators
Alternative – Dictionary unpacking
Alternative – ChainMap from the collections module
Assignment expressions
Type-hinting generics
Positional-only parameters
zoneinfo module
graphlib module
Not that new, but still shiny
breakpoint() function
Development mode
Module-level __getattr__() and __dir__() functions
Formatting strings with f-strings
Underscores in numeric literals
secrets module
What may come in the future?
Union types with the | operator
Structural pattern matching
Summary
Chapter 4: Python in Comparison with Other Languages
Technical requirements
Class model and object-oriented programming
Accessing super-classes
Multiple inheritance and Method Resolution Order
Class instance initialization
Attribute access patterns
Descriptors
Real-life example – lazily evaluated attributes
Properties
Dynamic polymorphism
Operator overloading
Dunder methods (language protocols)
Comparison to C++
Function and method overloading
Single-dispatch functions
Data classes
Functional programming
Lambda functions
The map(), filter(), and reduce() functions
Partial objects and partial functions
Generators
Generator expressions
Decorators
Enumerations
Summary
Chapter 5: Interfaces, Patterns, and Modularity
Technical requirements
Interfaces
A bit of history: zope.interface
Using function annotations and abstract base classes
Using collections.abc
Interfaces through type annotations
Inversion of control and dependency injection
Inversion of control in applications
Using dependency injection frameworks
Summary
Chapter 6: Concurrency
Technical requirements
What is concurrency?
Multithreading
What is multithreading?
How Python deals with threads
When should we use multithreading?
Application responsiveness
Multiuser applications
Work delegation and background processing
An example of a multithreaded application
Using one thread per item
Using a thread pool
Using two-way queues
Dealing with errors in threads
Throttling
Multiprocessing
The built-in multiprocessing module
Using process pools
Using multiprocessing.dummy as the multithreading interface
Asynchronous programming
Cooperative multitasking and asynchronous I/O
Python async and await keywords
A practical example of asynchronous programming
Integrating non-asynchronous code with async using futures
Executors and futures
Using executors in an event loop
Summary
Chapter 7: Event-Driven Programming
Technical requirements
What exactly is event-driven programming?
Event-driven != asynchronous
Event-driven programming in GUIs
Event-driven communication
Various styles of event-driven programming
Callback-based style
Subject-based style
Topic-based style
Event-driven architectures
Event and message queues
Summary
Chapter 8: Elements of Metaprogramming
Technical requirements
What is metaprogramming?
Using decorators to modify function behavior before use
One step deeper: class decorators
Intercepting the class instance creation process
Metaclasses
The general syntax
Metaclass usage
Metaclass pitfalls
Using the __init__subclass__() method as an alternative to metaclasses
Code generation
exec, eval, and compile
The abstract syntax tree
Import hooks
Notable examples of code generation in Python
Falcon's compiled router
Hy
Summary
Chapter 9: Bridging Python with C and C++
Technical requirements
C and C++ as the core of Python extensibility
Compiling and loading Python C extensions
The need to use extensions
Improving performance in critical code sections
Integrating existing code written in different languages
Integrating third-party dynamic libraries
Creating efficient custom datatypes
Writing extensions
Pure C extensions
A closer look at the Python/C API
Calling and binding conventions
Exception handling
Releasing GIL
Reference counting
Writing extensions with Cython
Cython as a source-to-source compiler
Cython as a language
Downsides of using extensions
Additional complexity
Harder debugging
Interfacing with dynamic libraries without extensions
The ctypes module
Loading libraries
Calling C functions using ctypes
Passing Python functions as C callbacks
CFFI
Summary
Chapter 10: Testing and Quality Automation
Technical requirements
The principles of test-driven development
Writing tests with pytest
Test parameterization
pytest's fixtures
Using fakes
Mocks and the unittest.mock module
Quality automation
Test coverage
Style fixers and code linters
Static type analysis
Mutation testing
Useful testing utilities
Faking realistic data values
Faking time values
Summary
Chapter 11: Packaging and Distributing Python Code
Technical requirements
Packaging and distributing libraries
The anatomy of a Python package
setup.py
setup.cfg
MANIFEST.in
Essential package metadata
Trove classifiers
Types of package distributions
sdist distributions
bdist and wheel distributions
Registering and publishing packages
Package versioning and dependency management
The SemVer standard for semantic versioning
CalVer for calendar versioning
Installing your own packages
Installing packages directly from sources
Installing packages in editable mode
Namespace packages
Package scripts and entry points
Packaging applications and services for the web
The Twelve-Factor App manifesto
Leveraging Docker
Handling environment variables
The role of environment variables in application frameworks
Creating standalone executables
When standalone executables are useful
Popular tools
PyInstaller
cx_Freeze
py2exe and py2app
Security of Python code in executable packages
Summary
Chapter 12: Observing Application Behavior and Performance
Technical requirements
Capturing errors and logs
Python logging essentials
Logging system components
Logging configuration
Good logging practices
Distributed logging
Capturing errors for later review
Instrumenting code with custom metrics
Using Prometheus
Distributed application tracing
Distributed tracing with Jaeger
Summary
Chapter 13: Code Optimization
Technical requirements
Common culprits for bad performance
Code complexity
Cyclomatic complexity
The big O notation
Excessive resource allocation and leaks
Excessive I/O and blocking operations
Code profiling
Profiling CPU usage
Macro-profiling
Micro-profiling
Profiling memory usage
Using the objgraph module
C code memory leaks
Reducing complexity by choosing appropriate data structures
Searching in a list
Using sets
Using the collections module
deque
defaultdict
namedtuple
Leveraging architectural trade-offs
Using heuristics and approximation algorithms
Using task queues and delayed processing
Using probabilistic data structures
Caching
Deterministic caching
Non-deterministic caching
Summary
Why subscribe?
Packt Page
Other Books You May Enjoy
Index
