Get your guided tour through the Python 3.9 interpreter: Unlock the inner workings of the Python language, compile the Python interpreter from source code, and participate in the development of CPython.
Are there certain parts of Python that just seem like magic?
This book explains the concepts, ideas, and technicalities of the Python interpreter in an approachable and hands-on fashion.
Once you see how Python works at the interpreter level, you can optimize your applications and fully leverage the power of Python.
By the End of the Book You’ll Be Able To:
• Read and navigate the CPython 3.9 interpreter source code. You’ll deeply comprehend and appreciate the inner workings of concepts like lists, dictionaries, and generators.
• Make changes to the Python syntax and compile your own version of CPython, from scratch. You’ll customize the Python core data types with new functionality and run CPython’s automated test suite.
• Master Python’s memory management capabilities and scale your Python code with parallelism and concurrency.
• Debug C and Python code like a true professional. Profile and benchmark the performance of your Python code and the runtime.
• Participate in the development of CPython and know how to contribute to future versions of the Python interpreter and standard library. How great would it feel to give back to the community as a “Python Core Developer?”
With this book you’ll cover the critical concepts behind the internals of CPython and how they work with visual explanations as you go along.
Each page in the book has been carefully laid out with beautiful typography, syntax highlighting for code examples.
About the Author:
Anthony Shaw is a CPython contributor and a PyCon speaker. He has a B.Sc. in Computer Science and over 8 years of experience working with Python. Anthony is also a Fellow of the Python Software Foundation and member of the Open-Source Apache Foundation.
Author(s): Anthony Shaw
Edition: 1
Publisher: Real Python
Year: 2021
Language: English
Commentary: Early Access version
Pages: 394
Tags: Debugging; Python; Concurrency; Parallel Programming; Memory Management; Benchmarking; Profiling; Best Practices; Object-Oriented Programming; C; Testing; Standard Library; Tracing; Abstract Syntax Trees
Contents
Foreword
Introduction
How to Use This Book
Bonus Material & Learning Resources
Getting the CPython Source Code
Setting up Your Development Environment
IDE or Editor?
Setting up Visual Studio
Setting up Visual Studio Code
Setting up JetBrains CLion
Setting up Vim
Conclusion
Compiling CPython
Compiling CPython on macOS
Compiling CPython on Linux
Installing a Custom Version
A Quick Primer on Make
CPython's Make Targets
Compiling CPython on Windows
Profile Guided Optimization
Conclusion
The Python Language and Grammar
Why CPython Is Written in C and Not Python
The Python Language Specification
Using the Parser Generator
The Parser Generator
Regenerating Grammar
A More Complex Example
Conclusion
Configuration and Input
Configuration State
Build Configuration
Building a Module From Input
Conclusion
Lexing and Parsing with Syntax Trees
Concrete Syntax Tree Generation
The CPython Parser-Tokenizer
Abstract Syntax Trees
Important Terms to Remember
Example: Adding an Almost Equal Comparison Operator
Conclusion
The Compiler
Related Source Files
Important Terms
Instantiating a Compiler
Future Flags and Compiler Flags
Symbol Tables
Core Compilation Process
Assembly
Creating a Code Object
Using Instaviz to Show a Code Object
Example: Implementing the ``Almost-Equal'' Operator
Conclusion
The Evaluation Loop
Stack Frames
Related Source Files
Important Terms
Constructing Thread State
Constructing Frame Objects
Frame Execution
The Value Stack
Example: Adding an Item to a List
Conclusion
Memory Management
Memory Allocation in C
Design of the Python Memory Management System
The CPython Memory Allocator
The Object and PyMem Memory Allocation Domains
The Raw Memory Allocation Domain
Custom Domain Allocators
Custom Memory Allocation Sanitizers
The PyArena Memory Arena
Reference Counting
Garbage Collection
Conclusion
Parallelism and Concurrency
Models of Parallelism and Concurrency
The Structure of a Process
Multi-Process Parallelism
Multithreading
Asynchronous Programming
Generators
Coroutines
Asynchronous Generators
Subinterpreters
Conclusion
Objects and Types
Examples in This Chapter
Builtin Types
Object and Variable Object Types
The type Type
Bool and Long Integer Type
Unicode String Type
Dictionary Type
Conclusion
The Standard Library
Python Modules
Python and C Modules
The Test Suite
Running the Test Suite on Windows
Running the Test Suite on Linux/macOS
Test Flags
Running Specific Tests
Testing Modules
Test Utilities
Conclusion
Debugging
Using the Crash Handler
Compiling Debug Support
Using Lldb for macOS
Using Gdb
Using Visual Studio Debugger
Using CLion Debugger
Conclusion
Benchmarking, Profiling, and Tracing
Using Timeit for Micro-Benchmarks
Using the Python Benchmark Suite for Runtime Benchmarks
Profiling Python Code with cProfile
Profiling C Code with Dtrace
Conclusion
Conclusion
Writing C Extensions for CPython
Using This Knowledge to Improve Your Python Applications
Using This Knowledge to Contribute to the CPython Project
Keep Learning
Appendix 1 : Introduction to C for Python Programmers
C Preprocessor
Basic C Syntax
Conclusion
