Become an ace Python programmer by learning best coding practices and advance-level concepts with Python 3.5 About This Book Based on the latest stable version of Python (version 3.5) Creating well manageable code that will run in various environments with different sets of dependencies Packed with advanced concepts and best practices to write efficient Python code Who This Book Is For The book would appeal to web developers and Python programmers who want to start using version 3.5 and write code efficiently. Basic knowledge of Python programming is expected. What You Will Learn Conventions and best practices that are widely adopted in the python community Package python code effectively for community and production use Easy and lightweight ways to automate code deployment on remote systems Improve your code's quality, reliability, and performance Write concurrent code in python Extend python with code written in different languages In Detail Python is a dynamic programming language, used in a wide range of domains by programmers who find it simple, yet powerful. Even if you find writing Python code easy, writing code that is efficient and easy to maintain and reuse is a challenge. The focus of the book is to familiarize you with common conventions, best practices, useful tools and standards used by python professionals on a daily basis when working with code. You will begin with knowing new features in Python 3.5 and quick tricks for improving productivity. Next, you will learn advanced and useful python syntax elements brought to this new version. Using advanced object-oriented concepts and mechanisms available in python, you will learn different approaches to implement metaprogramming. You will learn to choose good names, write packages, and create standalone executables easily. You will also be using some powerful tools such as buildout and vitualenv to release and deploy the code on remote servers for production use. Moving on, you will learn to effectively create Python extensions with C, C++, cython, and pyrex. The important factors while writing code such as code management tools, writing clear documentation, and test-driven development are also covered. You will now dive deeper to make your code efficient with general rules of optimization, strategies for finding bottlenecks, and selected tools for application optimization. By the end of the book, you will be an expert in writing efficient and maintainable code. Style and approach An easy-to-follow guide that covers industry followed best practices in Python programming"
Author(s): Michal Jaworski; Tarek Ziadé
Edition: 2
Publisher: Packt
Year: 2016
Cover
Copyright
Credits
About the Authors
About the Reviewer
www.PacktPub.com
Table of Contents
Preface
Chapter 1: Current Status of Python
Where are we now and where we are going?
Why and how does Python change?
Getting up to date with changes – PEP documents
Python 3 adoption at the time of writing this book
The main differences between Python 3 and Python 2
Why should I care?
The main syntax differences and common pitfalls
Syntax changes
Changes in the standard library
Changes in datatypes and collections
The popular tools and techniques used for maintaining cross-version compatibility
Not only CPython
Why should I care?
Stackless Python
Jython
IronPython
PyPy
Modern approaches to Python development
Application-level isolation of Python environments
Why isolation?
Popular solutions
virtualenv
venv
buildout
Which one to choose?
System-level environment isolation
Virtual development environments using Vagrant
Containerization versus virtualization
Popular productivity tools
Custom Python shells – IPython, bpython, ptpython, and so on
Setting up the PYTHONSTARTUP environment variable
IPython
bpython
ptpython
Interactive debuggers
Useful resources
Summary
Chapter 2: Syntax Best Practices – below the Class Level
Python's built-in types
Strings and bytes
Implementation details
String concatenation
Collections
Lists and tuples
Dictionaries
Sets
Beyond basic collections – the collections module
Advanced syntax
Iterators
The yield statement
Decorators
General syntax and possible implementations
Usage and useful examples
Context managers – the with statement
General syntax and possible implementations
Other syntax elements you may not know yet
The for … else … statement
Function annotations
The general syntax
The possible uses
Summary
Chapter 3: Syntax Best Practices – above the Class Level
Subclassing built-in types
Accessing methods from superclasses
Old-style classes and super in Python 2
Understanding Python's Method Resolution Order
super pitfalls
Mixing super and explicit class calls
Heterogeneous arguments
Best practices
Advanced attribute access patterns
Descriptors
Real-life example – lazily evaluated attributes
Properties
Slots
Metaprogramming
Decorators – a method of metaprogramming
Class decorators
Using the __new__() method to override instance creation process
Metaclasses
The general syntax
New Python 3 syntax for metaclasses
Metaclass usage
Metaclass pitfalls
Some tips on code generation
exec, eval, and compile
Abstract Syntax Tree
Projects using code generation patterns
Summary
Chapter 4: Choosing Good Names
PEP 8 and naming best practices
Why and when to follow PEP 8?
Beyond PEP 8 – team-specific style guidelines
Naming styles
Variables
Constants
Naming and usage
Public and private variables
Functions and methods
The private controversy
Special methods
Arguments
Properties
Classes
Modules and packages
The naming guide
Using the has or is prefix for Boolean elements
Using plurals for variables that are collections
Using explicit names for dictionaries
Avoiding generic names
Avoiding existing names
Best practices for arguments
Building arguments by iterative design
Trust the arguments and your tests
Using *args and **kwargs magic arguments carefully
Class names
Module and package names
Useful tools
Pylint
pep8 and flake8
Summary
Chapter 5: Writing a Package
Creating a package
The confusing state of Python packaging tools
The current landscape of Python packaging thanks to PyPA
Tool recommendations
Project configuration
setup.py
setup.cfg
MANIFEST.in
Most important metadata
Trove classifiers
Common patterns
The custom setup command
Working with packages during development
setup.py install
Uninstalling packages
setup.py develop or pip -e
Namespace packages
Why is it useful?
PEP 420 – implicit namespace packages
Namespace packages in previous Python versions
Uploading a package
PyPI – Python Package Index
Uploading to PyPI – or other package index
.pypirc
Source packages versus built packages
sdist
bdist and wheels
Standalone executables
When are standalone executables useful?
Popular tools
PyInstaller
cx_Freeze
py2exe and py2app
Security of Python code in executable packages
Making decompilation harder
Summary
Chapter 6: Deploying Code
The Twelve-Factor App
Deployment automation using Fabric
Your own package index or index mirror
PyPI mirroring
Deployment using a package
Common conventions and practices
The filesystem hierarchy
Isolation
Using process supervision tools
Application code should be run in user space
Using reverse HTTP proxies
Reloading processes gracefully
Code instrumentation and monitoring
Logging errors – sentry/raven
Monitoring system and application metrics
Dealing with application logs
Basic low-level log practices
Tools for log processing
Summary
Chapter 7: Python Extensions in
Other Languages
Different language means – C or C++
How do extensions in C or C++ work
Why you might want to use extensions
Improving performance in critical code sections
Integrating existing code written in different languages
Integrating third-party dynamic libraries
Creating custom datatypes
Writing extensions
Pure C extensions
A closer look at Python/C API
Calling and binding conventions
Exception handling
Releasing GIL
Reference counting
Cython
Cython as a source to source compiler
Cython as a language
Challenges
Additional complexity
Debugging
Interfacing with dynamic libraries without extensions
ctypes
Loading libraries
Calling C functions using ctypes
Passing Python functions as C callbacks
CFFI
Summary
Chapter 8: Managing Code
Version control systems
Centralized systems
Distributed systems
Distributed strategies
Centralized or distributed?
Use Git if you can
Git flow and GitHub flow
Continuous development processes
Continuous integration
Testing every commit
Merge testing through CI
Matrix testing
Continuous delivery
Continuous deployment
Popular tools for continuous integration
Jenkins
Buildbot
Travis CI
GitLab CI
Choosing the right tool and common pitfalls
Problem 1 – too complex build strategies
Problem 2 – too long building time
Problem 3 – external job definitions
Problem 4 – lack of isolation
Summary
Chapter 9: Documenting Your Project
The seven rules of technical writing
Write in two steps
Target the readership
Use a simple style
Limit the scope of information
Use realistic code examples
Use a light but sufficient approach
Use templates
A reStructuredText primer
Section structure
Lists
Inline markup
Literal block
Links
Building the documentation
Building the portfolio
Design
Usage
Operations
Making your own portfolio
Building the landscape
Producer's layout
Consumer's layout
Documentation building and continuous integration
Summary
Chapter 10: Test-Driven Development
I don't test
Test-driven development principles
Preventing software regression
Improving code quality
Providing the best developer documentation
Producing robust code faster
What kind of tests?
Acceptance tests
Unit tests
Functional tests
Integration tests
Load and performance testing
Code quality testing
Python standard test tools
unittest
doctest
I do test
unittest pitfalls
unittest alternatives
nose
py.test
Testing coverage
Fakes and mocks
Building a fake
Using mocks
Testing environment and dependency compatibility
Dependency matrix testing
Document-driven development
Writing a story
Summary
Chapter 11: Optimization – General Principles and Profiling Techniques
The three rules of optimization
Make it work first
Work from the user's point of view
Keep the code readable and maintainable
Optimization strategy
Find another culprit
Scale the hardware
Writing a speed test
Finding bottlenecks
Profiling CPU usage
Macro-profiling
Micro-profiling
Measuring Pystones
Profiling memory usage
How Python deals with memory
Profiling memory
C code memory leaks
Profiling network usage
Summary
Chapter 12: Optimization – Some Powerful Techniques
Reducing the complexity
Cyclomatic complexity
The big O notation
Simplifying
Searching in a list
Using a set instead of a list
Using collections
deque
defaultdict
namedtuple
Using architectural trade-offs
Using heuristics and approximation algorithms
Using task queues and delayed processing
Using probabilistic data structures
Caching
Deterministic caching
Nondeterministic caching
Cache services
Memcached
Summary
Chapter 13: Concurrency
Why concurrency?
Multithreading
What is multithreading?
How Python deals with threads
When should threading be used?
Building responsive interfaces
Delegating work
Multiuser applications
An example of a threaded application
Multiprocessing
The built-in multiprocessing module
Using process pools
Using multiprocessing.dummy as a multithreading interface
Asynchronous programming
Cooperative multitasking and asynchronous I/O
Python async and await keywords
asyncio in older versions of Python
A practical example of asynchronous programming
Integrating nonasynchronous code with async using futures
Executors and futures
Using executors in an event loop
Summary
Chapter 14: Useful Design Patterns
Creational patterns
Singleton
Structural patterns
Adapter
Interfaces
Proxy
Facade
Behavioral patterns
Observer
Visitor
Template
Summary
Index
