Math for Programmers

Math for Programmers
brief contents
contents
preface
How this book was designed
Mathematical ideas we cover
acknowledgments
about this book
Who should read this book?
How this book is organized
About the code
liveBook discussion forum
about the author
about the cover illustration
1 Learning math with code
1.1 Solving lucrative problems with math and software
1.1.1 Predicting financial market movements
1.1.2 Finding a good deal
1.1.3 Building 3D graphics and animations
1.1.4 Modeling the physical world
1.2 How not to learn math
1.2.1 Jane wants to learn some math
1.2.2 Slogging through math textbooks
1.3 Using your well-trained left brain
1.3.1 Using a formal language
1.3.2 Build your own calculator
1.3.3 Building abstractions with functions
Summary
Part 1 Vectors and graphics
2 Drawing with 2D vectors
2.1 Picturing 2D vectors
2.1.1 Representing 2D vectors
2.1.2 2D drawing in Python
2.1.3 Exercises
2.2 Plane vector arithmetic
2.2.1 Vector components and lengths
2.2.2 Multiplying vectors by numbers
2.2.3 Subtraction, displacement, and distance
2.2.4 Exercises
2.3 Angles and trigonometry in the plane
2.3.1 From angles to components
2.3.2 Radians and trigonometry in Python
2.3.3 From components back to angles
2.3.4 Exercises
2.4 Transforming collections of vectors
2.4.1 Combining vector transformations
2.4.2 Exercises
2.5 Drawing with Matplotlib
Summary
3 Ascending to the 3D world
3.1 Picturing vectors in 3D space
3.1.1 Representing 3D vectors with coordinates
3.1.2 3D drawing with Python
3.1.3 Exercises
3.2 Vector arithmetic in 3D
3.2.1 Adding 3D vectors
3.2.2 Scalar multiplication in 3D
3.2.3 Subtracting 3D vectors
3.2.4 Computing lengths and distances
3.2.5 Computing angles and directions
3.2.6 Exercises
3.3 The dot product: Measuring vector alignment
3.3.1 Picturing the dot product
3.3.2 Computing the dot product
3.3.3 Dot products by example
3.3.4 Measuring angles with the dot product
3.3.5 Exercises
3.4 The cross product: Measuring oriented area
3.4.1 Orienting ourselves in 3D
3.4.2 Finding the direction of the cross product
3.4.3 Finding the length of the cross product
3.4.4 Computing the cross product of 3D vectors
3.4.5 Exercises
3.5 Rendering a 3D object in 2D
3.5.1 Defining a 3D object with vectors
3.5.2 Projecting to 2D
3.5.3 Orienting faces and shading
3.5.4 Exercises
Summary
4 Transforming vectors and graphics
4.1 Transforming 3D objects
4.1.1 Drawing a transformed object
4.1.2 Composing vector transformations
4.1.3 Rotating an object about an axis
4.1.4 Inventing your own geometric transformations
4.1.5 Exercises
4.2 Linear transformations
4.2.1 Preserving vector arithmetic
4.2.2 Picturing linear transformations
4.2.3 Why linear transformations?
4.2.4 Computing linear transformations
4.2.5 Exercises
Summary
5 Computing transformations with matrices
5.1 Representing linear transformations with matrices
5.1.1 Writing vectors and linear transformations as matrices
5.1.2 Multiplying a matrix with a vector
5.1.3 Composing linear transformations by matrix multiplication
5.1.4 Implementing matrix multiplication
5.1.5 3D animation with matrix transformations
5.1.6 Exercises
5.2 Interpreting matrices of different shapes
5.2.1 Column vectors as matrices
5.2.2 What pairs of matrices can be multiplied?
5.2.3 Viewing square and non-square matrices as vector functions
5.2.4 Projection as a linear map from 3D to 2D
5.2.5 Composing linear maps
5.2.6 Exercises
5.3 Translating vectors with matrices
5.3.1 Making plane translations linear
5.3.2 Finding a 3D matrix for a 2D translation
5.3.3 Combining translation with other linear transformations
5.3.4 Translating 3D objects in a 4D world
5.3.5 Exercises
Summary
6 Generalizing to higher dimensions
6.1 Generalizing our definition of vectors
6.1.1 Creating a class for 2D coordinate vectors
6.1.2 Improving the Vec2 class
6.1.3 Repeating the process with 3D vectors
6.1.4 Building a vector base class
6.1.5 Defining vector spaces
6.1.6 Unit testing vector space classes
6.1.7 Exercises
6.2 Exploring different vector spaces
6.2.1 Enumerating all coordinate vector spaces
6.2.2 Identifying vector spaces in the wild
6.2.3 Treating functions as vectors
6.2.4 Treating matrices as vectors
6.2.5 Manipulating images with vector operations
6.2.6 Exercises
6.3 Looking for smaller vector spaces
6.3.1 Identifying subspaces
6.3.2 Starting with a single vector
6.3.3 Spanning a bigger space
6.3.4 Defining the word dimension
6.3.5 Finding subspaces of the vector space of functions
6.3.6 Subspaces of images
6.3.7 Exercises
Summary
7 Solving systems of linear equations
7.1 Designing an arcade game
7.1.1 Modeling the game
7.1.2 Rendering the game
7.1.3 Shooting the laser
7.1.4 Exercises
7.2 Finding intersection points of lines
7.2.1 Choosing the right formula for a line
7.2.2 Finding the standard form equation for a line
7.2.3 Linear equations in matrix notation
7.2.4 Solving linear equations with NumPy
7.2.5 Deciding whether the laser hits an asteroid
7.2.6 Identifying unsolvable systems
7.2.7 Exercises
7.3 Generalizing linear equations to higher dimensions
7.3.1 Representing planes in 3D
7.3.2 Solving linear equations in 3D
7.3.3 Studying hyperplanes algebraically
7.3.4 Counting dimensions, equations, and solutions
7.3.5 Exercises
7.4 Changing basis by solving linear equations
7.4.1 Solving a 3D example
7.4.2 Exercises
Summary
Part 2 Calculus and physical simulation
8 Understanding rates of change
8.1 Calculating average flow rate from volume
8.1.1 Implementing an average_flow_rate function
8.1.2 Picturing the average flow rate with a secant line
8.1.3 Negative rates of change
8.1.4 Exercises
8.2 Plotting the average flow rate over time
8.2.1 Finding the average flow rate in different time intervals
8.2.2 Plotting the interval flow rates
8.2.3 Exercises
8.3 Approximating instantaneous flow rates
8.3.1 Finding the slope of small secant lines
8.3.2 Building the instantaneous flow rate function
8.3.3 Currying and plotting the instantaneous flow rate function
8.3.4 Exercises
8.4 Approximating the change in volume
8.4.1 Finding the change in volume for a short time interval
8.4.2 Breaking up time into smaller intervals
8.4.3 Picturing the volume change on the flow rate graph
8.4.4 Exercises
8.5 Plotting the volume over time
8.5.1 Finding the volume over time
8.5.2 Picturing Riemann sums for the volume function
8.5.3 Improving the approximation
8.5.4 Definite and indefinite integrals
Summary
9 Simulating moving objects
9.1 Simulating a constant velocity motion
9.1.1 Adding velocities to the asteroids
9.1.2 Updating the game engine to move the asteroids
9.1.3 Keeping the asteroids on the screen
9.1.4 Exercises
9.2 Simulating acceleration
9.2.1 Accelerating the spaceship
9.3 Digging deeper into Euler’s method
9.3.1 Carrying out Euler’s method by hand
9.3.2 Implementing the algorithm in Python
9.4 Running Euler’s method with smaller time steps
9.4.1 Exercises
Summary
10 Working with symbolic expressions
10.1 Finding an exact derivative with a computer algebra system
10.1.1 Doing symbolic algebra in Python
10.2 Modeling algebraic expressions
10.2.1 Breaking an expression into pieces
10.2.2 Building an expression tree
10.2.3 Translating the expression tree to Python
10.2.4 Exercises
10.3 Putting a symbolic expression to work
10.3.1 Finding all the variables in an expression
10.3.2 Evaluating an expression
10.3.3 Expanding an expression
10.3.4 Exercises
10.4 Finding the derivative of a function
10.4.1 Derivatives of powers
10.4.2 Derivatives of transformed functions
10.4.3 Derivatives of some special functions
10.4.4 Derivatives of products and compositions
10.4.5 Exercises
10.5 Taking derivatives automatically
10.5.1 Implementing a derivative method for expressions
10.5.2 Implementing the product rule and chain rule
10.5.3 Implementing the power rule
10.5.4 Exercises
10.6 Integrating functions symbolically
10.6.1 Integrals as antiderivatives
10.6.2 Introducing the SymPy library
10.6.3 Exercises
Summary
11 Simulating force fields
11.1 Modeling gravity with a vector field
11.1.1 Modeling gravity with a potential energy function
11.2 Modeling gravitational fields
11.2.1 Defining a vector field
11.2.2 Defining a simple force field
11.3 Adding gravity to the asteroid game
11.3.1 Making game objects feel gravity
11.3.2 Exercises
11.4 Introducing potential energy
11.4.1 Defining a potential energy scalar field
11.4.2 Plotting a scalar field as a heatmap
11.4.3 Plotting a scalar field as a contour map
11.5 Connecting energy and forces with the gradient
11.5.1 Measuring steepness with cross sections
11.5.2 Calculating partial derivatives
11.5.3 Finding the steepness of a graph with the gradient
11.5.4 Calculating force fields from potential energy with the gradient
11.5.5 Exercises
Summary
12 Optimizing a physical system
12.1 Testing a projectile simulation
12.1.1 Building a simulation with Euler’s method
12.1.2 Measuring properties of the trajectory
12.1.3 Exploring different launch angles
12.1.4 Exercises
12.2 Calculating the optimal range
12.2.1 Finding the projectile range as a function of the launch angle
12.2.2 Solving for the maximum range
12.2.3 Identifying maxima and minima
12.2.4 Exercises
12.3 Enhancing our simulation
12.3.1 Adding another dimension
12.3.2 Modeling terrain around the cannon
12.3.3 Solving for the range of the projectile in 3D
12.3.4 Exercises
12.4 Optimizing range using gradient ascent
12.4.1 Plotting range versus launch parameters
12.4.2 The gradient of the range function
12.4.3 Finding the uphill direction with the gradient
12.4.4 Implementing gradient ascent
12.4.5 Exercises
Summary
13 Analyzing sound waves with a Fourier series
13.1 Combining sound waves and decomposing them
13.2 Playing sound waves in Python
13.2.1 Producing our first sound
13.2.2 Playing a musical note
13.2.3 Exercises
13.3 Turning a sinusoidal wave into a sound
13.3.1 Making audio from sinusoidal functions
13.3.2 Changing the frequency of a sinusoid
13.3.3 Sampling and playing the sound wave
13.3.4 Exercises
13.4 Combining sound waves to make new ones
13.4.1 Adding sampled sound waves to build a chord
13.4.2 Picturing the sum of two sound waves
13.4.3 Building a linear combination of sinusoids
13.4.4 Building a familiar function with sinusoids
13.4.5 Exercises
13.5 Decomposing a sound wave into its Fourier series
13.5.1 Finding vector components with an inner product
13.5.2 Defining an inner product for periodic functions
13.5.3 Writing a function to find Fourier coefficients
13.5.4 Finding the Fourier coefficients for the square wave
13.5.5 Fourier coefficients for other waveforms
13.5.6 Exercises
Summary
Part 3 Machine learning applications
14 Fitting functions to data
14.1 Measuring the quality of fit for a function
14.1.1 Measuring distance from a function
14.1.2 Summing the squares of the errors
14.1.3 Calculating cost for car price functions
14.1.4 Exercises
14.2 Exploring spaces of functions
14.2.1 Picturing cost for lines through the origin
14.2.2 The space of all linear functions
14.2.3 Exercises
14.3 Finding the line of best fit using gradient descent
14.3.1 Rescaling the data
14.3.2 Finding and plotting the line of best fit
14.3.3 Exercises
14.4 Fitting a nonlinear function
14.4.1 Understanding the behavior of exponential functions
14.4.2 Finding the exponential function of best fit
14.4.3 Exercises
Summary
15 Classifying data with logistic regression
15.1 Testing a classification function on real data
15.1.1 Loading the car data
15.1.2 Testing the classification function
15.1.3 Exercises
15.2 Picturing a decision boundary
15.2.1 Picturing the space of cars
15.2.2 Drawing a better decision boundary
15.2.3 Implementing the classification function
15.2.4 Exercises
15.3 Framing classification as a regression problem
15.3.1 Scaling the raw car data
15.3.2 Measuring the “BMWness” of a car
15.3.3 Introducing the sigmoid function
15.3.4 Composing the sigmoid function with other functions
15.3.5 Exercises
15.4 Exploring possible logistic functions
15.4.1 Parameterizing logistic functions
15.4.2 Measuring the quality of fit for a logistic function
15.4.3 Testing different logistic functions
15.4.4 Exercises
15.5 Finding the best logistic function
15.5.1 Gradient descent in three dimensions
15.5.2 Using gradient descent to find the best fit
15.5.3 Testing and understanding the best logistic classifier
15.5.4 Exercises
Summary
16 Training neural networks
16.1 Classifying data with neural networks
16.2 Classifying images of handwritten digits
16.2.1 Building the 64-dimensional image vectors
16.2.2 Building a random digit classifier
16.2.3 Measuring performance of the digit classifier
16.2.4 Exercises
16.3 Designing a neural network
16.3.1 Organizing neurons and connections
16.3.2 Data flow through a neural network
16.3.3 Calculating activations
16.3.4 Calculating activations in matrix notation
16.3.5 Exercises
16.4 Building a neural network in Python
16.4.1 Implementing an MLP class in Python
16.4.2 Evaluating the MLP
16.4.3 Testing the classification performance of an MLP
16.4.4 Exercises
16.5 Training a neural network using gradient descent
16.5.1 Framing training as a minimization problem
16.5.2 Calculating gradients with backpropagation
16.5.3 Automatic training with scikit-learn
16.5.4 Exercises
16.6 Calculating gradients with backpropagation
16.6.1 Finding the cost in terms of the last layer weights
16.6.2 Calculating the partial derivatives for the last layer weights using the chain rule
16.6.3 Exercises
Summary
appendix A Getting set up with Python
A.1 Checking for an existing Python installation
A.2 Downloading and installing Anaconda
A.3 Using Python in interactive mode
A.3.1 Creating and running a Python script file
A.3.2 Using Jupyter notebooks
appendix B Python tips and tricks
B.1 Python numbers and math
B.1.1 The math module
B.1.2 Random numbers
B.2 Collections of data in Python
B.2.1 Lists
B.2.2 Other iterables
B.2.3 Generators
B.2.4 Tuples
B.2.5 Sets
B.2.6 NumPy arrays
B.2.7 Dictionaries
B.2.8 Useful collection functions
B.3 Working with functions
B.3.1 Giving functions more inputs
B.3.2 Keyword arguments
B.3.3 Functions as data
B.3.4 Lambdas: Anonymous functions
B.3.5 Applying functions to NumPy arrays
B.4 Plotting data with Matplotlib
B.4.1 Making a scatter plot
B.4.2 Making a line chart
B.4.3 More plot customizations
B.5 Object-oriented programming in Python
B.5.1 Defining classes
B.5.2 Defining methods
B.5.3 Special methods
B.5.4 Operator overloading
B.5.5 Class methods
B.5.6 Inheritance and abstract classes
appendix C Loading and rendering 3D Models with OpenGL and PyGame
C.1 Recreating the octahedron from chapter 3
C.2 Changing our perspective
C.3 Loading and rendering the Utah teapot
C.4 Exercises
index
Symbols
Numerics
A
B
C
D
E
F
G
H
I
J
L
M
N
O
P
Q
R
S
T
U
V
W
Z
Math for Programmers - back