Introduction to the Tools of Scientific Computing

Preface
Preface to the Second Edition
Acknowledgments
Preface to the First Edition
Acknowledgments
Contents
Chapter 1 Introduction
Background
Pitfalls and Limitations
Programming Languages
The C Language
Program Packages Matlab, Maple, Mathematica
The Golden Middle Course: Python
Part I Background
Part II Core Languages
Part III Commercial Computing Environments
Part IV Distributed Computing
Part V Specialized Programming Frameworks
Part I Background
Chapter 2 Mathematical Foundations of Programming
2.1 A Simple Machine Model
Machine Language and Higher-Level Programming Languages
Church’s Thesis
The Turing Machine
Limits of Computability
Repercussions in Mathematics
Extended Number Ranges
2.2 Digital Computers
Number Representation
Natural Numbers
Integer Numbers
Real Numbers, Floating Point Numbers
Higher-Level Programming Languages
Part II Core Languages
Chapter 3 Python, the Fundamentals
3.1 Python Interpreter
A First Example
Interactive Mode
3.2 Elementary Data Types
Numbers and Arithmetic
Integer Numbers
Real Numbers, Floating Point Numbers
Arithmetical Operations
Error Propagation in float Computations
The Types complex and bool
Complex Numbers
Boolean Operators, Comparisons
3.3 Variables and Value Assignments
Basic Rule
3.4 Control Structures
Loops
While Loops
For Loops
Interlude: the range Function
The break and continue Statements
Conditional Statements
Floating Point Precision Revisited: Machine Epsilon
3.5 Collection Types: Lists, Tuples, Dictionaries and Sets
Lists
list is a Reference Type
Sieve of Eratosthenes
Sublists
Concatenation
List Comprehensions
Tuples
Dictionaries
Sets
3.6 Functions
Anonymous Functions, Lambda Expressions
Functions as Arguments
Functions as Return Values
Recursion
The Quicksort Algorithm
3.7 String Interpolation and Formatting
3.8 Writing and Reading Files
Writing Strings
Interlude: Access Paths
Reading
Numbers
Binary Files and pickle
3.9 Object-Oriented Programming and Classes
Polynomials
Inheritance
Exercises
Machine Epsilon
Polynomial Class
Linear Algebra
Numerical Integration with the Trapezoidal Rule
The Taylor Series
The Newton Method
Fermat’s Last Theorem
Chapter 4 Python in Scientific Computation
4.1 NumPy
NumPy Arrays
Vectors and Matrices
Vectors
Matrices
Access to Matrix Components
Flatten and Reshape
Standard Matrices
Advanced Construction Methods
The Methods diag and kron, the Poisson Matrix
4.2 Conjugate Gradient Method
Quadratic Form
The Method
4.3 SciPy
4.4 Linear Algebra
Matrix Algorithms
Linear Equations
LU Decomposition
Cholesky Decomposition
Application: Method of Least Squares
QR Decomposition
QR Decomposition in Rectangular Matrices
The Gaussian Normal Equation
Eigenvectors and Eigenvalues
Sparse Matrices
Sparse Poisson Matrix
4.5 Graphics with Matplotlib
4.6 Nonlinear Equations, Optimization
Single Equations
Equation Systems
Minimization
4.7 Numerical Integration, Ordinary Differential Equations
Integration
Ordinary Differential Equations ODEs
Initial Value Problems, solve_ivp
The Solver solve_ivp
Lotka–Volterra Equations
Pendulum
Boundary Value Problems BVPs, solve_ivp
4.8 Partial Differential Equations
Discretization
Solution Program
Sparse Poisson Matrix
Nonhomogeneous Boundary Conditions
The SciPy Program
Verification
4.9 Round off: Random Numbers
Monte Carlo Method for the Computation of ?
Exercises
Linear Equations
Nonlinear Equations
Newton’s Method in Rn
Integration
Optimization
Ordinary Differential Equations, Initial Value Problems
Ordinary Differential Equations, Boundary Value Problems
Partial Differential Equations
Chapter 5 Python in Computer Algebra
5.1 Symbolic Calculation, Numbers
Number Types
Symbolic Expressions
Convention
Value Assignments
Advanced Transformations
Functions
5.2 Equation Systems
5.3 Linear Algebra
Matrices and Vectors
Vector Space Bases
Eigenvectors and Eigenvalues
Matrices with ? ≥ 5
5.4 Calculus
Limits
Differential Calculus
Integration
Series Expansion
5.5 Ordinary Differential Equations
5.6 Galerkin Method
The Model Equation
The Variational Problem
The Method
The SymPy Program
Exact Solution
Exercises
Linear Equations
Eigenvalues
Nonlinear Equations
Differential Equations
Function Spaces
Chapter 6 The C Language
Integrated Development Environment
Command Line Compiler
6.1 Basics
Numbers and Arithmetic Operations
Number Representations
Arithmetic Operators
Output
Variables and Assignments
6.2 Control Structures: Branches, Loops
Conditional Execution
Loops
6.3 Functions
Functions as Arguments
6.4 Arrays
Arrays as Function Arguments
Matrices
6.5 Pointers
Arrays and Pointers
Pointers as Arrays
6.6 Structures
Named Structures
Sparse Matrices
6.7 Files, Input and Output
6.8 Conclusion
Chapter 7 The C++ Language
7.1 Transfer from C
7.2 Basics
The Output Operator <<, Input/Output to Files
Writing to Files
Reading from Files
Namespaces
7.3 Lambda Expressions
7.4 Data Type vector
Sparse Vectors
Matrices
7.5 Reference Operator
7.6 Classes
A Class Matrix
Declaration Order
7.7 Header Files
Compiling and Executing the Program
7.8 Subclassing
The Subclass Parabola
7.9 Summary and Outlook
Exercises
Numerical Integration, Simpson’s Rule
Polynomial Interpolation
Ordinary Differential Equations
Classes
Polynomials
Quaternions
Chapter 8 Julia
8.1 Basics
Arithmetical Expressions
Variables and Assignments
Rational and Complex Numbers
Mathematical Functions and Constants
Boolean Operators, Comparisons
8.2 Control Structures: Branching, Loops
Conditional Statements
While Loop
For Loop
Interlude: Ranges
The break and continue Statements
8.3 Functions
Recursive Functions
Functions as Arguments and Return Values
Multiple Dispatch
Interlude: The REPL and Just in Time Compilation
8.4 Collection Types
Arrays and Vectors
Vectors and Mutating Functions
The filter Function
Comprehension
Tuples
Dictionaries
Sets
8.5 Composite Types
Structures
Interlude: String Interpolation
Mutable Structures
Structures vs. Classes
8.6 Linear Algebra
Linear Equations
Conjugate Gradient
Special Matrices
Submatrices
Reshape
The Standard Library Module Linear Algebra
The Poisson Matrix
Sparse Matrices
8.7 Ordinary Differential Equations
Equation Systems
Second Order ODEs
Boundary Value Problems BVPs
8.8 Partial Differential Equations
The Solution Program
8.9 Working with Files
Julia Set Fractals
Exercises
Monte Carlo Method
Continued Fractions
Differential Equations
Classes vs. Structs
Spline Interpolation
Part III Commercial Computing Environments
Chapter 9 Matlab
9.1 Basics
Numbers and Arithmetic
Variables
Boolean Operators, Comparisons
9.2 Vectors and Matrices
Vectors
The Colon Operator and linspace
Dynamic Vector Extension
Comparison Operators and Logical Arrays
Matrices
The Colon Operator for Matrices
Standard Matrices
Special Matrices
Matrix Operations
Division
Elementwise Operations
9.3 Control Structures: Branching, Loops
Conditional Statements
Loops
Loops vs. Vectors
9.4 Functions
Scalar or Elementary Functions
Vector Functions
Matrix Functions
9.5 M-Files
Script Files
Function Files
Subfunctions
Functions as Arguments
Anonymous Functions
Save and Load Files
9.6 Linear Algebra
Linear Equation Systems
Matrix Decompositions
LU Decomposition
Cholesky Decomposition
QR Decomposition
9.7 Ordinary Differential Equations
The Matlab Function ode45
Initial Value Problems
Second Order ODEs
Boundary Value Problems
The Bratu Equation
9.8 Partial Differential Equations
The Matlab PDE Toolbox
Exercises
Matrix Computations
ODE, Initial Value Problem
ODE, Boundary Value Problem
Partial Differential Equations PDEs
Advanced Topic: The Mandelbrot Set in Complex Numbers
Chapter 10 Maple
10.1 Basics
10.2 Functions
User Defined Functions
The Arrow Operator
The unapply Command
The proc Definition
Visualization
Equation Systems
10.3 Linear Algebra
Function Values as Matrix Entries
Vector Space Bases
The LinearAlgebra Package
Eigenvectors and Eigenvalues
? × ? Matrices with ? ≥ 5
10.4 Calculus
Derivation
Integration
Integral Norm
10.5 Interpolation with Spline Functions
Polygonal Chains
10.6 Differential Equations
Solution by Direct Integration
The Euler Method as Maple Procedure
10.7 Galerkin Method
Variational Problem
The Maple Program
Exact Solution
10.8 Finite Element Method
The Maple Program
Exercises
Ordinary Differential Equations
Polynomial Approximation and Interpolation
Interpolation with Legendre Polynomials
Spline Interpolation
Chapter 11 Mathematica
11.1 Basics
Variables and Expressions
Control Structures
Compound Expressions
Loops
Conditional Statements
11.2 Functions
Visualization
11.3 Nonlinear Equations, Optimization
11.4 Linear Algebra
Polynomial Vector Space
Sparse Matrices
11.5 Calculus
Derivation
Integration
11.6 Interpolation and Piecewise Functions
11.7 Differential Equations
11.8 Galerkin Method
11.9 Finite Element Method
Exercises
Polynomial Interpolation and Approximation
Splines
Ordinary Differential Equations
Partial Differential Equations
Part IV Distributed Computing
Chapter 12 A Python Approach to Message Passing
12.1 Introduction to the Message Passing Interface
Installation
Testing
The MPI Module
12.2 Communicating Processes
Deadlocks
Notation Convention
Summation
Point-to-Point vs. Collective Communication, the reduce and bcast Commands
The reduce Command
The bcast Command
Load Balancing
12.3 Integral Approximation
The Midpoint Rule
The Trapezoidal Rule
The MPI Program
12.4 Vector Dot Product
Dot Product Evenly Divided
Interlude: Message Passing with pickle and NumPy array
Dot Product with Load Balance
12.5 Laplace Equation
Distributed Iteration
The MPI Program
12.6 Conjugate Gradient Method
The Program
Exercises
Matrix Operations
The alltoall Operator
The Sieve of Eratosthenes
Chapter 13 Parallel Computing in C/C++
Installation
The mpi.h Library
13.1 Integral Approximation of ?
13.2 Scatter and Gather
13.3 Conjugate Gradient
Compilation and Testing
13.4 Shared Memory Programming
Hello World
Compilation and Program Execution
Message Passing with Shared Memory
The barrier Directive
13.5 Parallelizing Loops
The Bane of Shared Memory Computation: Race Conditions
The reduction Clause
The atomic Directive
The The reduction Clause
13.6 Integral Approximation of ?
13.7 Parallelized Function Modules
Recursive Functions
13.8 Hybrid Message Passing and Shared Memory Programming
Hybrid Conjugate Gradient
Exercises
Message passing in C/C++
Shared Memory
Hybrid Message Passing and Shared Memory Programming
Chapter 14 Distributed Processing in Julia
14.1 Point-to-Point Communication
14.2 Distributed Loops, Reductions
14.3 Monte Carlo Method
14.4 Fibonacci Sequence
14.5 Shared Arrays
14.6 Distributed Arrays
Vector Dot Product
Matrix-Vector Product
Exercises
Part V Specialized Programming Frameworks
Chapter 15 Automated Solution of PDEs with FEniCS
Installation
15.1 Finite Element Method, One-Dimensional
A Simple Ordinary Differential Equation
Variational Form
Galerkin Method
Hat Functions
Discretization
Manual Solution
Nonhomogeneous Boundary Conditions
15.2 FEniCS Implementation, One-Dimensional
15.3 Poisson Equation
Method of Manufactured Solutions
Transformation into a Variational Problem
Discretization and FEniCS Implementation
Regular Triangulation
Discrete Function Space
Source Function
Boundary Condition
The Discrete Variational Equation
15.4 Time-Dependent Poisson Equation
A Discrete-Time Approach
Transformation into a Variational Problem
Manufactured Solution
FEniCS Implementation
15.5 Nonlinear Equations
Variational Form
Picard Iteration
The FEniCS Program
Direct Solution
15.6 Neumann Boundary Conditions
Mixed Dirichlet-Neumann Boundary Conditions
Manufactured Solution
Variational Form
FEniCS Implementation
Pure Neumann Boundary Conditions
Manufactured Solution
Variational Form
FEniCS Implementation
15.7 Stokes Equation
Mathematical Preliminaries
Variational Problem
FEniCS Implementation
The Program
Visualizing
15.8 Adaptive Mesh Refinement
FEniCS Implementation
15.9 User Defined Meshes
The Package mshr
Mesh Generation
Application Example
Manual Mesh Refinement
Mesh Libraries
15.10 Final Note: Parallel Processing in FEniCS
Exercises
First Order ODEs
Second Order ODEs
Partial Differential Equations PDEs
The Third Mode of solve
A Neumann Challenge
Chapter 16 Scientific Machine Learning with PyTorch
Overview
Installation
16.1 Linear Regression
Gradient Descent
16.2 Gradient Descent in PyTorch
Automatic Differentiation
Optimizer
The Built-In Linear Module
Module
Summary and Outlook
16.3 Backpropagation
Optimization
Nonlinear Equations
Linear Algebra
Effect of the Loss Criterion
The Quadratic Form
16.4 Neural Networks
Universal Approximation Theorem, Threshold Version
Sigmoid Neurons
Universal Approximation Theorem, Sigmoid Version
Neural Networks in PyTorch
16.5 Function Approximation
The Runge Function
The Rosenbrock Function
16.6 Derivation
Derivative Functions
Higher-Order Derivatives
A Basic Differential Operator
Partial Derivatives
16.7 Integration
16.8 Ordinary Differential Equations ODEs
Coupled Equation Systems
16.9 Second Order ODEs
Second Order Initial Value Problems
16.10 Partial Differential Equations PDEs
Nonhomogeneous Boundary Conditions
Exercises
Curve Fitting
Activation Functions
Derivatio
Integration
Differential Equations
Ordinary Differential Equations
Partial Differential Equations
Supplement I. Logical Operators
Boolean Operators
Threshold Neuronal Networks
Linear Separability
Sigmoid Neuronal Networks
Supplement II. Character Recognition
The Database
Digit Recognition in PyTorch
Data Preparation
The Neural Network
Training
Evaluation
References
Index
Subjects and Names
Python
C/C++
Julia
Matlab
Maple
Mathematica