This book develops a conceptual understanding of Artificial Intelligence (AI), Deep Learning and Machine Learning in the truest sense of the word. It is an earnest endeavor to unravel what is happening at the algorithmic level, to grasp how applications are being built and to show the long adventurous road in the future.
An Intuitive Exploration of Artificial Intelligence offers insightful details on how AI works and solves problems in computer vision, natural language understanding, speech understanding, reinforcement learning and synthesis of new content. From the classic problem of recognizing cats and dogs, to building autonomous vehicles, to translating text into another language, to automatically converting speech into text and back to speech, to generating neural art, to playing games, and the author’s own experience in building solutions in industry, this book is about explaining how exactly the myriad applications of AI flow out of its immense potential.
The book is intended to serve as a textbook for graduate and senior-level undergraduate courses in AI. Moreover, since the book provides a strong geometrical intuition about advanced mathematical foundations of AI, practitioners and researchers will equally benefit from the book.
Author(s): Simant Dube
Edition: 1st ed. 2021
Publisher: Springer
Year: 2021
Language: English
Pages: 374
Preface
Contents
Acronyms
Author's Note
Part I Foundations
1 AI Sculpture
1.1 Manifolds in High Dimensions
1.2 Sculpting Process
1.3 Notational Convention
1.4 Regression and Classification
1.4.1 Linear Regression and Logistic Regression
1.4.2 Regression Loss and Cross-Entropy Loss
1.4.3 Sculpting with Shades
1.5 Discriminative and Generative AI
1.6 Success of Discriminative Methods
1.7 Feature Engineering in Classical ML
1.8 Supervised and Unsupervised AI
1.9 Beyond Manifolds
1.10 Chapter Summary
2 Make Me Learn
2.1 Learnable Parameters
2.1.1 The Power of a Single Neuron
2.1.2 Neurons Working Together
2.2 Backpropagation of Gradients
2.2.1 Partial Derivatives
2.2.2 Forward and Backward Passes
2.3 Stochastic Gradient Descent
2.3.1 Handling Difficult Landscapes
2.3.2 Stabilization of Training
2.4 Chapter Summary
3 Images and Sequences
3.1 Convolutional Neural Networks
3.1.1 The Biology of the Visual Cortex
3.1.2 Pattern Matching
3.1.3 3-D Convolution
3.2 Recurrent Neural Networks
3.2.1 Neurons with States
3.2.2 The Power of Recurrence
3.2.3 Going Both Ways
3.2.4 Attention
3.3 Self-Attention
3.4 LSTM
3.5 Beyond Images and Sequences
3.6 Chapter Summary
4 Why AI Works
4.1 Convex Polytopes
4.2 Piecewise Linear Function
4.2.1 Subdivision of the Input Space
4.2.2 Piecewise Non-linear Function
4.2.3 Carving Out the Feature Spaces
4.3 Expressive Power of AI
4.4 Convolutional Neural Network
4.5 Recurrent Neural Network
4.6 Architectural Variations
4.7 Attention and Carving
4.8 Optimization Landscape
4.8.1 Graph-Induced Polynomial
4.8.2 Gradient of the Loss Function
4.8.3 Visualization
4.8.4 Critical Points
4.9 The Mathematics of Loss Landscapes
4.9.1 Random Polynomial Perspective
4.9.2 Random Matrix Perspective
4.9.3 Spin Glass Perspective
4.9.4 Computational Complexity Perspective
4.9.5 SGD and Critical Points
4.9.6 Confluence of Perspectives
4.10 Distributed Representation and Intrinsic Dimension
4.11 Chapter Summary
5 Novice to Maestro
5.1 How AI Learns to Sculpt
5.1.1 Training Data
5.1.2 Evaluation Metrics
5.1.3 Hyperparameter Search
5.1.4 Regularization
5.1.5 Bias and Variance
5.1.6 A Fairy Tale in the Land of ML
5.2 Learning Curves
5.3 From the Lab to the Dirty Field
5.4 System Design
5.5 Flavors of Learning
5.6 Ingenuity and Big Data in the Success of AI
5.7 Chapter Summary
6 Unleashing the Power of Generation
6.1 Creating Universes
6.2 To Recognize It, Learn to Draw It
6.3 General Definition
6.4 Generative Parameters
6.5 Generative AI Models
6.5.1 Restricted Boltzmann Machines
6.5.2 Autoencoders
6.5.3 Variational Autoencoder
6.5.4 Pixel Recursive Models
6.5.5 Generative Adversarial Networks
6.5.6 Wasserstein Generative Adversarial Networks
6.6 The Carving Process in Generative AI
6.7 Representation of Individual Signals
6.8 Chapter Summary
7 The Road Most Rewarded
7.1 Reinforcement Learning
7.2 Learning an Optimal Policy
7.3 Deep Q-Learning
7.4 Policy Gradient
7.4.1 Intuition
7.4.2 Mathematical Analysis
7.5 Let's Play and Explore
7.6 Chapter Summary
8 The Classical World
8.1 Maximum Likelihood Estimation
8.2 Uncertainty in Estimation
8.3 Linear Models
8.3.1 Linear Regression
8.3.2 The Geometry of Linear Regression
8.3.3 Regularization
8.3.4 Logistic Regression
8.4 Classical ML
8.4.1 k-Nearest Neighbors
8.4.2 Naive Bayes Classifier
8.4.3 FDA, LDA, and QDA
8.4.4 Support Vector Machines
8.4.5 Neural Networks
8.4.6 Decision Trees
8.4.7 Gaussian Process Regression
8.4.8 Unsupervised Methods
8.5 XGBoost
8.5.1 Relevance Ranking
8.6 Chapter Summary
Part II Applications
9 To See Is to Believe
9.1 Image Classification
9.1.1 Motivating Examples
9.1.2 LeNet
9.1.3 Stacked Autoencoders
9.1.4 AlexNet
9.1.5 VGG
9.1.6 ResNet
9.1.7 Inception V3
9.1.8 Showroom of Models
9.1.9 Your Own Network
9.2 Object Detection as Classification
9.2.1 Sliding Window Method
9.2.2 Region Proposal Method
9.3 Regression on Images
9.3.1 Motivating Examples
9.3.2 Object Detection as Regression
9.3.2.1 Regression Output
9.3.2.2 Grid-Based Approach
9.4 Attention in Computer Vision
9.5 Semantic Segmentation
9.6 Image Similarity
9.7 Video Analysis
9.8 3-D Data
9.9 Self-Driving Vehicles
9.10 Present and Future
9.11 Protein Folding
9.12 Chapter Summary
10 Read, Read, Read
10.1 Natural Language Understanding
10.2 Embedding Words in a Semantic Space
10.3 Sequence to Sequence
10.3.1 Encoder-Decoder Architecture
10.3.2 Neural Machine Translation
10.4 Attention Mechanism
10.5 Self-Attention
10.6 Creativity in NLU Solutions
10.7 AI and Human Culture
10.8 Recommender Systems
10.9 Reward-Based Formulations
10.10 Chapter Summary
11 Lend Me Your Ear
11.1 Classical Speech Recognition
11.2 Spectrogram to Transcription
11.2.1 Alignment-Free Temporal Connections
11.2.2 End-to-End Solution
11.2.3 Don't Listen to Others
11.3 Speech Synthesis
11.4 Handwriting Recognition
11.5 Chapter Summary
12 Create Your Shire and Rivendell
12.1 From Neurons to Art
12.1.1 DeepDream
12.1.2 Style Transfer
12.2 Image Translation
12.3 DeepFake
12.4 Creative Applications
12.5 Chapter Summary
13 Math to Code to Petaflops
13.1 Software Frameworks
13.1.1 The Twentieth Century
13.1.2 The Twenty-First Century
13.1.3 AI Frameworks
13.2 Let's Crunch Numbers
13.2.1 Computing Hardware
13.2.2 GPU Machines
13.2.3 Cloud GPU Instances
13.2.4 Training Script
13.2.5 Deployment
13.3 Speeding Up Training
13.3.1 Data Parallelism
13.3.2 Delayed and Compressed SGD
13.4 Open Ecosystem and Efficient Hardware
13.5 Chapter Summary
14 AI and Business
14.1 Strategy
14.2 Organization
14.3 Execution
14.4 Evaluation
14.5 Startups
14.6 Chapter Summary
Part III The Road Ahead
15 Keep Marching On
15.1 Robust AI
15.1.1 Adversarial Examples
15.1.2 Learning from Human Vision
15.1.3 Fusion of Evidence
15.1.4 Interpretable AI
15.2 AI Extraordinaire
15.3 Chapter Summary
16 Benevolent AI for All
16.1 Benefits of AI
16.2 AI in Medicine
16.3 Dangers of AI
16.4 AI-Human Conflict
16.5 Choices Ahead
16.6 Chapter Summary
17 Am I Looking at Myself?
17.1 Is It Computable or Non-computable?
17.2 Is Consciousness Everywhere?
17.3 Who Is the Storyteller?
17.4 Chapter Summary
A Solutions
Answer of Exercise 1
Answer of Exercise 2
Answer of Exercise 3
Answer of Exercise 4
Answer of Exercise 5
Answer of Exercise 6
Answer of Exercise 7
Answer of Exercise 9
Answer of Exercise 8
Answer of Exercise 10
Answer of Exercise 11
Answer of Exercise 12
Answer of Exercise 13
Answer of Exercise 14
Answer of Exercise 15
Answer of Exercise 16
Answer of Exercise 17
Answer of Exercise 18
Answer of Exercise 19
Answer of Exercise 20
Answer of Exercise 21
Answer of Exercise 22
Answer of Exercise 23
Answer of Exercise 24
Answer of Exercise 25
Answer of Exercise 26
Answer of Exercise 27
Answer of Exercise 28
Answer of Exercise 29
Answer of Exercise 30
Answer of Exercise 31
Answer of Exercise 32
Answer of Exercise 33
Answer of Exercise 34
Answer of Exercise 35
B Lab Exercises and Projects
B.1 Exercises
B.2 Exploration
B.3 Debate and Discussion
Further Reading
Glossary
References
Index
