Over the past few years, a growing community of engineers and researchers have quietly rewritten the rules for how computers interact with the physical world. The result, a technology known as “edge artificial intelligence,” promises to upend a century of computer history and touch the lives of every human being.
With a tiny software update, edge AI technology can grant cheap, energy-efficient processors—already inside everything from dishwashers to thermostats—the ability to perceive and understand the world. We can empower everyday objects with their own intelligence, no longer dependent on data-hungry centralized servers. And next-generation tools put this magic in reach of everyone, from high school students to conservation researchers.
Author(s): Situnayake, Daniel & Plunkett, Jenny
Publisher: O’Reilly Media, Inc.
Year: 2023
Language: English
Commentary: Converted from EPUB
City: Sebastopol, CA
Foreword
Preface
About This Book
What to Expect
What You Need to Know Already
Responsible, Ethical, and Effective AI
Further Resources
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
1. A Brief Introduction to Edge AI
Defining Key Terms
Embedded
The Edge (and the Internet of Things)
Artificial Intelligence
Machine Learning
Edge AI
Embedded Machine Learning and Tiny Machine Learning
Digital Signal Processing
Why Do We Need Edge AI?
To Understand the Benefits of Edge AI, Just BLERP
Edge AI for Good
Key Differences Between Edge AI and Regular AI
Summary
2. Edge AI in the Real World
Common Use Cases for Edge AI
Greenfield and Brownfield Projects
Real-World Products
Types of Applications
Keeping Track of Objects
Understanding and Controlling Systems
Understanding People and Living Things
Transforming Signals
Building Applications Responsibly
Responsible Design and AI Ethics
Black Boxes and Bias
Technology That Harms, Not Helps
Summary
3. The Hardware of Edge AI
Sensors, Signals, and Sources of Data
Types of Sensors and Signals
Acoustic and Vibration
Visual and Scene
Motion and Position
Force and Tactile
Optical, Electromagnetic, and Radiation
Environmental, Biological, and Chemical
Other Signals
Processors for Edge AI
Edge AI Hardware Architecture
Microcontrollers and Digital Signal Processors
System-on-Chip
Deep Learning Accelerators
FPGAs and ASICs
Edge Servers
Multi-Device Architectures
Devices and Workloads
Summary
4. Algorithms for Edge AI
Feature Engineering
Working with Data Streams
Digital Signal Processing Algorithms
Combining Features and Sensors
Artificial Intelligence Algorithms
Algorithm Types by Functionality
Algorithm Types by Implementation
Optimization for Edge Devices
On-Device Training
Summary
5. Tools and Expertise
Building a Team for AI at the Edge
Domain Expertise
Diversity
Stakeholders
Roles and Responsibilities
Hiring for Edge AI
Learning Edge AI Skills
Tools of the Trade
Software Engineering
Working with Data
Algorithm Development
Running Algorithms On-Device
Embedded Software Engineering and Electronics
End-to-End Platforms for Edge AI
Summary
6. Understanding and Framing Problems
The Edge AI Workflow
Responsible AI in the Edge AI Workflow
Do I Need Edge AI?
Describing a Problem
Do I Need to Deploy to the Edge?
Do I Need Machine Learning?
Practical Exercise
Determining Feasibility
Moral Feasibility
Business Feasibility
Dataset Feasibility
Technological Feasibility
Making a Final Decision
Planning an Edge AI Project
Summary
7. How to Build a Dataset
What Does a Dataset Look Like?
The Ideal Dataset
Datasets and Domain Expertise
Data, Ethics, and Responsible AI
Minimizing Unknowns
Ensuring Domain Expertise
Data-Centric Machine Learning
Estimating Data Requirements
A Practical Workflow for Estimating Data Requirements
Getting Your Hands on Data
The Unique Challenges of Capturing Data at the Edge
Storing and Retrieving Data
Getting Data into Stores
Collecting Metadata
Ensuring Data Quality
Ensuring Representative Datasets
Reviewing Data by Sampling
Label Noise
Common Data Errors
Drift and Shift
The Uneven Distribution of Errors
Preparing Data
Labeling
Formatting
Data Cleaning
Feature Engineering
Splitting Your Data
Data Augmentation
Data Pipelines
Building a Dataset over Time
Summary
8. Designing Edge AI Applications
Product and Experience Design
Design Principles
Scoping a Solution
Setting Design Goals
Architectural Design
Hardware, Software, and Services
Basic Application Architectures
Complex Application Architectures and Design Patterns
Working with Design Patterns
Accounting for Choices in Design
Design Deliverables
Summary
9. Developing Edge AI Applications
An Iterative Workflow for Edge AI Development
Exploration
Goal Setting
Bootstrapping
Test and Iterate
Deployment
Support
Summary
10. Evaluating, Deploying, and Supporting Edge AI Applications
Evaluating Edge AI Systems
Ways to Evaluate a System
Useful Metrics
Techniques for Evaluation
Evaluation and Responsible AI
Deploying Edge AI Applications
Predeployment Tasks
Mid-Deployment Tasks
Postdeployment Tasks
Supporting Edge AI Applications
Postdeployment Monitoring
Improving a Live Application
Ethics and Long-Term Support
What Comes Next
11. Use Case: Wildlife Monitoring
Problem Exploration
Solution Exploration
Goal Setting
Solution Design
What Solutions Already Exist?
Solution Design Approaches
Design Considerations
Environmental Impact
Bootstrapping
Define Your Machine Learning Classes
Dataset Gathering
Edge Impulse
Choose Your Hardware and Sensors
Data Collection
iNaturalist
Dataset Limitations
Dataset Licensing and Legal Obligations
Cleaning Your Dataset
Uploading Data to Edge Impulse
DSP and Machine Learning Workflow
Digital Signal Processing Block
Machine Learning Block
Testing the Model
Live Classification
Model Testing
Test Your Model Locally
Deployment
Create Library
Mobile Phone and Computer
Prebuilt Binary Flashing
Impulse Runner
GitHub Source Code
Iterate and Feedback Loops
AI for Good
Related Works
Datasets
Research
12. Use Case: Food Quality Assurance
Problem Exploration
Solution Exploration
Goal Setting
Solution Design
What Solutions Already Exist?
Solution Design Approaches
Design Considerations
Environmental and Social Impact
Bootstrapping
Define Your Machine Learning Classes
Dataset Gathering
Edge Impulse
Choose Your Hardware and Sensors
Data Collection
Data Ingestion Firmware
Uploading Data to Edge Impulse
Cleaning Your Dataset
Dataset Licensing and Legal Obligations
DSP and Machine Learning Workflow
Digital Signal Processing Block
Machine Learning Block
Testing the Model
Live Classification
Model Testing
Deployment
Prebuilt Binary Flashing
GitHub Source Code
Iterate and Feedback Loops
Related Works
Research
News and Other Articles
13. Use Case: Consumer Products
Problem Exploration
Goal Setting
Solution Design
What Solutions Already Exist?
Solution Design Approaches
Design Considerations
Environmental and Social Impact
Bootstrapping
Define Your Machine Learning Classes
Dataset Gathering
Edge Impulse
Choose Your Hardware and Sensors
Data Collection
Data Ingestion Firmware
Cleaning Your Dataset
Dataset Licensing and Legal Obligations
DSP and Machine Learning Workflow
Digital Signal Processing Block
Machine Learning Blocks
Testing the Model
Live Classification
Model Testing
Deployment
Prebuilt Binary Flashing
GitHub Source Code
Iterate and Feedback Loops
Related Works
Research
News and Other Articles
Index
About the Authors
