Edge AI is transforming the way computers interact with the real world, allowing IoT devices to make decisions using the 99% of sensor data that was previously discarded due to cost, bandwidth, or power limitations. With techniques like embedded machine learning, developers can capture human intuition and deploy it to any target--from ultra-low power microcontrollers to embedded Linux devices.
This practical guide gives engineering professionals, including product managers and technology leaders, an end-to-end framework for solving real-world industrial, commercial, and scientific problems with edge AI. You'll explore every stage of the process, from data collection to model optimization to tuning and testing, as you learn how to design and support edge AI and embedded ML products. Edge AI is destined to become a standard tool for systems engineers. This high-level road map helps you get started.
- Develop your expertise in AI and ML for edge devices
- Understand which projects are best solved with edge AI
- Explore key design patterns for edge AI apps
- Learn an iterative workflow for developing AI systems
- Build a team with the skills to solve real-world problems
- Follow a responsible AI process to create effective products
Author(s): Daniel Situnayake, Jenny Plunkett
Edition: 1
Publisher: O'Reilly Media
Year: 2023
Language: English
Commentary: eBook has crop/trim marks | Published: January 2023 | Revision History: 2023-01-10: First Release
Pages: xxiv, 488
City: Sebastopol, CA
Tags: Artificial Intelligence; Machine Learning; Embedded Computer Systems; Internet of Things; Neural Networks; Natural Language Processing
Copyright
Table of Contents
Foreword
Preface
About This Book
What to Expect
What You Need to Know Already
Responsible, Ethical, and Effective AI
Further Resources
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
Chapter 1. A Brief Introduction to Edge AI
Defining Key Terms
Embedded
The Edge (and the Internet of Things)
Artificial Intelligence
Machine Learning
Edge AI
Embedded Machine Learning and Tiny Machine Learning
Digital Signal Processing
Why Do We Need Edge AI?
To Understand the Benefits of Edge AI, Just BLERP
Edge AI for Good
Key Differences Between Edge AI and Regular AI
Summary
Chapter 2. Edge AI in the Real World
Common Use Cases for Edge AI
Greenfield and Brownfield Projects
Real-World Products
Types of Applications
Keeping Track of Objects
Understanding and Controlling Systems
Understanding People and Living Things
Transforming Signals
Building Applications Responsibly
Responsible Design and AI Ethics
Black Boxes and Bias
Technology That Harms, Not Helps
Summary
Chapter 3. The Hardware of Edge AI
Sensors, Signals, and Sources of Data
Types of Sensors and Signals
Acoustic and Vibration
Visual and Scene
Motion and Position
Force and Tactile
Optical, Electromagnetic, and Radiation
Environmental, Biological, and Chemical
Other Signals
Processors for Edge AI
Edge AI Hardware Architecture
Microcontrollers and Digital Signal Processors
System-on-Chip
Deep Learning Accelerators
FPGAs and ASICs
Edge Servers
Multi-Device Architectures
Devices and Workloads
Summary
Chapter 4. Algorithms for Edge AI
Feature Engineering
Working with Data Streams
Digital Signal Processing Algorithms
Combining Features and Sensors
Artificial Intelligence Algorithms
Algorithm Types by Functionality
Algorithm Types by Implementation
Optimization for Edge Devices
On-Device Training
Summary
Chapter 5. Tools and Expertise
Building a Team for AI at the Edge
Domain Expertise
Diversity
Stakeholders
Roles and Responsibilities
Hiring for Edge AI
Learning Edge AI Skills
Tools of the Trade
Software Engineering
Working with Data
Algorithm Development
Running Algorithms On-Device
Embedded Software Engineering and Electronics
End-to-End Platforms for Edge AI
Summary
Chapter 6. Understanding and Framing Problems
The Edge AI Workflow
Responsible AI in the Edge AI Workflow
Do I Need Edge AI?
Describing a Problem
Do I Need to Deploy to the Edge?
Do I Need Machine Learning?
Practical Exercise
Determining Feasibility
Moral Feasibility
Business Feasibility
Dataset Feasibility
Technological Feasibility
Making a Final Decision
Planning an Edge AI Project
Summary
Chapter 7. How to Build a Dataset
What Does a Dataset Look Like?
The Ideal Dataset
Datasets and Domain Expertise
Data, Ethics, and Responsible AI
Minimizing Unknowns
Ensuring Domain Expertise
Data-Centric Machine Learning
Estimating Data Requirements
A Practical Workflow for Estimating Data Requirements
Getting Your Hands on Data
The Unique Challenges of Capturing Data at the Edge
Storing and Retrieving Data
Getting Data into Stores
Collecting Metadata
Ensuring Data Quality
Ensuring Representative Datasets
Reviewing Data by Sampling
Label Noise
Common Data Errors
Drift and Shift
The Uneven Distribution of Errors
Preparing Data
Labeling
Formatting
Data Cleaning
Feature Engineering
Splitting Your Data
Data Augmentation
Data Pipelines
Building a Dataset over Time
Summary
Chapter 8. Designing Edge AI Applications
Product and Experience Design
Design Principles
Scoping a Solution
Setting Design Goals
Architectural Design
Hardware, Software, and Services
Basic Application Architectures
Complex Application Architectures and Design Patterns
Working with Design Patterns
Accounting for Choices in Design
Design Deliverables
Summary
Chapter 9. Developing Edge AI Applications
An Iterative Workflow for Edge AI Development
Exploration
Goal Setting
Bootstrapping
Test and Iterate
Deployment
Support
Summary
Chapter 10. Evaluating, Deploying, and Supporting Edge AI Applications
Evaluating Edge AI Systems
Ways to Evaluate a System
Useful Metrics
Techniques for Evaluation
Evaluation and Responsible AI
Deploying Edge AI Applications
Predeployment Tasks
Mid-Deployment Tasks
Postdeployment Tasks
Supporting Edge AI Applications
Postdeployment Monitoring
Improving a Live Application
Ethics and Long-Term Support
What Comes Next
Chapter 11. Use Case: Wildlife Monitoring
Problem Exploration
Solution Exploration
Goal Setting
Solution Design
What Solutions Already Exist?
Solution Design Approaches
Design Considerations
Environmental Impact
Bootstrapping
Define Your Machine Learning Classes
Dataset Gathering
Edge Impulse
Choose Your Hardware and Sensors
Data Collection
iNaturalist
Dataset Limitations
Dataset Licensing and Legal Obligations
Cleaning Your Dataset
Uploading Data to Edge Impulse
DSP and Machine Learning Workflow
Digital Signal Processing Block
Machine Learning Block
Testing the Model
Live Classification
Model Testing
Test Your Model Locally
Deployment
Create Library
Mobile Phone and Computer
Prebuilt Binary Flashing
Impulse Runner
GitHub Source Code
Iterate and Feedback Loops
AI for Good
Related Works
Datasets
Research
Chapter 12. Use Case: Food Quality Assurance
Problem Exploration
Solution Exploration
Goal Setting
Solution Design
What Solutions Already Exist?
Solution Design Approaches
Design Considerations
Environmental and Social Impact
Bootstrapping
Define Your Machine Learning Classes
Dataset Gathering
Edge Impulse
Choose Your Hardware and Sensors
Data Collection
Data Ingestion Firmware
Uploading Data to Edge Impulse
Cleaning Your Dataset
Dataset Licensing and Legal Obligations
DSP and Machine Learning Workflow
Digital Signal Processing Block
Machine Learning Block
Testing the Model
Live Classification
Model Testing
Deployment
Prebuilt Binary Flashing
GitHub Source Code
Iterate and Feedback Loops
Related Works
Research
News and Other Articles
Chapter 13. Use Case: Consumer Products
Problem Exploration
Goal Setting
Solution Design
What Solutions Already Exist?
Solution Design Approaches
Design Considerations
Environmental and Social Impact
Bootstrapping
Define Your Machine Learning Classes
Dataset Gathering
Edge Impulse
Choose Your Hardware and Sensors
Data Collection
Data Ingestion Firmware
Cleaning Your Dataset
Dataset Licensing and Legal Obligations
DSP and Machine Learning Workflow
Digital Signal Processing Block
Machine Learning Blocks
Testing the Model
Live Classification
Model Testing
Deployment
Prebuilt Binary Flashing
GitHub Source Code
Iterate and Feedback Loops
Related Works
Research
News and Other Articles
Index
About the Authors
Colophon
