AI is nothing without somewhere to run it. Now that mobile devices have become the primary computing device for most people, it's essential that mobile developers add AI to their toolbox. This insightful book is your guide to creating and running models on popular mobile platforms such as iOS and Android.
Laurence Moroney, lead AI advocate at Google, offers an introduction to machine learning techniques and tools, then walks you through writing Android and iOS apps powered by common ML models like computer vision and text recognition, using tools such as ML Kit, TensorFlow Lite, and Core ML. If you're a mobile developer, this book will help you take advantage of the ML revolution today.
• Explore the options for implementing ML and AI on mobile devices
• Create ML models for iOS and Android
• Write ML Kit and TensorFlow Lite apps for iOS and Android, and Core ML/Create ML apps for iOS
• Choose the best techniques and tools for your use case, such as cloud-based versus on-device inference and high-level versus low-level APIs
• Learn privacy and ethics best practices for ML on devices
Author(s): Laurence Moroney
Edition: 1
Publisher: O'Reilly Media, Inc.
Year: 2021
Language: English
Commentary: Vector PDF
Pages: 300
City: Sebastopol, CA
Tags: Cloud Computing; Machine Learning; Deep Learning; Computer Vision; Ethics; iOS; Privacy; Docker; Android; AutoML; Text Processing; Kotlin; Mobile Applications; TensorFlow Lite; ML Kit
Cover
Copyright
Table of Contents
Preface
Who Should Read This Book?
Why I Wrote This Book
Navigating This Book
Technology You Need to Understand
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgements
Chapter 1. Introduction to AI and Machine Learning
What Is Artificial Intelligence?
What Is Machine Learning?
Moving from Traditional Programming to Machine Learning
How Can a Machine Learn?
Comparing Machine Learning with Traditional Programming
Building and Using Models on Mobile
Summary
Chapter 2. Introduction to Computer Vision
Using Neurons for Vision
Your First Classifier: Recognizing Clothing Items
The Data: Fashion MNIST
A Model Architecture to Parse Fashion MNIST
Coding the Fashion MNIST Model
Transfer Learning for Computer Vision
Summary
Chapter 3. Introduction to ML Kit
Building a Face Detection App on Android
Step 1: Create the App with Android Studio
Step 2: Add and Configure ML Kit
Step 3: Define the User Interface
Step 4: Add the Images as Assets
Step 5: Load the UI with a Default Picture
Step 6: Call the Face Detector
Step 7: Add the Bounding Rectangles
Building a Face Detector App for iOS
Step 1: Create the Project in Xcode
Step 2: Using CocoaPods and Podfiles
Step 3: Create the User Interface
Step 4: Add the Application Logic
Summary
Chapter 4. Computer Vision Apps with ML Kiton Android
Image Labeling and Classification
Step 1: Create the App and Configure ML Kit
Step 2: Create the User Interface
Step 3: Add the Images as Assets
Step 4: Load an Image to the ImageView
Step 5: Write the Button Handler Code
Next Steps
Object Detection
Step 1: Create the App and Import ML Kit
Step 2: Create the Activity Layout XML
Step 3: Load an Image into the ImageView
Step 4: Set Up the Object Detector Options
Step 5: Handling the Button Interaction
Step 6: Draw the Bounding Boxes
Step 7: Label the Objects
Detecting and Tracking Objects in Video
Exploring the Layout
The GraphicOverlay Class
Capturing the Camera
The ObjectAnalyzer Class
The ObjectGraphic Class
Putting It All Together
Summary
Chapter 5. Text Processing Apps with ML Kiton Android
Entity Extraction
Start Creating the App
Create the Layout for the Activity
Write the Entity Extraction Code
Putting It All Together
Handwriting and Other Recognition
Start the App
Creating a Drawing Surface
Parsing the Ink with ML Kit
Smart Reply to Conversations
Start the App
Mock a Conversation
Generating a Smart Reply
Summary
Chapter 6. Computer Vision Apps with ML Kit on iOS
Image Labeling and Classification
Step 1: Create the App in Xcode
Step 2: Create the Podfile
Step 3: Set Up the Storyboard
Step 4: Edit the View Controller Code to Use ML Kit
Object Detection in iOS with ML Kit
Step 1: Get Started
Step 2: Create Your UI on the Storyboard
Step 3: Create a Subview for Annotation
Step 4: Perform the Object Detection
Step 5: Handle the Callback
Combining Object Detection with Image Classification
Object Detection and Tracking in Video
Summary
Chapter 7. Text Processing Apps with ML Kit on iOS
Entity Extraction
Step 1: Create the App and Add the ML Kit Pods
Step 2: Create the Storyboard with Actions and Outlets
Step 3: Allow Your View Controller to be Used for Text Entry
Step 4: Initialize the Model
Step 5: Extract Entities from Text
Handwriting Recognition
Step 1: Create the App and Add the ML Kit Pods
Step 2: Create the Storyboard, Actions, and Outlets
Step 3: Strokes, Points, and Ink
Step 4: Capture User Input
Step 5: Initialize the Model
Step 6: Do the Ink Recognition
Smart Reply to Conversations
Step 1: Create an App and Integrate ML Kit
Step 2: Create Storyboard, Outlets, and Actions
Step 3: Create a Simulated Conversation
Step 4: Get Smart Reply
Summary
Chapter 8. Going Deeper: Understanding TensorFlow Lite
What Is TensorFlow Lite?
Getting Started with TensorFlow Lite
Save the Model
Convert the Model
Testing the Model with a Standalone Interpreter
Create an Android App to Host TFLite
Import the TFLite File
Write Kotlin Code to Interface with the Model
Going Beyond the Basics
Create an iOS App to Host TFLite
Step 1: Create a Basic iOS App
Step 2: Add TensorFlow Lite to Your Project
Step 3: Create the User Interface
Step 4: Add and Initialize the Model Inference Class
Step 5: Perform the Inference
Step 6: Add the Model to Your App
Step 7: Add the UI Logic
Moving Beyond “Hello World”: Processing Images
Exploring Model Optimization
Quantization
Using Representative Data
Summary
Chapter 9. Creating Custom Models
Creating a Model with TensorFlow Lite Model Maker
Creating a Model with Cloud AutoML
Using AutoML Vision Edge
Creating a Model with TensorFlow and Transfer Learning
Creating Language Models
Create a Language Model with Model Maker
Summary
Chapter 10. Using Custom Models in Android
Bridging Models to Android
Building an Image Classification App from a Model Maker Output
Using a Model Maker Output with ML Kit
Using Language Models
Creating an Android App for Language Classification
Summary
Chapter 11. Using Custom Models in iOS
Bridging Models to iOS
A Custom Model Image Classifier
Step 1: Create the App and Add the TensorFlow Lite Pod
Step 2: Create the UI and Image Assets
Step 3: Load and Navigate Through the Image Assets
Step 4: Load the Model
Step 5: Convert an Image to an Input Tensor
Step 6: Get Inference for the Tensor
Use a Custom Model in ML Kit
Building an App for Natural Language Processing in Swift
Step 1: Load the Vocab
Step 2: Convert the Sentence to a Sequence
Step 3: Extend Array to Handle Unsafe Data
Step 4: Copy the Array to a Data Buffer
Step 5: Run Inference on the Data and Process the Results
Summary
Chapter 12. Productizing Your App Using Firebase
Why Use Firebase Custom Model Hosting?
Create Multiple Model Versions
Using Firebase Model Hosting
Step 1: Create a Firebase Project
Step 2: Use Custom Model Hosting
Step 3: Create a Basic Android App
Step 4: Add Firebase to the App
Step 5: Get the Model from Firebase Model Hosting
Step 6: Use Remote Configuration
Step 7: Read Remote Configuration in Your App
Next Steps
Summary
Chapter 13. Create ML and Core ML for Simple iOS Apps
A Core ML Image Classifier Built Using Create ML
Making a Core ML App That Uses a Create ML Model
Add the MLModel File
Run the Inference
Using Create ML to Build a Text Classifier
Use the Model in an App
Summary
Chapter 14. Accessing Cloud-Based Models from Mobile Apps
Installing TensorFlow Serving
Installing Using Docker
Installing Directly on Linux
Building and Serving a Model
Accessing a Server Model from Android
Accessing a Server Model from iOS
Summary
Chapter 15. Ethics, Fairness, and Privacy for Mobile Apps
Ethics, Fairness, and Privacy with Responsible AI
Responsibly Defining Your Problem
Avoiding Bias in Your Data
Building and Training Your Model
Evaluating Your Model
Google’s AI Principles
Summary
Index
About the Author
Colophon
