Data Science on AWS: Implementing End-to-End, Continuous AI and Machine Learning Pipelines

Copyright
Table of Contents
Preface
Overview of the Chapters
Who Should Read This Book
Other Resources
Conventions Used in This Book
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
Chapter 1. Introduction to Data Science on AWS
Benefits of Cloud Computing
Agility
Cost Savings
Elasticity
Innovate Faster
Deploy Globally in Minutes
Smooth Transition from Prototype to Production
Data Science Pipelines and Workflows
Amazon SageMaker Pipelines
AWS Step Functions Data Science SDK
Kubeflow Pipelines
Managed Workflows for Apache Airflow on AWS
MLflow
TensorFlow Extended
Human-in-the-Loop Workflows
MLOps Best Practices
Operational Excellence
Security
Reliability
Performance Efficiency
Cost Optimization
Amazon AI Services and AutoML with Amazon SageMaker
Amazon AI Services
AutoML with SageMaker Autopilot
Data Ingestion, Exploration, and Preparation in AWS
Data Ingestion and Data Lakes with Amazon S3 and AWS Lake Formation
Data Analysis with Amazon Athena, Amazon Redshift, and Amazon QuickSight
Evaluate Data Quality with AWS Deequ and SageMaker Processing Jobs
Label Training Data with SageMaker Ground Truth
Data Transformation with AWS Glue DataBrew, SageMaker Data Wrangler, and SageMaker Processing Jobs
Model Training and Tuning with Amazon SageMaker
Train Models with SageMaker Training and Experiments
Built-in Algorithms
Bring Your Own Script (Script Mode)
Bring Your Own Container
Pre-Built Solutions and Pre-Trained Models with SageMaker JumpStart
Tune and Validate Models with SageMaker Hyper-Parameter Tuning
Model Deployment with Amazon SageMaker and AWS Lambda Functions
SageMaker Endpoints
SageMaker Batch Transform
Serverless Model Deployment with AWS Lambda
Streaming Analytics and Machine Learning on AWS
Amazon Kinesis Streaming
Amazon Managed Streaming for Apache Kafka
Streaming Predictions and Anomaly Detection
AWS Infrastructure and Custom-Built Hardware
SageMaker Compute Instance Types
GPUs and Amazon Custom-Built Compute Hardware
GPU-Optimized Networking and Custom-Built Hardware
Storage Options Optimized for Large-Scale Model Training
Reduce Cost with Tags, Budgets, and Alerts
Summary
Chapter 2. Data Science Use Cases
Innovation Across Every Industry
Personalized Product Recommendations
Recommend Products with Amazon Personalize
Generate Recommendations with Amazon SageMaker and TensorFlow
Generate Recommendations with Amazon SageMaker and Apache Spark
Detect Inappropriate Videos with Amazon Rekognition
Demand Forecasting
Predict Energy Consumption with Amazon Forecast
Predict Demand for Amazon EC2 Instances with Amazon Forecast
Identify Fake Accounts with Amazon Fraud Detector
Enable Privacy-Leak Detection with Amazon Macie
Conversational Devices and Voice Assistants
Speech Recognition with Amazon Lex
Text-to-Speech Conversion with Amazon Polly
Speech-to-Text Conversion with Amazon Transcribe
Text Analysis and Natural Language Processing
Translate Languages with Amazon Translate
Classify Customer-Support Messages with Amazon Comprehend
Extract Resume Details with Amazon Textract and Comprehend
Cognitive Search and Natural Language Understanding
Intelligent Customer Support Centers
Industrial AI Services and Predictive Maintenance
Home Automation with AWS IoT and Amazon SageMaker
Extract Medical Information from Healthcare Documents
Self-Optimizing and Intelligent Cloud Infrastructure
Predictive Auto Scaling for Amazon EC2
Anomaly Detection on Streams of Data
Cognitive and Predictive Business Intelligence
Ask Natural-Language Questions with Amazon QuickSight
Train and Invoke SageMaker Models with Amazon Redshift
Invoke Amazon Comprehend and SageMaker Models from Amazon Aurora SQL Database
Invoke SageMaker Model from Amazon Athena
Run Predictions on Graph Data Using Amazon Neptune
Educating the Next Generation of AI and ML Developers
Build Computer Vision Models with AWS DeepLens
Learn Reinforcement Learning with AWS DeepRacer
Understand GANs with AWS DeepComposer
Program Nature’s Operating System with Quantum Computing
Quantum Bits Versus Digital Bits
Quantum Supremacy and the Quantum Computing Eras
Cracking Cryptography
Molecular Simulations and Drug Discovery
Logistics and Financial Optimizations
Quantum Machine Learning and AI
Programming a Quantum Computer with Amazon Braket
AWS Center for Quantum Computing
Increase Performance and Reduce Cost
Automatic Code Reviews with CodeGuru Reviewer
Improve Application Performance with CodeGuru Profiler
Improve Application Availability with DevOps Guru
Summary
Chapter 3. Automated Machine Learning
Automated Machine Learning with SageMaker Autopilot
Track Experiments with SageMaker Autopilot
Train and Deploy a Text Classifier with SageMaker Autopilot
Train and Deploy with SageMaker Autopilot UI
Train and Deploy a Model with the SageMaker Autopilot Python SDK
Predict with Amazon Athena and SageMaker Autopilot
Train and Predict with Amazon Redshift ML and SageMaker Autopilot
Automated Machine Learning with Amazon Comprehend
Predict with Amazon Comprehend’s Built-in Model
Train and Deploy a Custom Model with the Amazon Comprehend UI
Train and Deploy a Custom Model with the Amazon Comprehend Python SDK
Summary
Chapter 4. Ingest Data into the Cloud
Data Lakes
Import Data into the S3 Data Lake
Describe the Dataset
Query the Amazon S3 Data Lake with Amazon Athena
Access Athena from the AWS Console
Register S3 Data as an Athena Table
Update Athena Tables as New Data Arrives with AWS Glue Crawler
Create a Parquet-Based Table in Athena
Continuously Ingest New Data with AWS Glue Crawler
Build a Lake House with Amazon Redshift Spectrum
Export Amazon Redshift Data to S3 Data Lake as Parquet
Share Data Between Amazon Redshift Clusters
Choose Between Amazon Athena and Amazon Redshift
Reduce Cost and Increase Performance
S3 Intelligent-Tiering
Parquet Partitions and Compression
Amazon Redshift Table Design and Compression
Use Bloom Filters to Improve Query Performance
Materialized Views in Amazon Redshift Spectrum
Summary
Chapter 5. Explore the Dataset
Tools for Exploring Data in AWS
Visualize Our Data Lake with SageMaker Studio
Prepare SageMaker Studio to Visualize Our Dataset
Run a Sample Athena Query in SageMaker Studio
Dive Deep into the Dataset with Athena and SageMaker
Query Our Data Warehouse
Run a Sample Amazon Redshift Query from SageMaker Studio
Dive Deep into the Dataset with Amazon Redshift and SageMaker
Create Dashboards with Amazon QuickSight
Detect Data-Quality Issues with Amazon SageMaker and Apache Spark
SageMaker Processing Jobs
Analyze Our Dataset with Deequ and Apache Spark
Detect Bias in Our Dataset
Generate and Visualize Bias Reports with SageMaker Data Wrangler
Detect Bias with a SageMaker Clarify Processing Job
Integrate Bias Detection into Custom Scripts with SageMaker Clarify Open Source
Mitigate Data Bias by Balancing the Data
Detect Different Types of Drift with SageMaker Clarify
Analyze Our Data with AWS Glue DataBrew
Reduce Cost and Increase Performance
Use a Shared S3 Bucket for Nonsensitive Athena Query Results
Approximate Counts with HyperLogLog
Dynamically Scale a Data Warehouse with AQUA for Amazon Redshift
Improve Dashboard Performance with QuickSight SPICE
Summary
Chapter 6. Prepare the Dataset for Model Training
Perform Feature Selection and Engineering
Select Training Features Based on Feature Importance
Balance the Dataset to Improve Model Accuracy
Split the Dataset into Train, Validation, and Test Sets
Transform Raw Text into BERT Embeddings
Convert Features and Labels to Optimized TensorFlow File Format
Scale Feature Engineering with SageMaker Processing Jobs
Transform with scikit-learn and TensorFlow
Transform with Apache Spark and TensorFlow
Share Features Through SageMaker Feature Store
Ingest Features into SageMaker Feature Store
Retrieve Features from SageMaker Feature Store
Ingest and Transform Data with SageMaker Data Wrangler
Track Artifact and Experiment Lineage with Amazon SageMaker
Understand Lineage-Tracking Concepts
Show Lineage of a Feature Engineering Job
Understand the SageMaker Experiments API
Ingest and Transform Data with AWS Glue DataBrew
Summary
Chapter 7. Train Your First Model
Understand the SageMaker Infrastructure
Introduction to SageMaker Containers
Increase Availability with Compute and Network Isolation
Deploy a Pre-Trained BERT Model with SageMaker JumpStart
Develop a SageMaker Model
Built-in Algorithms
Bring Your Own Script
Bring Your Own Container
A Brief History of Natural Language Processing
BERT Transformer Architecture
Training BERT from Scratch
Masked Language Model
Next Sentence Prediction
Fine Tune a Pre-Trained BERT Model
Create the Training Script
Setup the Train, Validation, and Test Dataset Splits
Set Up the Custom Classifier Model
Train and Validate the Model
Save the Model
Launch the Training Script from a SageMaker Notebook
Define the Metrics to Capture and Monitor
Configure the Hyper-Parameters for Our Algorithm
Select Instance Type and Instance Count
Putting It All Together in the Notebook
Download and Inspect Our Trained Model from S3
Show Experiment Lineage for Our SageMaker Training Job
Show Artifact Lineage for Our SageMaker Training Job
Evaluate Models
Run Some Ad Hoc Predictions from the Notebook
Analyze Our Classifier with a Confusion Matrix
Visualize Our Neural Network with TensorBoard
Monitor Metrics with SageMaker Studio
Monitor Metrics with CloudWatch Metrics
Debug and Profile Model Training with SageMaker Debugger
Detect and Resolve Issues with SageMaker Debugger Rules and Actions
Profile Training Jobs
Interpret and Explain Model Predictions
Detect Model Bias and Explain Predictions
Detect Bias with a SageMaker Clarify Processing Job
Feature Attribution and Importance with SageMaker Clarify and SHAP
More Training Options for BERT
Convert TensorFlow BERT Model to PyTorch
Train PyTorch BERT Models with SageMaker
Train Apache MXNet BERT Models with SageMaker
Train BERT Models with PyTorch and AWS Deep Java Library
Reduce Cost and Increase Performance
Use Small Notebook Instances
Test Model-Training Scripts Locally in the Notebook
Profile Training Jobs with SageMaker Debugger
Start with a Pre-Trained Model
Use 16-Bit Half Precision and bfloat16
Mixed 32-Bit Full and 16-Bit Half Precision
Quantization
Use Training-Optimized Hardware
Spot Instances and Checkpoints
Early Stopping Rule in SageMaker Debugger
Summary
Chapter 8. Train and Optimize Models at Scale
Automatically Find the Best Model Hyper-Parameters
Set Up the Hyper-Parameter Ranges
Run the Hyper-Parameter Tuning Job
Analyze the Best Hyper-Parameters from the Tuning Job
Show Experiment Lineage for Our SageMaker Tuning Job
Use Warm Start for Additional SageMaker Hyper-Parameter Tuning Jobs
Run HPT Job Using Warm Start
Analyze the Best Hyper-Parameters from the Warm-Start Tuning Job
Scale Out with SageMaker Distributed Training
Choose a Distributed-Communication Strategy
Choose a Parallelism Strategy
Choose a Distributed File System
Launch the Distributed Training Job
Reduce Cost and Increase Performance
Start with Reasonable Hyper-Parameter Ranges
Shard the Data with ShardedByS3Key
Stream Data on the Fly with Pipe Mode
Enable Enhanced Networking
Summary
Chapter 9. Deploy Models to Production
Choose Real-Time or Batch Predictions
Real-Time Predictions with SageMaker Endpoints
Deploy Model Using SageMaker Python SDK
Track Model Deployment in Our Experiment
Analyze the Experiment Lineage of a Deployed Model
Invoke Predictions Using the SageMaker Python SDK
Invoke Predictions Using HTTP POST
Create Inference Pipelines
Invoke SageMaker Models from SQL and Graph-Based Queries
Auto-Scale SageMaker Endpoints Using Amazon CloudWatch
Define a Scaling Policy with AWS-Provided Metrics
Define a Scaling Policy with a Custom Metric
Tuning Responsiveness Using a Cooldown Period
Auto-Scale Policies
Strategies to Deploy New and Updated Models
Split Traffic for Canary Rollouts
Shift Traffic for Blue/Green Deployments
Testing and Comparing New Models
Perform A/B Tests to Compare Model Variants
Reinforcement Learning with Multiarmed Bandit Testing
Monitor Model Performance and Detect Drift
Enable Data Capture
Understand Baselines and Drift
Monitor Data Quality of Deployed SageMaker Endpoints
Create a Baseline to Measure Data Quality
Schedule Data-Quality Monitoring Jobs
Inspect Data-Quality Results
Monitor Model Quality of Deployed SageMaker Endpoints
Create a Baseline to Measure Model Quality
Schedule Model-Quality Monitoring Jobs
Inspect Model-Quality Monitoring Results
Monitor Bias Drift of Deployed SageMaker Endpoints
Create a Baseline to Detect Bias
Schedule Bias-Drift Monitoring Jobs
Inspect Bias-Drift Monitoring Results
Monitor Feature Attribution Drift of Deployed SageMaker Endpoints
Create a Baseline to Monitor Feature Attribution
Schedule Feature Attribution Drift Monitoring Jobs
Inspect Feature Attribution Drift Monitoring Results
Perform Batch Predictions with SageMaker Batch Transform
Select an Instance Type
Set Up the Input Data
Tune the SageMaker Batch Transform Configuration
Prepare the SageMaker Batch Transform Job
Run the SageMaker Batch Transform Job
Review the Batch Predictions
AWS Lambda Functions and Amazon API Gateway
Optimize and Manage Models at the Edge
Deploy a PyTorch Model with TorchServe
TensorFlow-BERT Inference with AWS Deep Java Library
Reduce Cost and Increase Performance
Delete Unused Endpoints and Scale In Underutilized Clusters
Deploy Multiple Models in One Container
Attach a GPU-Based Elastic Inference Accelerator
Optimize a Trained Model with SageMaker Neo and TensorFlow Lite
Use Inference-Optimized Hardware
Summary
Chapter 10. Pipelines and MLOps
Machine Learning Operations
Software Pipelines
Machine Learning Pipelines
Components of Effective Machine Learning Pipelines
Steps of an Effective Machine Learning Pipeline
Pipeline Orchestration with SageMaker Pipelines
Create an Experiment to Track Our Pipeline Lineage
Define Our Pipeline Steps
Configure the Pipeline Parameters
Create the Pipeline
Start the Pipeline with the Python SDK
Start the Pipeline with the SageMaker Studio UI
Approve the Model for Staging and Production
Review the Pipeline Artifact Lineage
Review the Pipeline Experiment Lineage
Automation with SageMaker Pipelines
GitOps Trigger When Committing Code
S3 Trigger When New Data Arrives
Time-Based Schedule Trigger
Statistical Drift Trigger
More Pipeline Options
AWS Step Functions and the Data Science SDK
Kubeflow Pipelines
Apache Airflow
MLflow
TensorFlow Extended
Human-in-the-Loop Workflows
Improving Model Accuracy with Amazon A2I
Active-Learning Feedback Loops with SageMaker Ground Truth
Reduce Cost and Improve Performance
Cache Pipeline Steps
Use Less-Expensive Spot Instances
Summary
Chapter 11. Streaming Analytics and Machine Learning
Online Learning Versus Offline Learning
Streaming Applications
Windowed Queries on Streaming Data
Stagger Windows
Tumbling Windows
Sliding Windows
Streaming Analytics and Machine Learning on AWS
Classify Real-Time Product Reviews with Amazon Kinesis, AWS Lambda, and Amazon SageMaker
Implement Streaming Data Ingest Using Amazon Kinesis Data Firehose
Create Lambda Function to Invoke SageMaker Endpoint
Create the Kinesis Data Firehose Delivery Stream
Put Messages on the Stream
Summarize Real-Time Product Reviews with Streaming Analytics
Setting Up Amazon Kinesis Data Analytics
Create a Kinesis Data Stream to Deliver Data to a Custom Application
Create AWS Lambda Function to Send Notifications via Amazon SNS
Create AWS Lambda Function to Publish Metrics to Amazon CloudWatch
Transform Streaming Data in Kinesis Data Analytics
Understand In-Application Streams and Pumps
Amazon Kinesis Data Analytics Applications
Calculate Average Star Rating
Detect Anomalies in Streaming Data
Calculate Approximate Counts of Streaming Data
Create Kinesis Data Analytics Application
Start the Kinesis Data Analytics Application
Put Messages on the Stream
Classify Product Reviews with Apache Kafka, AWS Lambda, and Amazon SageMaker
Reduce Cost and Improve Performance
Aggregate Messages
Consider Kinesis Firehose Versus Kinesis Data Streams
Enable Enhanced Fan-Out for Kinesis Data Streams
Summary
Chapter 12. Secure Data Science on AWS
Shared Responsibility Model Between AWS and Customers
Applying AWS Identity and Access Management
IAM Users
IAM Policies
IAM User Roles
IAM Service Roles
Specifying Condition Keys for IAM Roles
Enable Multifactor Authentication
Least Privilege Access with IAM Roles and Policies
Resource-Based IAM Policies
Identity-Based IAM Policies
Isolating Compute and Network Environments
Virtual Private Cloud
VPC Endpoints and PrivateLink
Limiting Athena APIs with a VPC Endpoint Policy
Securing Amazon S3 Data Access
Require a VPC Endpoint with an S3 Bucket Policy
Limit S3 APIs for an S3 Bucket with a VPC Endpoint Policy
Restrict S3 Bucket Access to a Specific VPC with an S3 Bucket Policy
Limit S3 APIs with an S3 Bucket Policy
Restrict S3 Data Access Using IAM Role Policies
Restrict S3 Bucket Access to a Specific VPC with an IAM Role Policy
Restrict S3 Data Access Using S3 Access Points
Encryption at Rest
Create an AWS KMS Key
Encrypt the Amazon EBS Volumes During Training
Encrypt the Uploaded Model in S3 After Training
Store Encryption Keys with AWS KMS
Enforce S3 Encryption for Uploaded S3 Objects
Enforce Encryption at Rest for SageMaker Jobs
Enforce Encryption at Rest for SageMaker Notebooks
Enforce Encryption at Rest for SageMaker Studio
Encryption in Transit
Post-Quantum TLS Encryption in Transit with KMS
Encrypt Traffic Between Training-Cluster Containers
Enforce Inter-Container Encryption for SageMaker Jobs
Securing SageMaker Notebook Instances
Deny Root Access Inside SageMaker Notebooks
Disable Internet Access for SageMaker Notebooks
Securing SageMaker Studio
Require a VPC for SageMaker Studio
SageMaker Studio Authentication
Securing SageMaker Jobs and Models
Require a VPC for SageMaker Jobs
Require Network Isolation for SageMaker Jobs
Securing AWS Lake Formation
Securing Database Credentials with AWS Secrets Manager
Governance
Secure Multiaccount AWS Environments with AWS Control Tower
Manage Accounts with AWS Organizations
Enforce Account-Level Permissions with SCPs
Implement Multiaccount Model Deployments
Auditability
Tag Resources
Log Activities and Collect Events
Track User Activity and API Calls
Reduce Cost and Improve Performance
Limit Instance Types to Control Cost
Quarantine or Delete Untagged Resources
Use S3 Bucket KMS Keys to Reduce Cost and Increase Performance
Summary
Index
About the Authors
Colophon