Scaling Python with Ray: Adventures in Cloud and Serverless Patterns

Cover
Copyright
Table of Contents
Foreword
Preface
What You Will Learn
A Note on Responsibility
Conventions Used in This Book
License
Using Code Examples
O’Reilly Online Learning
How to Contact Us
Acknowledgments
From Holden
From Boris
Chapter 1. What Is Ray, and Where Does It Fit?
Why Do You Need Ray?
Where Can You Run Ray?
Running Your Code with Ray
Where Does It Fit in the Ecosystem?
Big Data / Scalable DataFrames
Machine Learning
Workflow Scheduling
Streaming
Interactive
What Ray Is Not
Conclusion
Chapter 2. Getting Started with Ray (Locally)
Installation
Installing for x86 and M1 ARM
Installing (from Source) for ARM
Hello Worlds
Ray Remote (Task/Futures) Hello World
Data Hello World
Actor Hello World
Conclusion
Chapter 3. Remote Functions
Essentials of Ray Remote Functions
Composition of Remote Ray Functions
Ray Remote Best Practices
Bringing It Together with an Example
Conclusion
Chapter 4. Remote Actors
Understanding the Actor Model
Creating a Basic Ray Remote Actor
Implementing the Actor’s Persistence
Scaling Ray Remote Actors
Ray Remote Actors Best Practices
Conclusion
Chapter 5. Ray Design Details
Fault Tolerance
Ray Objects
Serialization/Pickling
cloudpickle
Apache Arrow
Resources / Vertical Scaling
Autoscaler
Placement Groups: Organizing Your Tasks and Actors
Namespaces
Managing Dependencies with Runtime Environments
Deploying Ray Applications with the Ray Job API
Conclusion
Chapter 6. Implementing Streaming Applications
Apache Kafka
Basic Kafka Concepts
Kafka APIs
Using Kafka with Ray
Scaling Our Implementation
Building Stream-Processing Applications with Ray
Key-Based Approach
Key-Independent Approach
Going Beyond Kafka
Conclusion
Chapter 7. Implementing Microservices
Understanding Microservice Architecture in Ray
Deployment
Additional Deployment Capabilities
Deployment Composition
Using Ray Serve for Model Serving
Simple Model Service Example
Considerations for Model-Serving Implementations
Speculative Model Serving Using the Ray Microservice Framework
Conclusion
Chapter 8. Ray Workflows
What Is Ray Workflows?
How Is It Different from Other Solutions?
Ray Workflows Features
What Are the Main Features?
Workflow Primitives
Working with Basic Workflow Concepts
Workflows, Steps, and Objects
Dynamic Workflows
Virtual Actors
Workflows in Real Life
Building Workflows
Managing Workflows
Building a Dynamic Workflow
Building Workflows with Conditional Steps
Handling Exceptions
Handling Durability Guarantees
Extending Dynamic Workflows with Virtual Actors
Integrating Workflows with Other Ray Primitives
Triggering Workflows (Connecting to Events)
Working with Workflow Metadata
Conclusion
Chapter 9. Advanced Data with Ray
Creating and Saving Ray Datasets
Using Ray Datasets with Different Tools
Using Tools on Ray Datasets
pandas-like DataFrames with Dask
Indexing
Shuffles
Embarrassingly Parallel Operations
Working with Multiple DataFrames
What Does Not Work
What’s Slower
Handling Recursive Algorithms
What Other Functions Are Different
pandas-like DataFrames with Modin
Big Data with Spark
Working with Local Tools
Using Built-in Ray Dataset Operations
Implementing Ray Datasets
Conclusion
Chapter 10. How Ray Powers Machine Learning
Using scikit-learn with Ray
Using Boosting Algorithms with Ray
Using XGBoost
Using LightGBM
Using PyTorch with Ray
Reinforcement Learning with Ray
Hyperparameter Tuning with Ray
Conclusion
Chapter 11. Using GPUs and Accelerators with Ray
What Are GPUs Good At?
The Building Blocks
Higher-Level Libraries
Acquiring and Releasing GPU and Accelerator Resources
Ray’s ML Libraries
Autoscaler with GPUs and Accelerators
CPU Fallback as a Design Pattern
Other (Non-GPU) Accelerators
Conclusion
Chapter 12. Ray in the Enterprise
Ray Dependency Security Issues
Interacting with the Existing Tools
Using Ray with CI/CD Tools
Authentication with Ray
Multitenancy on Ray
Credentials for Data Sources
Permanent Versus Ephemeral Clusters
Ephemeral Clusters
Permanent Clusters
Monitoring
Instrumenting Your Code with Ray Metrics
Wrapping Custom Programs with Ray
Conclusion
Appendix A. Space Beaver Case Study: Actors, Kubernetes, and More
High-Level Design
Implementation
Outbound Mail Client
Shared Actor Patterns and Utilities
Mail Server Actor
Satellite Actor
User Actor
SMS Actor and Serve Implementation
Testing
Deployment
Conclusion
Appendix B. Installing and Deploying Ray
Installing Ray Locally
Using Ray Docker Images
Using Ray Clusters
Installing Ray on AWS
Installing Ray on IBM Cloud
Installing Ray on Kubernetes
Installing Ray on a kind Cluster
Using ray up
Using the Ray Kubernetes Operator
Installing Ray on OpenShift
Conclusion
Appendix C. Debugging with Ray
General Debugging Tips with Ray
Serialization Errors
Local Debugging with Ray Local
Remote Debugging
Ray’s Integrated Debugger (via Pdb)
Other Tools
Ray and Container Exit Codes
Ray Logs
Container Errors
Native Errors
Conclusion
Index
About the Authors
Colophon