Demystify the creation of efficient AI systems using the model-based reinforcement learning Unity ML-Agents - a powerful bridge between the world of Unity and Python.
We will start with an introduction to the field of AI, then discuss the progression of AI and where we are today. We will follow this up with a discussion of moral and ethical considerations. You will then learn how to use the powerful machine learning tool and investigate different potential real-world use cases. We will examine how AI agents perceive the simulated world and how to use inputs, outputs, and rewards to train efficient and effective neural networks. Next, you'll learn how to use Unity ML-Agents and how to incorporate them into your game or product. This book will thoroughly introduce you to ML-Agents in Unity and how to use them in your next project.
Explore the world of machine learning through Unity ML-Agents. In this book, you’ll learn about the impact of artificial intelligence and learn to build a reinforcement learning agent using the Unity ML-Agents package. It’s strongly recommended to go into this book with a solid understanding of the Unity engine and C#. The instructional chapters are written for Microsoft Windows 10, and some steps may vary across operating systems. The book also includes a sample repository with the code we will cover in this book and the solution to the challenge proposed in Chapter 8.
Machine Learning, neural networks, Deep Learning, and Artificial Intelligence are all words that you’ve already heard. While these terms are similar, they do have differences. Artificial Intelligence is a field of computer science in which we give a machine the ability to algorithmically process data and make decisions. On the other hand, Machine Learning is a subset of Artificial Intelligence covering the process in which a machine learns to think, much like the human brain in a process called reinforcement learning.
Author(s): Dylan Engelbrecht
Publisher: Apress
Year: 2023
Language: english
Pages: 213
Table of Contents
About the Author
About the Technical Reviewers
Acknowledgments
Introduction
Chapter 1: Introduction
What Is Machine Learning?
How We Use Machine Learning in the Modern Day
Serving Content Recommendations
Autonomous Vehicles
Power and Electrical Grid Management
Vaccines and Medical Drugs
Farming
Security and Surveillance
Military
City Planning
Prerequisites
Conclusion
Chapter 2: History of AI and Where We Are Today
The People Who Shaped Artificial Intelligence
Alan Mathison Turing
John McCarthy
Marvin Lee Minsky
Guido van Rossum
Modern-Day Companies Paving the Future of AI
Python Software Foundation
Nvidia
IBM
Google
Tesla
OpenAI
How AI Has Evolved in Games, from Chess to Dota 2
So, Where Are We Now with AI in Game Development?
GitHub Copilot
A Neural State Machine for Character-Scene Interactions
BLOOM, a BigScience Initiative
Conclusion
Chapter 3: The Future of AI and Ethical Implications
The Future of AI
Law and Justice
Healthcare
Taxes and Governance
Life Extension and Brain-Computer Interfaces
Entertainment
Avoiding a Bad Future
Bias and Why We Need Diverse Datasets
So, What Is Bias in AI?
Why We Need Diverse Datasets
Discussing the Moral and Ethical Implications
Why AI?
Flavors of AI
AI Road Map and Classification
Reactive Machines
Limited Memory
Theory of Mind
Artificial General Intelligence (AGI)
Self-Aware
Artificial Superintelligence (ASI)
Machine Learning with Unity ML-Agents
Reinforcement Learning
Imitation Learning
Neuroevolution
Practical Use Cases for Unity ML-Agents
Learning How to Build Machine Learning Agents
Self-Driving Cars
Game AI
Robotics
Simulated Space for Agent Training
Training Gym for Agents
Conclusion
Chapter 4: Dopamine for Machines
Dopamine
Dopamine in Humans
Dopamine in Animals
Dopamine in Machines
Training Reinforcement Learning Agents
How and When to Reward Your ML-Agents
A Sound Reward System Makes for Great ML-Agents
How Reward Systems Influence Training Time
Various Aspects of Rewarding and Punishing ML-Agents
Team-Based Rewards
Conclusion
Chapter 5: ML-Agents Setup
Unity Setup
New Project Setup
ML-Agents Unity Package Setup
Installing the ML-Agents Extensions Package
Opening the Example GitHub Project
Creating a GitHub Issue
Python Setup
Creating a Virtual Environment
Installing ML-Agents and Dependencies
Validating Our ML-Agents Installation with Samples
Conclusion
Chapter 6: Unity ML-Agents
ML-Agent Components
Behavior Parameters
Behavior Name
Vector Observations
Actions
Model
Behavior Type
Team Id
Use Child Sensors
Observable Attributes
The Decision Requester
Learning Environments
The Agent
Agent Override Methods
override void Initialize()
override void CollectObservations(VectorSensor sensor)
override void OnActionReceived(ActionBuffers actionBuffers)
override void OnEpisodeBegin()
override void Heuristic(in ActionBuffers actionsOut)
void RequestDecision()
void AddReward(float increment)
void SetReward(float reward)
void EndEpisode()
Inputs and Outputs
Inputs, Observations, and Sensors
So How Do We Create Observations?
Collecting Observations
Using the Observable Attribute
Creating Sensors
Building Sensors
GetObservationSpec
Vector Observation Spec
Visual Observation Spec
Variable Length Observation Spec
Write
ObservationWriter
Writer[index] = observation
AddList(IList, Int32)
Add(Vector3, Int32)
Add(Quaternion, Int32)
Add(Vector4, Int32)
GetCompressedObservation
Update
Reset
GetCompressionSpec
GetName
Visual Observations
Actions
Continuous
Discrete
Heuristics
Rewards
Training an Agent
Conclusion
Chapter 7: Creating Your First AI in Unity
Planning an Agent
The Avoidance Sample
Reward Scheme
Observation Plans
Actions Planning
Expected Challenges
Building Your First ML-Agent
The Grid Sensor
The Ray Perception Sensor
Building the Environment
Understanding Hyperparameters
Training Your Agent
Duplicating Your Training Zones
TensorBoard and Why It’s Essential for Training
Connecting Stand-Alone Builds to Python
Exporting and Loading Your Model
Conclusion
Chapter 8: Solve a Challenge with AI
The Challenge
Grazer Agents
Predator Agents
Bonus Objective
Before You Start
Other Techniques to Consider
CL (Curriculum Learning)
BC (Behavioral Cloning)
Self-Play
Tips
Conclusion
Chapter 9: Next Steps
Explore
Additional ML-Agent Functionality
Documentation
Additional Reading
Conclusion
Index
