Bootstrapping Microservices with Docker, Kubernetes, and Terraform: A project-based guide

Bootstrapping Microservices with Docker, Kubernetes, and Terraform
brief contents
contents
preface
acknowledgments
about this book
Who should read this book
How this book is organized: A roadmap
About the code
liveBook discussion forum
about the author
about the cover illustration
Chapter 1: Why microservices?
1.1 This book is practical
1.2 What will I learn?
1.3 What do I need to know?
1.4 Managing complexity
1.5 What is a microservice?
1.6 What is a microservices application?
1.7 What’s wrong with the monolith?
1.8 Why are microservices popular now?
1.9 Benefits of microservices
1.10 Drawbacks of microservices
1.11 Modern tooling for microservices
1.12 Designing a microservices application
1.13 An example application
Chapter 2: Creating your first microservice
2.1 New tools
2.2 Getting the code
2.3 Why Node.js?
2.4 Our philosophy of development
2.5 Establishing our single-service development environment
2.5.1 Installing Git
2.5.2 Cloning the code repo
2.5.3 Getting Visual Studio (VS) Code
2.5.4 Installing Node.js
2.6 Building an HTTP server for video streaming
2.6.1 Creating a Node.js project
2.6.2 Installing Express
2.6.3 Creating the Express boilerplate
2.6.4 Running our simple web server
2.6.5 Adding streaming video
2.6.6 Configuring our microservice
2.6.7 Setting up for production
2.6.8 Live reloading for fast iteration
2.6.9 Running the finished code from this chapter
2.7 Node.js review
2.8 Continue your learning
Chapter 3: Publishing your first microservice
3.1 New tools
3.2 Getting the code
3.3 What is a container?
3.4 What is an image?
3.5 Why Docker?
3.6 What are we doing with Docker?
3.7 Extending our development environment with Docker
3.7.1 Installing Docker
3.7.2 Checking your Docker installation
3.8 Packaging our microservice
3.8.1 Creating a Dockerfile
3.8.2 Packaging and checking our Docker image
3.8.3 Booting our microservice in a container
3.9 Publishing our microservice
3.9.1 Creating a private container registry
3.9.2 Pushing our microservice to the registry
3.9.3 Booting our microservice from the registry
3.10 Docker review
3.11 Continue your learning
Chapter 4: Data management for microservices
4.1 New tools
4.2 Getting the code
4.3 Developing microservices with Docker Compose
4.3.1 Why Docker Compose?
4.3.2 Installing Docker Compose
4.3.3 Creating our Docker Compose file
4.3.4 Booting our microservices application
4.3.5 Working with the application
4.3.6 Shutting down the application
4.3.7 Can we use Docker Compose for production?
4.4 Adding file storage to our application
4.4.1 Using Azure Storage
4.4.2 Updating the video-streaming microservice
4.4.3 Adding our new microservice to the Docker Compose file
4.4.4 Testing the updated application
4.4.5 Cloud storage vs. cluster storage
4.4.6 What did we achieve?
4.5 Adding a database to our application
4.5.1 Why MongoDB?
4.5.2 Adding a database server in development
4.5.3 Adding a database server in production
4.5.4 Database-per-microservice or database-per-application?
4.5.5 What did we achieve?
4.6 Docker Compose review
4.7 Continue your learning
Chapter 5: Communication between microservices
5.1 New and familiar tools
5.2 Getting the code
5.3 Getting our microservices talking
5.4 Introducing the history microservice
5.5 Live reload for fast iterations
5.5.1 Creating a stub for the history microservice
5.5.2 Augmenting the microservice for live reload
5.5.3 Splitting our Dockerfile for development and production
5.5.4 Updating the Docker Compose file for live reload
5.5.5 Trying out live reload
5.5.6 Testing production mode in development
5.5.7 What have we achieved?
5.6 Methods of communication for microservices
5.6.1 Direct messaging
5.6.2 Indirect messaging
5.7 Direct messaging with HTTP
5.7.1 Why HTTP?
5.7.2 Directly targeting messages at particular microservices
5.7.3 Sending a message with HTTP POST
5.7.4 Receiving a message with HTTP POST
5.7.5 Testing the updated application
5.7.6 Sequencing direct messages
5.7.7 What have we achieved?
5.8 Indirect messaging with RabbitMQ
5.8.1 Why RabbitMQ?
5.8.2 Indirectly targeting messages to microservices
5.8.3 Creating a RabbitMQ server
5.8.4 Investigating the RabbitMQ dashboard
5.8.5 Connecting our microservice to the message queue
5.8.6 Single-recipient indirect messaging
5.8.7 Multiple-recipient messages
5.8.8 Sequencing indirect messages
5.8.9 What have we achieved?
5.9 Microservices communication review
5.10 Continue your learning
Chapter 6: Creating your production environment
6.1 New tools
6.2 Getting the code
6.3 Getting to production
6.4 Infrastructure as code
6.5 Hosting microservices on Kubernetes
6.5.1 Why Kubernetes?
6.5.2 How does Kubernetes work?
6.6 Working with the Azure CLI
6.6.1 Installing the Azure CLI
6.6.2 Authenticating with Azure
6.6.3 Which version of Kubernetes?
6.6.4 What have we achieved?
6.7 Creating infrastructure with Terraform
6.7.1 Why Terraform?
6.7.2 Installing Terraform
6.7.3 Terraform project setup
6.8 Creating an Azure resource group for your application
6.8.1 Evolutionary architecture with Terraform
6.8.2 Scripting infrastructure creation
6.8.3 Initializing Terraform
6.8.4 By-products of Terraform initialization
6.8.5 Fixing provider version numbers
6.8.6 Building your infrastructure
6.8.7 Understanding Terraform state
6.8.8 Destroying and recreating your infrastructure
6.8.9 What have we achieved?
6.9 Creating your container registry
6.9.1 Continuing the evolution of our infrastructure
6.9.2 Creating the container registry
6.9.3 Terraform outputs
6.9.4 What have we achieved?
6.10 Refactoring to share configuration data
6.10.1 Continuing the evolution of our infrastructure
6.10.2 Introducing Terraform variables
6.11 Creating our Kubernetes cluster
6.11.1 Scripting creation of your cluster
6.11.2 Cluster authentication with Azure
6.11.3 Building your cluster
6.11.4 What have we achieved?
6.12 Interacting with Kubernetes
6.12.1 Kubernetes authentication
6.12.2 The Kubernetes CLI
6.12.3 The Kubernetes dashboard
6.12.4 What have we achieved?
6.13 Terraform review
6.14 Continue your learning
Chapter 7: Getting to continuous delivery
7.1 New and familiar tools
7.2 Getting the code
7.3 Continuing to evolve our infrastructure
7.4 Continuous delivery (CD)
7.5 Deploying containers with Terraform
7.5.1 Configuring the Kubernetes provider
7.5.2 Deploying our database
7.5.3 Preparing for continuous delivery
7.5.4 Testing the new database server
7.5.5 Deploying and testing RabbitMQ
7.5.6 Tightening our security
7.5.7 What have we achieved?
7.6 Deploying our first microservice with Terraform
7.6.1 Using local variables to compute configuration
7.6.2 Building and publishing the Docker image
7.6.3 Authentication with the container registry
7.6.4 Deploying the video-streaming microservice
7.6.5 Testing your microservice
7.6.6 What have we achieved?
7.7 Continuous delivery with Bitbucket Pipelines
7.7.1 Why Bitbucket Pipelines?
7.7.2 Importing the example code to Bitbucket
7.7.3 Creating a deployment shell script
7.7.4 Managing Terraform state
7.7.5 The Bitbucket Pipelines script
7.7.6 Configuring environment variables
7.7.7 Testing your deployment pipeline
7.7.8 Debugging your deployment pipeline
7.7.9 What have we achieved?
7.8 Continue your learning
Chapter 8: Automated testing for microservices
8.1 New tools
8.2 Getting the code
8.3 Testing for microservices
8.4 Automated testing
8.5 Testing with Jest
8.5.1 Why Jest?
8.5.2 Setting up Jest
8.5.3 The math library to test
8.5.4 Your first Jest test
8.5.5 Running your first test
8.5.6 Live reload with Jest
8.5.7 Interpreting test failures
8.5.8 Invoking Jest with npm
8.5.9 Populating your test suite
8.5.10 Mocking with Jest
8.5.11 What have we achieved?
8.6 Unit testing
8.6.1 The metadata microservice
8.6.2 Creating unit tests with Jest
8.6.3 Running the tests
8.6.4 What have we achieved?
8.7 Integration testing
8.7.1 The code to test
8.7.2 Running a MongoDB database
8.7.3 Loading database fixtures
8.7.4 Creating an integration test with Jest
8.7.5 Running the test
8.7.6 What have we achieved?
8.8 End-to-end testing
8.8.1 Why Cypress?
8.8.2 Installing Cypress
8.8.3 Starting the Cypress UI
8.8.4 Setting up database fixtures
8.8.5 Booting your application
8.8.6 Creating an end-to-end test with Cypress
8.8.7 Invoking Cypress with npm
8.8.8 What have we achieved?
8.9 Automated testing in the CD pipeline
8.10 Review of testing
8.11 Continue your learning
Chapter 9: Exploring FlixTube
9.1 No new tools!
9.2 Getting the code
9.3 Revisiting essential skills
9.4 Overview of FlixTube
9.4.1 FlixTube microservices
9.4.2 Microservice project structure
9.4.3 FlixTube project structure
9.5 Running FlixTube in development
9.5.1 Booting a microservice
9.5.2 Booting the application
9.6 Testing FlixTube in development
9.6.1 Testing a microservice with Jest
9.6.2 Testing the application with Cypress
9.7 FlixTube deep dive
9.7.1 Database fixtures
9.7.2 Mocking storage
9.7.3 The gateway
9.7.4 The user interface (UI)
9.7.5 Video streaming
9.7.6 Video upload
9.8 Manually deploying FlixTube to production with Terraform
9.8.1 The Terraform scripts structure
9.8.2 Prerequisites
9.8.3 Azure authentication
9.8.4 Configuring storage
9.8.5 Deploying the application
9.8.6 Checking that it works
9.8.7 Teardown
9.8.8 Terraform modules
9.9 Continuous delivery to production
9.9.1 Prerequisites
9.9.2 Setting up your code repository
9.9.3 Preparing the backend
9.9.4 The deployment shell script
9.9.5 FlixTube’s CD configuration
9.9.6 Testing the continuous delivery (CD) pipeline
9.9.7 Adding automated testing
9.10 Review
9.11 FlixTube in the future
9.12 Continue your learning
Chapter 10: Healthy microservices
10.1 Maintaining healthy microservices
10.2 Monitoring your microservices
10.2.1 Logging in development
10.2.2 Error handling
10.2.3 Logging with Docker Compose
10.2.4 Basic logging with Kubernetes
10.2.5 Roll your own log aggregation for Kubernetes
10.2.6 Enterprise logging, monitoring and alerts
10.2.7 Automatic restarts with Kubernetes health checks
10.2.8 Tracing across microservices
10.3 Debugging microservices
10.3.1 The debugging process
10.3.2 Debugging production microservices
10.4 Reliability and recovery
10.4.1 Practice defensive programming
10.4.2 Practice defensive testing
10.4.3 Protect your data
10.4.4 Replication and redundancy
10.4.5 Fault isolation and graceful degradation
10.4.6 Simple techniques for fault tolerance
10.4.7 Advanced techniques for fault tolerance
10.5 Continue your learning
Chapter 11: Pathways to scalability
11.1 Our future is scalable
11.2 Scaling the development process
11.2.1 Multiple teams
11.2.2 Independent microservices
11.2.3 Splitting the code repository
11.2.4 Splitting the continuous delivery (CD) pipeline
11.2.5 The meta-repo
11.2.6 Creating multiple environments
11.2.7 Production workflow
11.3 Scaling performance
11.3.1 Vertically scaling the cluster
11.3.2 Horizontally scaling the cluster
11.3.3 Horizontally scaling an individual microservice
11.3.4 Elastic scaling for the cluster
11.3.5 Elastic scaling for an individual microservice
11.3.6 Scaling the database
11.3.7 Managing changes to infrastructure
11.4 Security
11.4.1 Trust models
11.4.2 Sensitive configuration
11.5 Refactoring to microservices
11.6 Microservices on a budget
11.7 From simple beginnings . . .
11.8 Continue your learning
appendix A: Creating a development environment with Vagrant
A.1 Installing VirtualBox
A.2 Installing Vagrant
A.3 Creating your virtual machine (VM)
A.4 Connecting to your VM
A.5 Installing software in the VM
A.6 Using the example setup
A.7 Turning your VM off
appendix B: Bootstrapping Microservices cheat sheet
B.1 Node.js commands
B.2 Docker commands
B.3 Docker Compose commands
B.4 Terraform commands
B.5 Testing commands
index
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
R
S
T
U
V
W
X
Y
Z