Start thinking about your development pipeline as a mission-critical application. Discover techniques for implementing code-driven infrastructure and CI/CD workflows using Jenkins, Docker, Terraform, and cloud-native services.
In Pipeline as Code, you will master:
• Building and deploying a Jenkins cluster from scratch
• Writing pipeline as code for cloud-native applications
• Automating the deployment of Dockerized and Serverless applications
• Containerizing applications with Docker and Kubernetes
• Deploying Jenkins on AWS, GCP and Azure
• Managing, securing and monitoring a Jenkins cluster in production
• Key principles for a successful DevOps culture
Pipeline as Code is a practical guide to automating your development pipeline in a cloud-native, service-driven world. You’ll use the latest infrastructure-as-code tools like Packer and Terraform to develop reliable CI/CD pipelines for numerous cloud-native applications. Follow this book's insightful best practices, and you’ll soon be delivering software that’s quicker to market, faster to deploy, and with less last-minute production bugs.
about the technology
Treat your CI/CD pipeline like the real application it is. With the Pipeline as Code approach, you create a collection of scripts that replace the tedious web UI wrapped around most CI/CD systems. Code-driven pipelines are easy to use, modify, and maintain, and your entire CI pipeline becomes more efficient because you directly interact with core components like Jenkins, Terraform, and Docker.
about the book
In Pipeline as Code you’ll learn to build reliable CI/CD pipelines for cloud-native applications. With Jenkins as the backbone, you’ll programmatically control all the pieces of your pipeline via modern APIs. Hands-on examples include building CI/CD workflows for distributed Kubernetes applications, and serverless functions. By the time you’re finished, you’ll be able to swap manual UI-based adjustments with a fully automated approach!
What's inside
• Build and deploy a Jenkins cluster on scale
• Write pipeline as code for cloud-native applications
• Automate the deployment of Dockerized and serverless applications
• Deploy Jenkins on AWS, GCP, and Azure
• Grasp key principles of a successful DevOps culture
About the reader
For developers familiar with Jenkins and Docker. Examples in Go.
About the author
Mohamed Labouardy is the CTO and co-founder of Crew.work, a Jenkins contributor, and a DevSecOps evangelist.
Author(s): Mohamed Labouardy
Edition: 1
Publisher: Manning
Year: 2021
Language: English
Commentary: Vector PDF
Pages: 528
City: Shelter Island, NY
Tags: DevOps; Google Cloud Platform; DigitalOcean; Amazon Web Services; Microsoft Azure; Monitoring; Pipelines; Scalability; Docker; GitHub; Continuous Delivery; Kubernetes; Jenkins; Continuous Integration; Automation; Testing; AWS Lambda; Docker Swarm; Terraform; Serverless Applications; Continuous Deployment; Amazon Virtual Private Cloud; Packer; Manageability
Pipeline as Code
brief contents
contents
preface
acknowledgments
about this book
Who should read this book
How this book is organized
About the code
liveBook discussion forum
Other online resources
about the author
about the cover illustration
Part 1 Getting started with Jenkins
1 What’s CI/CD?
1.1 Going cloud native
1.1.1 Monolithic
1.1.2 Microservices
1.1.3 Cloud native
1.1.4 Serverless
1.2 Defining continuous integration
1.3 Defining continuous deployment
1.4 Defining continuous delivery
1.5 Embracing CI/CD practices
1.6 Using essential CI/CD tools
1.6.1 Choosing a CI/CD tool
1.6.2 Introducing Jenkins
Summary
2 Pipeline as code with Jenkins
2.1 Introducing the Jenkinsfile
2.1.1 Blue Ocean plugin
2.1.2 Scripted pipeline
2.1.3 Declarative pipeline
2.2 Understanding multibranch pipelines
2.3 Exploring the GitFlow branch model
2.4 Test-driven development with Jenkins
2.4.1 The Jenkins Replay button
2.4.2 Command-line pipeline linter
2.4.3 IDE integrations
Summary
Part 2 Operating a self-healing Jenkins cluster
3 Defining Jenkins architecture
3.1 Understanding master-worker architecture
3.2 Managing Jenkins workers
3.2.1 SSH
3.2.2 Command line
3.2.3 JNLP
3.2.4 Windows service
3.3 Architecting Jenkins for scale in AWS
3.3.1 Preparing the AWS environment
3.3.2 Configuring the AWS CLI
3.3.3 Creating and managing the IAM user
Summary
4 Baking machine images with Packer
4.1 Immutable infrastructure
4.2 Introducing Packer
4.2.1 How does it work?
4.2.2 Installation and configuration
4.2.3 Baking a machine image
4.3 Baking the Jenkins master AMI
4.3.1 Configuring Jenkins upon startup
4.3.2 Discovering Jenkins plugins
4.4 Baking the Jenkins worker AMI
Summary
5 Discovering Jenkins as code with Terraform
5.1 Introducing infrastructure as code
5.1.1 Terraform usage
5.2 Provisioning an AWS VPC
5.2.1 AWS VPC
5.2.2 VPC subnets
5.2.3 VPC route tables
5.2.4 VPC bastion host
5.3 Setting up a self-healing Jenkins master
5.4 Running Jenkins with native SSL/HTTPS
5.5 Dynamically autoscaling the Jenkins worker pool
5.5.1 Launch configuration
5.5.2 Auto Scaling group
5.5.3 Autoscaling scaling policies
5.5.4 Workers CPU utilization load
Summary
6 Deploying HA Jenkins on multiple cloud providers
6.1 Google Cloud Platform
6.1.1 Building Jenkins VM images
6.1.2 Configuring a GCP network with Terraform
6.1.3 Deploying Jenkins on Google Compute Engine
6.1.4 Launching automanaged workers on GCP
6.2 Microsoft Azure
6.2.1 Building golden Jenkins VM images in Azure
6.2.2 Deploying a private virtual network
6.2.3 Deploying a Jenkins master virtual machine
6.2.4 Applying autoscaling to Jenkins workers
6.3 DigitalOcean
6.3.1 Creating Jenkins DigitalOcean Snapshots
6.3.2 Deploying a Jenkins master Droplet
6.3.3 Building Jenkins worker Droplets
Summary
Part 3 Hands-on CI/CD pipelines
7 Defining a pipeline as code for microservices
7.1 Introducing microservices-based applications
7.2 Defining multibranch pipeline jobs
7.3 Git and GitHub integration
7.4 Discovering Jenkins jobs’ XML configuration
7.5 Configuring SSH authentication with Jenkins
7.6 Triggering Jenkins builds with GitHub webhooks
Summary
8 Running automated tests with Jenkins
8.1 Running unit tests inside Docker containers
8.2 Automating code linter integration with Jenkins
8.3 Generating code coverage reports
8.4 Injecting security in the CI pipeline
8.5 Running parallel tests with Jenkins
8.6 Improving quality with code analysis
8.7 Running mocked database tests
8.8 Generating HTML coverage reports
8.9 Automating UI testing with Headless Chrome
8.10 Integrating SonarQube Scanner with Jenkins
Summary
9 Building Docker images within a CI pipeline
9.1 Building Docker images
9.1.1 Using the Docker DSL
9.1.2 Docker build arguments
9.2 Deploying a Docker private registry
9.2.1 Nexus Repository OSS
9.2.2 Amazon Elastic Container Registry
9.2.3 Azure Container Registry
9.2.4 Google Container Registry
9.3 Tagging Docker images the right way
9.4 Scanning Docker images for vulnerabilities
9.5 Writing a Jenkins declarative pipeline
9.6 Managing pull requests with Jenkins
Summary
10 Cloud-native applications on Docker Swarm
10.1 Running a distributed Docker Swarm cluster
10.2 Defining a continuous deployment process
10.3 Integrating Jenkins with Slack notifications
10.4 Handling code promotion with Jenkins
10.5 Implementing the Jenkins delivery pipeline
Summary
11 Dockerized microservices on K8s
11.1 Setting up a Kubernetes cluster
11.2 Automating continuous deployment flow with Jenkins
11.2.1 Migrating Docker Compose to K8s manifests with Kompose
11.3 Walking through continuous delivery steps
11.4 Packaging Kubernetes applications with Helm
11.5 Running post-deployment smoke tests
11.6 Discovering Jenkins X
Summary
12 Lambda-based serverless functions
12.1 Deploying a Lambda-based application
12.2 Creating deployment packages
12.2.1 Mono-repo strategy
12.2.2 Multi-repo strategy
12.3 Updating Lambda function code
12.4 Hosting a static website on S3
12.5 Maintaining multiple Lambda environments
12.6 Configuring email notification in Jenkins
Summary
Part 4 Managing, scaling, and monitoring Jenkins
13 Collecting continuous delivery metrics
13.1 Monitoring Jenkins cluster health
13.2 Centralized logging for Jenkins logs with ELK
13.2.1 Streaming logs with Filebeat
13.2.2 Streaming logs with the Logstash plugin
13.3 Creating alerts based on metrics
Summary
14 Jenkins administration and best practices
14.1 Exploring Jenkins security and RBAC authorization
14.1.1 Matrix authorization strategy
14.1.2 Role-based authorization strategy
14.2 Configuring GitHub OAuth for Jenkins
14.3 Keeping track of Jenkins users’ actions
14.4 Extending Jenkins with shared libraries
14.5 Backing up and restoring Jenkins
14.6 Setting up cron jobs with Jenkins
14.7 Running Jenkins locally as a Docker container
Summary
Wrapping up
index
Symbols
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
X
Pipeline as Code-back
