Automating DevOps with GitLab CI/CD Pipelines: Build efficient CI/CD pipelines to verify, secure, and deploy your code using real-life examples

Cover
Title Page
Copyright and Credit
Dedicated
Contributors
Table of Contents
Preface
Part 1 Getting Started with DevOps, Git, and GitLab
Chapter 1: Understanding Life Before DevOps
Introducing the Hats for Cats web app
Building and verifying code manually
Building code manually
Verifying code manually
More challenges of verifying code
Security-testing code manually
Static code analysis
Secret detection
Dynamic analysis
Dependency scanning
Container scanning
Manual security testing summary
Packaging and deploying code manually
License compliance scanning
Deploying software
Problems with manual software development life cycle practices
Solving problems with DevOps
How GitLab implements DevOps
Summary
Chapter 2: Practicing Basic Git Commands
Technical requirements
Why use Git?
What is a version control system?
What problems does a VCS solve?
Why Git is popular
Drawbacks of Git
Committing code to keep it safe
Excluding files from a repository
Tagging commits to identify versions of code
Branching code for developing in an isolated space
Git commands for managing branches
Handling merge conflicts
Syncing local and remote copies of repositories
The “golden” repository
Configuring remote repositories
Pushing
Fetching
Pulling
Additional resources for learning Git
Summary
Chapter 3: Understanding GitLab Components
Technical requirements
Emphasizing the “why” over the “how”
Introducing the GitLab platform
What is GitLab?
What problem does GitLab solve?
The verify, secure, and release stages
Organizing work into projects and groups
Example – organizing your Hats for Cats work
Tracking work with issues
The structure of a GitLab issue
The kinds of tasks that issues can represent
Labels
Issue workflows
Editing files safely with commits, branches, and merge requests
Commit history
Merging one Git branch into another
The three amigos – issues, branches, and merge requests
When two amigos are enough
How are issues and merge requests different?
Enabling DevOps practices with GitLab flow
Summary
Chapter 4: Understanding GitLab’s CI/CD Pipeline Structure
Technical requirements
Defining the terms pipeline, CI, and CD
Understanding what a pipeline is
Defining one pipeline per project
Understanding different uses of the term “pipeline”
Viewing a list of pipelines
CI – finding out if your code is good
CD – finding out where your code should go (and putting it there)
GitLab Runners
Parts of a pipeline – stages, jobs, and commands
Stages
Jobs
Commands
Fitting the pipeline pieces together
Running GitLab CI/CD pipelines
Branch pipelines
Git tag pipelines
Other types of pipelines
Skipping pipelines
Reading GitLab CI/CD pipeline statuses
Configuring GitLab CI/CD pipelines
Summary
Part 2 Automating DevOps Stages with GitLab CI/CD Pipelines
Chapter 5: Installing and Configuring GitLab Runners
Technical requirements
Defining GitLab runners and their relationship to CI/CD
GitLab Runner is an open source application written in Go
GitLab Runner runs CI/CD jobs specified in .gitlab-ci.yml
The runner architecture and supported platforms
GitLab Runner is supported on most platforms and architectures
Runners can be specific, group, or shared
Each runner has a defined executor
Runner tags restrict which runners can pick up which jobs
Installing the Runner agent
Installing GitLab Runner
Registering a runner with GitLab
Considerations regarding the various runner types and executors
Performance considerations
Security considerations
Monitoring considerations
Summary
Chapter 6: Verifying Your Code
Technical requirements
Building code in a CI/CD pipeline
Compiling Java with javac
Compiling Java with Maven
Compiling C with Gnu Compiler Collection (GCC)
Storing built code as artifacts
Checking code quality in a CI/CD pipeline
Enabling Code Quality
Viewing Code Quality results
Running automated functional tests in a CI/CD pipeline
Enabling automated functional tests
Viewing automated functional test results
Fuzz testing in a CI/CD pipeline
The architecture and workflow of fuzz testing
A fuzz testing workflow
Viewing the results of fuzz testing
Extra considerations when fuzz testing
Fuzz testing with a corpus
Checking accessibility in a CI/CD pipeline
Enabling accessibility testing
Viewing accessibility testing results
Additional ways to verify your code
Code coverage
Browser performance testing
Load performance testing
Summary
Chapter 7: Securing Your Code
Technical requirements
Understanding GitLab’s security scanning strategy
GitLab uses open-source scanners
Scanners are packaged as Docker images
Some scanners use different analyzers for different languages
Vulnerabilities don’t stop the pipeline
Findings appear in three different reports
Pipelines can use non-GitLab-provided scanners
Using SAST to scan your source code for vulnerabilities
Understanding SAST
Enabling SAST
Configuring SAST
Viewing SAST’s findings
Using Secret Detection to find private information in your repository
Understanding Secret Detection
Enabling and configuring Secret Detection
Viewing Secret Detection’s findings
Using DAST to find vulnerabilities in web applications
Understanding DAST
Enabling and configuring DAST
Viewing DAST’s findings
Using Dependency Scanning to find vulnerabilities in dependencies
Understanding Dependency Scanning
Enabling and configuring Dependency Scanning
Viewing Dependency Scanning’s findings
Using Container Scanning to find vulnerabilities in Docker images
Understanding Container Scanning
Enabling and configuring Container Scanning
Viewing Container Scanning’s findings
Using License Compliance to manage licenses of dependencies
Understanding License Compliance
Enabling and configuring License Compliance
Viewing License Compliance’s findings
Using IaC Scanning to find problems in infrastructure configuration files
Understanding IaC Scanning
Enabling and configuring IaC scanning
Viewing IaC Scanning’s findings
Understanding the different types of security reports
Managing security vulnerabilities
Integrating outside security scanners
Summary
Chapter 8: Packaging and Deploying Code
Technical requirements
Storing code in GitLab’s package registry for later re-use
Locating GitLab’s container and package registries
Getting started with the package registry
Supported package formats
Authenticating to the registry
Building and pushing packages to the package registry
Building and pushing packages to the container registry
Storing code in GitLab’s container and package registries for later deployment
Using images from the container registry
Using packages from the package registry
Deploying to different environments using GitLab Flow
Deploying to a review app for testing
Deploying to real-world production environments
Deploying to a Kubernetes cluster
The CI/CD workflow
A GitOps workflow
Summary
Part 3 Next Steps for Improving Your Applications with GitLab
Chapter 9: Enhancing the Speed and Maintainability of CI/CD Pipelines
Accelerating pipelines with directed acyclic graphs and parent-child architecture
How to create a DAG in GitLab CI
Building code for multiple architectures
When and how to leverage caching or artifacts
Caching characteristics
Artifact characteristics
Using caching
Using artifacts
Leveraging artifacts as job dependencies
Reducing repeated configuration code with anchors and extensions
Anchors
The extends: keyword
Reference tags
Improving maintainability by combining multiple pipelines and leveraging parent-child pipelines
Leveraging includes for maintainability
Leveraging includes for reusability
Includes from remote areas
Leveraging parent-child pipelines
Securing and accelerating jobs with purpose-built containers
A purpose-built container example
Summary
Chapter 10: Extending the Reach of CI/CD Pipelines
Using CI/CD pipelines to spot performance problems
How to integrate browser performance testing
How to integrate load performance testing with k6
Using feature flags to allow business-driven release decisions
How to configure your application for feature flags
Integrating third-party tools into your CI/CD pipelines
Creating our tool container’s Dockerfile
Automating our container’s build
Container scanning
Invoking the third-party tool
Using CI/CD pipelines for developing mobile apps
Requirements
Fastlane
Fastlane – deployment
Fastlane – automated testing
Summary
Chapter 11: End-to-End Example
Technical requirements
Setting up your environment
Making a GitLab project
Planning work with GitLab issues
Setting up a local Git repository
Writing code
Creating a Git branch to work on
Creating an MR
Committing and pushing code
Establishing the pipeline infrastructure
Creating a pipeline
Creating a runner
Verifying your code
Adding functional tests to the pipeline
Adding Code Quality scanning to the pipeline
Adding a fuzz test to the pipeline
Securing your code
Adding SAST to the pipeline
Adding Secret Detection to the pipeline
Adding Dependency Scanning to the pipeline
Adding License Compliance to the pipeline
Integrating a third-party security scanner into the pipeline
Improving your pipeline
Using a DAG to speed up the pipeline
Breaking the pipeline into several files
Delivering your code to the right environment
Deploying the code
Summary
Chapter 12: Troubleshooting and the Road Ahead with GitLab
Technical requirements
Troubleshooting and best practices for common pipeline problems
Troubleshooting CI/CD syntax and logic
Troubleshooting pipeline operation and runner assignment
Managing your operational infrastructure using GitOps
Using Terraform to deploy and update infrastructure state
Using Ansible to manage resource configurations
Future industry trends
Automation will create more software at a larger scale
Abstraction will lead to everything-as-code business models
Reduced cycle time will help teams release better software faster
Conclusion and next steps
Index
Other Books You May Enjoy