Harbor is a major CNCF open source project, with thousands of production users all over the world. This book provides a comprehensive explanation of the open source cloud native registry: Harbor. Written by experts who contributed to and now maintain Harbor, the content covers its architecture, principles, functions, deployment and configuration, scanning artifacts, remote replication, operation and maintenance, customized development, API usage and success stories.
The book offers a valuable guide for Harbor users, developers and contributors, cloud native software development engineers, test engineers, operational and maintenance engineers, IT architects and IT technical managers. It will also benefit university students in computer-related disciplines.
Author(s): Haining Zhang, Yan Wang
Publisher: Springer
Year: 2022
Language: English
Pages: 326
City: Singapore
Foreword
Preface
Why We Wrote This Book
Highlights of This Book
Intended Audience
How This Book Is Organized
About The Authors and Translators
Acknowledgments
Contents
Chapter 1: Artifact Management in the Cloud Native Environment
1.1 Overview of Cloud Native Application
1.2 Introduction to the Container Technology
1.2.1 Development Background of the Container Technology
1.2.2 Basic Principles of Container
1.2.2.1 Namespace
1.2.2.2 cgroups
1.2.3 Container Runtime
1.2.3.1 OCI Runtime Specification
1.2.3.2 runC
1.2.3.3 containerd
1.2.3.4 Docker
1.2.3.5 CRI and CRI-O
1.3 Structure of Container Images
1.3.1 Development of Container Images
1.3.2 Structure of Docker Images
1.3.3 OCI Image Specification
1.3.3.1 Image Index
1.3.3.2 Manifest
1.3.3.3 Image Configuration
1.3.3.4 Layer
1.4 Image Management and Distribution
1.4.1 Docker Image Management and Distribution
1.4.2 OCI Distribution Specification
1.4.3 OCI Artifact
1.5 Registry
1.5.1 Role of Registry
1.5.2 Public Registry Services
1.5.3 Private Registry Services
1.5.4 Harbor Registry
1.6 Introduction to Harbor Components
1.6.1 Overall Architecture
1.6.2 Core Components
1.6.2.1 Core Functional Components
1.6.2.2 Data Storage Components
1.6.3 Optional Components
Chapter 2: Installation and Configuration
2.1 Installing Harbor in the Stand-alone Environment
2.1.1 Basic Configuration
2.1.1.1 hostname
2.1.1.2 HTTP and HTTPS
2.1.1.3 internal_tls
2.1.1.4 harbor_admin_password
2.1.1.5 database
2.1.1.6 data_volume
2.1.1.7 storage_service
2.1.1.8 clair
2.1.1.9 trivy
2.1.1.10 jobservice
2.1.1.11 notification
2.1.1.12 chart
2.1.1.13 log
2.1.1.14 external_database
2.1.1.15 external_redis
2.1.1.16 uaa
2.1.1.17 proxy
2.1.2 Offline Installation
2.1.3 Online Installation
2.1.4 Source Code Installation
2.2 Installing Harbor Through Helm Chart
2.2.1 Obtaining Helm Chart
2.2.2 Configuring Helm Chart
2.2.2.1 Configuring the Service Exposure Mode
2.2.2.2 Configuring an External Address
2.2.2.3 Configuring Data Persistence
2.2.3 Installing Helm Chart
2.3 High-availability Solution
2.3.1 High-availability Solution Based on Harbor Helm Chart
2.3.1.1 Basic Requirements for the Installation of Harbor
2.3.1.2 High-availability Architecture
2.3.1.3 Configuring Harbor Helm Chart
2.3.1.4 Installing Harbor Helm Chart
2.3.2 High-availability Solution for Multiple Kubernetes Clusters
2.3.2.1 Installing Harbor
2.3.2.2 Highly Available Architecture of Multiple Kubernetes clusters
2.3.3 High-availability Solution Based on the Offline Installation Package
2.3.3.1 The Setup of the Load Balancer
2.3.3.2 The Configuration of External Database
2.3.3.3 The Configuration of External Redis
2.3.3.4 The Configuration of External Storage
2.3.3.5 Files or Configuration that Must Be Shared Among Multiple Harbor Instances
2.4 Configuring the Storage System
2.4.1 Amazon S3
2.4.1.1 Creating an S3 Storage Bucket
2.4.1.2 Configuring harbor.yml
2.4.2 NFS
2.4.2.1 Configuring the NFS on Each Node
2.4.2.2 Configuring Harbor
2.4.3 OSS of Alibaba Cloud
2.4.3.1 Creating an OSS storage bucket
2.4.3.2 Configuring harbor.yml
2.5 First Experience of Harbor
2.5.1 Admin Console
2.5.1.1 Projects
2.5.1.2 Administration
2.5.1.3 Theme switching
2.5.1.4 API Explorer
2.5.1.5 Labels
2.5.2 Using Harbor in Docker
2.5.2.1 Pushing an Image to Harbor
2.5.2.2 Pulling an Image from Harbor
2.5.3 Using Harbor in Kubernetes
2.5.3.1 imagePullPolicy
2.5.3.2 imagePullSecrets attribute
2.6 FAQ
Chapter 3: Access Control
3.1 Overview
3.1.1 Authentication and Authorization
3.1.2 Resource Isolation
3.1.3 Client Authentication
3.2 User Authentication
3.2.1 Local Database Authentication
3.2.2 LDAP Authentication
3.2.3 OIDC Provider Authentication
3.3 Access Control and Authorization
3.3.1 Role-based Access Policy
3.3.2 Users and Groups
3.4 Robot Account
3.5 FAQ
Chapter 4: Security Policy
4.1 Trusted Content Distribution
4.1.1 Content Trust
4.1.1.1 Integrating Notary
4.1.1.2 Using Notary to Sign Artifacts
4.1.2 Signature of Helm 2 Chart
4.2 Pluggable Vulnerability Scanning
4.2.1 Overall Design
4.2.2 Scanner Management
4.2.3 Scanning API Specification
4.2.4 Scanning Management
4.2.5 Asynchronous Scanning Task
4.2.6 APIs Related to Scanning
4.3 Using the Vulnerability Scanning Function
4.3.1 System Scanner
4.3.2 Project Scanner
4.3.3 Project Vulnerability Scanning
4.3.4 Global Vulnerability Scanning
4.3.5 Automatic Scanning
4.3.6 Security Policy of Deployment Associated with Vulnerability
4.4 FAQ
Chapter 5: Remote Replication of Content
5.1 Basic Principles
5.2 Setting up a Registry Endpoint
5.3 Replication Policy
5.3.1 Replication Mode
5.3.2 Filter
5.3.3 Triggering Mode
5.3.4 Creating a Replication Policy
5.3.5 Executing a Replication Policy
5.4 Content Replication Between Harbor Instances
5.5 Content Replication Between Third-party Registry Services
5.5.1 Content Replication with Docker Hub
5.5.2 Content Replication with Docker Registry
5.5.3 Content Replication with Amazon ECR
5.5.4 Content Replication with GCR
5.5.5 Content Replication with Helm Hub
5.6 Typical Usage Scenarios
5.6.1 Artifact Distribution
5.6.2 Two-way Synchronization
5.6.3 Transfer of DevOps Images
5.6.4 Other Scenarios
Chapter 6: Advanced Management
6.1 Resource Quota Management
6.1.1 Principles
6.1.1.1 Composition of an OCI Artifact
6.1.1.2 Pushing an Artifact to an Artifact Repository
6.1.1.3 Layer Management and Sharing of Docker Distribution
6.1.1.4 PATCH Blob
6.1.1.5 PUT Blob
6.1.1.6 PUT Manifest
6.1.2 Setting Project Quota
6.1.3 Setting System Quota
6.1.4 Use of Quota
6.1.4.1 Pushing of Artifacts
6.1.4.2 Deletion of Artifacts
6.1.4.3 Copying of Artifacts
6.1.4.4 Remote Replication of Artifacts
6.1.4.5 Operations of Untagged Artifact
6.1.5 Message of Out-of-Quota
6.1.5.1 Insufficient Quota When a Docker Client Pushes Images
6.1.5.2 Insufficient Quota of Other Projects
6.2 Garbage Collection
6.2.1 Principles
6.2.1.1 Setting the Read-Only Mode
6.2.1.2 Marking Alternative Artifacts
6.2.1.3 Deleting Manifest
6.2.1.4 Performing Registry Garbage Collection
6.2.1.5 Releasing Quota Space
6.2.1.6 Cleaning Up the Cache
6.2.1.7 Restoring the Read/Write Status
6.2.2 Triggering Mode
6.2.3 Execution of Garbage Collection
6.2.4 Non-blocking Garbage Collection
6.2.4.1 Counting Reference of Layers
6.2.4.2 Using the Cloud Storage
Artifact Database
API for Deleting Layers and Manifest files
Non-blocking
State Control
Time Window
6.3 Immutable Artifact
6.3.1 Principles
6.3.1.1 Not Overwritable
6.3.1.2 Not Deletable
6.3.2 Setting Rules of Immutable Artifacts
6.3.3 Using Rules of Immutable Artifacts
6.3.3.1 Push
6.3.3.2 Delete
6.4 Artifact Retention Policy
6.4.1 Principles
6.4.2 Setting a Retention Policy
6.4.3 Dry Run of the Retention Policy
6.4.4 Triggering the Retention Policy
6.4.4.1 Manual
6.4.4.2 Scheduled
6.5 Webhook
6.5.1 Principles
6.5.1.1 Overall Architecture
6.5.1.2 Message Structure
6.5.1.3 Message Retry
6.5.2 Configuring Webhook
6.5.2.1 Creating a Webhook
6.5.2.2 Managing Webhook
6.5.2.3 Viewing a Webhook Policy
6.5.2.4 Flag to Enable or Disable Webhook
6.5.3 Interaction with Other Systems
6.6 Multiple Languages
6.7 FAQ
Chapter 7: Lifecycle Management
7.1 Backup and Restoration
7.1.1 Data Backup
7.1.2 Restoration of Harbor
7.1.3 Backup and Restoration of Helm Deployment
7.1.4 Backup and Restoration Using Image Replication
7.2 Upgrading Harbor Version
7.2.1 Data Migration
7.2.2 Upgrading Harbor
7.3 System Troubleshooting
7.4 Common Problems
7.4.1 The Configuration File Does Not Take Effect
7.4.2 Harbor Cannot Be Started After Docker Is Restarted
7.4.3 A Signed Image Cannot Be Deleted When the Secret Key Is Lost
7.4.4 The Password of the System Administrator Is Lost
Chapter 8: Harbor APIs
8.1 Overview of APIs
8.1.1 Overview of the Core Management APIs
8.1.1.1 API Version
8.1.1.2 Authentication Mode
8.1.1.3 Error Format
8.1.1.4 Query Keyword ``q´´
8.1.2 Overview of Registry APIs
8.1.2.1 Basic Auth
8.1.2.2 Bearer Token Authentication
8.2 Core Management APIs
8.2.1 User Management APIs
8.2.2 Project Management APIs
8.2.3 Repository Management APIs
8.2.4 Artifact Management APIs
8.2.5 Remote Replication APIs
8.2.6 Scanning APIs
8.2.7 Garbage Collection APIs
8.2.8 Project Quota APIs
8.2.9 Tag Retention APIs
8.2.10 Immutable Artifact APIs
8.2.11 Webhook APIs
8.2.12 System Service APIs
8.2.13 API Explorer
8.3 Registry APIs
8.3.1 Base API
8.3.2 Catalog API
8.3.3 Tag API
8.3.4 Manifest API
8.3.5 Blob API
8.4 Programming Example
8.5 Summary
Chapter 9: Governance Model of Harbor Community
9.1 Governance Model
9.2 Security Release Process
9.3 Community Participation
9.4 Participating in Code Contribution
9.4.1 Setting Up the Development Environment
9.4.2 The Process of Contributing Code
9.4.2.1 Forking the Code Repository
9.4.2.2 Creating a Branch
9.4.2.3 Develop, Build and Test
9.4.2.4 Synchronizing with the Upstream Code Repository
9.4.2.5 Submitting the Code
9.4.2.6 Submitting a Pull Request
9.4.2.7 Automated Check
9.4.2.8 Code Review
