Kubernetes is an open source system that is used to automate the deployment, scaling, and management of containerized applications. If you are running more containers or want automated management of your containers, you need Kubernetes at your disposal. To put things into perspective, Mastering Kubernetes walks you through the advanced management of Kubernetes clusters. To start with, you will learn the fundamentals of both Kubernetes architecture and Kubernetes design in detail. You will discover how to run complex stateful microservices on Kubernetes including advanced features such as horizontal pod autoscaling, rolling updates, resource quotas, and persistent storage backend. Using real-world use cases, you will explore the options for network configuration, and understand how to set up, operate, and troubleshoot various Kubernetes networking plugins. In addition to this, you will get to grips with custom resource development and utilization in automation and maintenance workflows. To scale up your knowledge of Kubernetes, you will encounter some additional concepts based on the Kubernetes 1.10 release, such as Promethus, Role-based access control, API aggregation, and more. By the end of this book, you'll know everything you need to graduate from intermediate to advanced level of understanding Kubernetes.
Author(s): Gigi Sayfan
Edition: 2nd
Publisher: Packt Publishing
Year: 2018
Language: English
Pages: 457
Tags: Containerized Applications, Kubernetes
Cover......Page 1
Title Page......Page 2
Copyright and Credits......Page 3
Packt Upsell......Page 4
Contributors......Page 5
Table of Contents......Page 7
Preface......Page 21
Chapter 1: Understanding Kubernetes Architecture......Page 26
What Kubernetes is not......Page 27
The benefits of containers......Page 28
Cattle versus pets......Page 29
Kubernetes concepts......Page 30
Cluster......Page 31
Pod......Page 32
Annotations......Page 33
Replication controllers and replica sets......Page 34
Volume......Page 35
Names......Page 36
Distributed systems design patterns......Page 37
Adapter pattern......Page 38
The Kubernetes APIs......Page 39
Workloads API......Page 40
Config and storage......Page 41
Master components......Page 42
Cloud controller manager......Page 43
Proxy......Page 45
Kubernetes runtimes......Page 46
The Container Runtime Interface (CRI)......Page 47
Docker......Page 49
App container......Page 50
Is rkt ready for use in production?......Page 51
Continuous integration and deployment......Page 52
What is a CI/CD pipeline?......Page 53
Summary......Page 54
Chapter 2: Creating Kubernetes Clusters......Page 55
On Windows......Page 56
On macOS......Page 57
Creating the cluster......Page 58
Troubleshooting......Page 59
Checking out the cluster......Page 60
Examining the cluster with the dashboard......Page 62
Getting ready......Page 64
Installing the required software......Page 65
The host file......Page 66
The playbook.yml file......Page 67
Initializing the master......Page 68
Setting up the pod network......Page 70
Adding the worker nodes......Page 71
The cloud-provider interface......Page 72
Amazon Web Services (AWS)......Page 73
Amazon Elastic Container Service for Kubernetes (EKS)......Page 74
Azure......Page 75
Use cases for bare metal......Page 76
The process......Page 77
Summary......Page 78
Chapter 3: Monitoring, Logging, and Troubleshooting......Page 79
cAdvisor......Page 80
Installing Heapster......Page 81
InfluxDB backend......Page 82
CPU......Page 83
Memory......Page 84
Uptime......Page 85
Grafana visualization......Page 86
Performance analysis with the dashboard......Page 87
Cluster......Page 88
Workloads......Page 92
Discovery and load balancing......Page 94
Adding central logging......Page 95
Fluentd......Page 96
Detecting node problems......Page 97
Problem daemons......Page 98
Hardware failure......Page 99
Quotas, shares, and limits......Page 100
Bad configuration......Page 102
Using Prometheus......Page 103
Installing Prometheus with kube-prometheus......Page 104
Monitoring your cluster with Prometheus......Page 105
Summary......Page 108
High-availability concepts......Page 109
Hot swapping......Page 110
Idempotency......Page 111
Creating highly available clusters......Page 112
Clustering etcd......Page 114
Installing the etcd operator......Page 115
Verifying the etcd cluster......Page 118
Running leader election with Kubernetes......Page 119
Leader election for your application......Page 120
Making your staging environment highly available......Page 122
Testing high availability......Page 123
Live cluster upgrades......Page 124
Rolling upgrades......Page 125
Complex deployments......Page 126
Managing data-contract changes......Page 127
Deprecating APIs......Page 128
Availability requirements......Page 129
Maintenance windows......Page 130
Quick recovery......Page 131
Zero-downtime......Page 132
Summary......Page 134
Chapter 5: Configuring Kubernetes Security, Limits, and Accounts......Page 136
Node challenges......Page 137
Image challenges......Page 140
Pod and container challenges......Page 141
Organizational, cultural, and process challenges......Page 142
Understanding service accounts in Kubernetes......Page 143
Accessing the API server......Page 145
Authenticating users......Page 146
Authorizing requests......Page 148
Using admission control plugins......Page 149
Securing pods......Page 151
ImagePullSecrets......Page 152
Specifying a security context......Page 153
Securing a pod with AppArmor......Page 154
Writing AppArmor profiles......Page 155
Pod security policies......Page 156
Authorizing pod security policies through RBAC......Page 157
Choosing a supported networking solution......Page 158
Defining a network policy......Page 159
Cross-namespace policies......Page 160
Configuring encryption at Rest......Page 161
Creating secrets......Page 162
Using secrets in a container......Page 163
Running a multiuser cluster......Page 164
Using namespaces for safe multitenancy......Page 165
Avoiding namespace pitfalls......Page 166
Summary......Page 167
Designing the Hue platform......Page 168
Defining the scope of Hue......Page 169
Hue components......Page 170
Hue microservices......Page 172
Automatic workflows......Page 173
Using Kubectl effectively......Page 174
Understanding Kubectl resource configuration files......Page 175
Creating pods......Page 177
Deploying long-running processes with deployments......Page 179
Updating a deployment......Page 180
Deploying an internal service......Page 181
Creating the hue-reminders service......Page 183
Ingress......Page 184
Using namespace to limit access......Page 186
Launching jobs......Page 188
Running jobs in parallel......Page 189
Scheduling cron jobs......Page 190
Inside-the-cluster-network components......Page 192
Using liveness probes to ensure your containers are alive......Page 193
Using readiness probes to manage dependencies......Page 194
Employing Init Containers for orderly pod bring-up......Page 195
Sharing with DaemonSet pods......Page 196
Advancing science with Hue......Page 197
Summary......Page 198
Persistent volumes walk-through......Page 199
Using emptyDir for intra-pod communication......Page 200
Using HostPath for intra-node communication......Page 201
Using local volumes for durable node storage......Page 203
Provisioning persistent volumes......Page 204
Creating persistent volumes......Page 205
Access modes......Page 206
Making persistent volume claims......Page 207
Mounting claims as volumes......Page 209
Raw block volumes......Page 210
Storage classes......Page 211
Demonstrating persistent volume storage end to end......Page 213
Public storage volume types – GCE, AWS, and Azure......Page 216
AWS Elastic Block Store (EBS)......Page 217
AWS Elastic File System......Page 218
GCE persistent disk......Page 219
Azure file storage......Page 220
Using GlusterFS......Page 222
Creating endpoints......Page 223
Creating pods......Page 224
Connecting to Ceph using RBD......Page 225
Connecting to Ceph using CephFS......Page 227
Flocker as a clustered container data volume manager......Page 228
Integrating enterprise storage into Kubernetes......Page 229
Projecting volumes......Page 230
Using out-of-tree volume plugins with FlexVolume......Page 231
The Container Storage Interface......Page 232
Summary......Page 233
Stateful versus stateless applications in Kubernetes......Page 234
Why manage state outside of Kubernetes?......Page 235
Accessing external data stores via environment variables......Page 236
Consuming a ConfigMap as an environment variable......Page 237
Utilizing StatefulSet......Page 239
The components of StatefulSet......Page 240
Running a Cassandra cluster in Kubernetes......Page 241
Quick introduction to Cassandra......Page 242
The Cassandra Docker image......Page 243
Exploring the run.sh script......Page 244
Digging into the Cassandra configuration......Page 249
The custom seed provider......Page 250
Creating a Cassandra headless service......Page 251
Dissecting the stateful set configuration file......Page 252
Dissecting the replication controller configuration file......Page 256
Assigning pods to nodes......Page 259
Using DaemonSet to distribute Cassandra......Page 260
Summary......Page 261
Horizontal pod autoscaling......Page 262
Declaring horizontal pod autoscaler......Page 263
Custom metrics......Page 265
Autoscaling with kubectl......Page 266
Performing rolling updates with autoscaling......Page 269
Handling scarce resources with limits and quotas......Page 270
Resource quota types......Page 271
Storage resource quota......Page 272
Object count quota......Page 273
Quota scopes......Page 274
Creating quotas......Page 275
Using limit ranges for default compute quotas......Page 279
Choosing your node types......Page 280
Trading off cost and response time......Page 281
Benefiting from elastic cloud resources......Page 282
Mind your cloud quotas......Page 283
Considering Hyper.sh (and AWS Fargate)......Page 284
Improving the performance and scalability of Kubernetes......Page 285
The pod life cycle event generator......Page 286
etcd3......Page 287
Measuring the performance and scalability of Kubernetes......Page 288
Measuring API responsiveness......Page 289
Measuring end-to-end pod startup time......Page 292
Testing Kubernetes at scale......Page 293
Comparing a Kubemark cluster to a real-world cluster......Page 294
Summary......Page 295
Chapter 10: Advanced Kubernetes Networking......Page 296
Inter-pod communication (pod to pod)......Page 297
External access......Page 298
Kubernetes networking versus Docker networking......Page 299
Lookup and discovery......Page 300
Services and endpoints......Page 301
Loosely coupled connectivity with data stores......Page 302
Network namespaces......Page 303
Maximum transmission unit......Page 304
Requirements......Page 305
Container Networking Interface (CNI)......Page 306
CNI plugin......Page 307
Contiv......Page 310
Open vSwitch......Page 311
Canal......Page 313
Flannel......Page 314
Romana......Page 316
Weave net......Page 317
Configuring network policies......Page 318
Implementing network policies......Page 319
External load balancer......Page 320
Configuring an external load balancer......Page 321
Finding the load balancer IP addresses......Page 322
Specifying original client IP address preservation......Page 323
Service load balancer......Page 324
Ingress......Page 325
HAProxy......Page 326
Running HAProxy Inside the Kubernetes cluster......Page 327
Træfic......Page 328
First look at the loopback plugin......Page 329
Building on the CNI plugin skeleton......Page 331
Reviewing the bridge plugin......Page 332
Summary......Page 334
Understanding cluster federation......Page 335
Important use cases for cluster federation......Page 336
Sensitive workloads......Page 337
The federation control plane......Page 338
Federation API server......Page 339
Federated ConfigMap......Page 340
Deleting a federated ConfigMap......Page 341
Federated Events......Page 342
Federated Horizontal Pod Scaling......Page 343
Federated ingress......Page 344
Handling failures with federated ingress......Page 345
Federated ReplicaSet......Page 346
The hard parts......Page 347
Strictly-coupled......Page 348
Uniformly-spread......Page 349
Federated auto-scaling......Page 350
Setting up cluster federation from the ground up......Page 351
Using the official Hyperkube image......Page 352
Registering Kubernetes clusters with the federation......Page 353
Updating KubeDNS......Page 354
Getting Kubefed......Page 355
Deploying a federation control plane......Page 356
Naming rules and customization......Page 357
Shutting down the federation......Page 358
Load balancing across multiple clusters......Page 359
Failing over across multiple clusters......Page 360
Federated migration......Page 361
Creating a federated service......Page 362
Adding backend pods......Page 363
Verifying public DNS records......Page 364
Discovering a federated service......Page 365
DNS expansion......Page 366
Handling failures of backend pods and whole clusters......Page 367
Federated service is created successfully but no service is created in the underlying clusters......Page 368
Summary......Page 369
Working with the Kubernetes API......Page 370
Exploring the Kubernetes API directly......Page 371
Using Postman to explore the Kubernetes API......Page 373
Filtering the output with httpie and jq......Page 374
Creating a pod via the Kubernetes API......Page 375
Accessing the Kubernetes API via the Python client......Page 376
Dissecting the CoreV1API group......Page 377
Listing objects......Page 378
Creating objects......Page 379
Invoking Kubectl programmatically......Page 380
Using Python subprocess to run Kubectl......Page 381
Extending the Kubernetes API......Page 382
Understanding the structure of a custom resource......Page 383
Developing custom resource definitions......Page 384
Integrating custom resources......Page 385
Finalizing custom resources......Page 386
Understanding API server aggregation......Page 387
Utilizing the service catalog......Page 388
Writing a custom scheduler plugin......Page 389
The scheduler......Page 390
Configuring the scheduler......Page 391
Packaging the scheduler......Page 392
Deploying the custom scheduler......Page 393
Assigning pods to the custom scheduler......Page 394
Employing access control webhooks......Page 395
Using an authentication webhook......Page 396
Using an authorization webhook......Page 398
Using an admission control webhook......Page 399
Providing custom metrics for horizontal pod autoscaling......Page 400
Extending Kubernetes with custom storage......Page 401
Benefitting from CSI......Page 402
Summary......Page 403
Understanding Helm......Page 404
Helm components......Page 405
Installing Helm......Page 406
Installing Tiller in-cluster......Page 407
Using Alternative Storage Backend......Page 408
Finding charts......Page 409
Installing packages......Page 411
Customizing a chart......Page 413
Upgrading and rolling back a release......Page 415
Deleting a release......Page 416
Working with repositories......Page 417
Taking advantage of starter packs......Page 418
The Chart.yaml file......Page 419
Deprecating charts......Page 420
Managing dependencies with requirements.yaml......Page 421
Using special fields in requirements.yaml......Page 422
Writing template files......Page 424
Using pipelines and functions......Page 425
Feeding values from a file......Page 426
Scope, dependencies, and values......Page 427
Summary......Page 428
The road ahead......Page 429
Kubernetes releases and milestones......Page 430
The value of bundling......Page 431
Cloud platforms......Page 432
Azure......Page 433
GitHub......Page 434
Mindshare......Page 435
OpenStack......Page 436
Education and training......Page 437
Modularization and out-of-tree plugins......Page 438
Service meshes......Page 439
Summary......Page 440
Other Books You May Enjoy......Page 441
Index......Page 444
