Jenkins 2: Up and Running Evolve Your Deployment Pipeline for Next Generation Automation

Foreword......Page 7
How to Use This Book......Page 8
Conventions Used in This Book......Page 9
Using Code Examples......Page 10
How to Contact Us......Page 11
Acknowledgments......Page 12
1. Introducing Jenkins 2......Page 15
The Jenkinsfile......Page 16
Declarative Pipelines......Page 19
Blue Ocean Interface......Page 20
New Job Types in Jenkins 2......Page 21
Pipeline......Page 23
Reasons for the Shift......Page 24
Configurability......Page 25
Pipeline Source Management......Page 26
Meeting the Challenges......Page 27
Pipeline Compatibility......Page 28
Plugin Compatibility......Page 29
Providing scriptable APIs......Page 30
Checking Compatibility......Page 35
Summary......Page 37
Syntax: Scripted Pipelines Versus Declarative Pipelines......Page 38
Choosing Between Scripted and Declarative Syntax......Page 40
Systems: Masters, Nodes, Agents, and Executors......Page 41
Agent......Page 42
Executor......Page 43
Creating Nodes......Page 44
A quick note about node labels......Page 46
Structure: Working with the Jenkins DSL......Page 47
node......Page 48
steps......Page 50
Understanding step syntax......Page 51
Supporting Environment: Developing a Pipeline Script......Page 52
Starting a Pipeline Project......Page 53
The Editor......Page 54
Working with the Snippet Generator......Page 56
Running a Pipeline......Page 61
Viewing logs......Page 64
Stage View with errors......Page 66
Replay......Page 68
Summary......Page 72
Triggering Jobs......Page 73
Build Periodically......Page 74
Cron syntax......Page 75
Poll SCM......Page 77
input......Page 78
Boolean......Page 82
Choice......Page 83
Credentials......Page 84
List Subversion tags......Page 86
Run......Page 87
Return Values from Multiple Input Parameters......Page 88
Using the parameters section......Page 89
Using the Jenkins application to parameterize the build......Page 91
Using a script block......Page 92
Using external code......Page 93
Flow Control Options......Page 94
timeout......Page 95
waitUntil......Page 97
Locking Resources with the lock Step......Page 99
Controlling Concurrent Builds with Milestones......Page 101
Running Tasks in Parallel......Page 103
Traditional parallel syntax......Page 104
stash and unstash......Page 106
Alternative parallel syntax for Declarative Pipelines......Page 109
parallel and failFast......Page 110
Conditional Execution......Page 113
Post-Processing......Page 114
try-catch-finally......Page 115
catchError......Page 116
Declarative Pipelines and Post-Processing......Page 117
Summary......Page 118
Email......Page 119
Jenkins Location......Page 120
E-mail Notification......Page 121
Sending email in pipelines......Page 122
Extended email notifications......Page 123
Global configuration......Page 124
Content......Page 125
Recipients......Page 126
Including logs......Page 128
Slack notifications......Page 130
Setup in Slack......Page 131
Global configuration in Jenkins......Page 132
Webhooks in Slack......Page 133
Sending Slack notifications in a job......Page 135
HipChat notifications......Page 137
Setup in HipChat for version 2 API use......Page 138
Global configuration in Jenkins......Page 139
Default notifications......Page 140
Sending HipChat notifications in a job......Page 141
Publishing HTML Reports......Page 142
Summary......Page 145
Securing Jenkins......Page 146
Enabling Security......Page 147
Unix user/group database......Page 148
Matrix-based security......Page 149
Other Global Security Settings......Page 150
CLI......Page 151
Plugin manager......Page 152
SSH server......Page 154
Credentials in Jenkins......Page 155
Credential Scopes......Page 156
Credential Providers......Page 157
Selecting Credential Providers......Page 158
Selecting Credential Types......Page 159
Creating and Managing Credentials......Page 160
Context Links......Page 163
Adding a New Domain and Credential......Page 164
Using the New Domain and Credential......Page 167
Advanced Credentials: Role-Based Access......Page 168
Basic Use......Page 169
Manage Roles......Page 170
Global role example......Page 171
Project example......Page 173
Slave role example......Page 174
Assign Roles......Page 175
Verifying the roles setup......Page 178
Role Strategy Macros......Page 180
Username and Password......Page 182
SSH Keys......Page 183
Controlling Script Security......Page 184
Script Checking......Page 185
Script Approval......Page 186
Groovy Sandboxing......Page 187
Using Jenkins Credentials with Vault......Page 189
Setup......Page 190
Authentication......Page 191
AppRole......Page 192
Using Vault in Jenkins......Page 193
Jenkins credentials for Vault......Page 194
Using Vault in a Pipeline......Page 195
Summary......Page 197
Trusted Versus Untrusted Libraries......Page 198
SSH access......Page 199
HTTP access......Page 201
External Libraries......Page 202
Modern SCM......Page 203
Legacy SCM......Page 204
Automatic Downloading of Libraries from Source Control......Page 205
The @Library annotation......Page 206
The library step......Page 207
Library Scope Within Jenkins Items......Page 208
Sample Library Routine......Page 209
src......Page 210
vars......Page 213
Automatic documentation references for global variables......Page 214
Using global variables like steps......Page 216
resources......Page 219
Using Third-Party Libraries......Page 221
Loading Code from an External SCM......Page 222
Replaying External Code and Libraries......Page 224
A Closer Look at Trusted Versus Untrusted Code......Page 227
Summary......Page 230
7. Declarative Pipelines......Page 231
Additional Assembly Required......Page 232
The Structure......Page 233
Section......Page 234
Conditionals......Page 235
The Building Blocks......Page 236
agent......Page 238
Labels and custom workspaces......Page 239
Agents and Docker......Page 240
Using the same node for Docker and non-Docker stages......Page 241
Credentials and environment variables......Page 242
tools......Page 243
options......Page 246
Options summary......Page 247
triggers......Page 249
Cron syntax......Page 250
parameters......Page 251
Using parameters in a pipeline......Page 253
libraries......Page 254
steps......Page 255
Conditional execution of a stage......Page 256
Conditional execution with and, or, not......Page 257
post......Page 258
Dealing with Nondeclarative Code......Page 259
Place Code Outside of the Pipeline Block......Page 260
Using parallel in a Stage......Page 261
Script Checking and Error Reporting......Page 262
Summary......Page 265
General......Page 267
Discard old builds......Page 268
Specifying the GitHub project property in pipeline projects......Page 269
Disabling Pipeline projects......Page 270
Quiet period......Page 271
Block build when upstream project is building......Page 272
Keep the build logs of dependencies......Page 273
Revision......Page 274
Build Triggers......Page 275
Remotely triggering pipeline builds......Page 276
GitHub hook trigger for Git polling......Page 277
GitHub triggering in a pipeline......Page 283
Provide configuration files......Page 284
Managing configuration files in a pipeline......Page 288
Abort the build if it’s stuck......Page 290
Add timestamps to console output......Page 292
Use secret text(s) or files(s)......Page 293
Post-Build Actions......Page 294
Freestyle Projects......Page 295
The Maven Project Type......Page 296
The Pipeline Project Type......Page 299
The External Job Project Type......Page 301
The Multiconfiguration Project Type......Page 304
Multiconfiguration example......Page 305
Ivy Projects......Page 310
Folders......Page 313
Creating a folder......Page 315
Moving existing items into a folder......Page 317
Multibranch Pipeline Projects......Page 318
Configuration......Page 319
Branch indexing......Page 320
Incorporating new branches......Page 321
GitHub Organization Projects......Page 323
Creating a GitHub Organization project......Page 324
Webhooks......Page 325
Bitbucket Team/Project Projects......Page 328
Summary......Page 331
9. The Blue Ocean Interface......Page 333
The Dashboard......Page 334
Simple pipeline Activity view......Page 339
Multibranch Pipeline Activity view......Page 342
Multibranch Pipeline Branches view......Page 344
Multibranch Pipeline Pull Requests view......Page 345
Pipeline......Page 351
Step logs......Page 353
Changes......Page 354
Tests......Page 357
Artifacts......Page 358
Creating a New Pipeline Without an Existing Jenkinsfile......Page 361
Working in the Editor......Page 367
Specifying global parts of the pipeline......Page 368
Adding a new stage......Page 372
Adding a step to a stage......Page 374
Saving and committing pipeline changes......Page 377
Editing an Existing Pipeline......Page 381
Importing and Editing Existing Pipelines......Page 385
Simple debugging and editing of an existing pipeline......Page 386
Debugging editor issues......Page 390
Adding code not supported in the editor......Page 395
Working with Pipelines from Non-GitHub Repositories......Page 397
Summary......Page 400
10. Conversions......Page 401
Access......Page 402
Shared Libraries......Page 403
Converting a Freestyle Pipeline to a Scripted Pipeline......Page 404
Source......Page 409
Compile......Page 414
Unit Tests......Page 419
Distributing content across nodes......Page 420
Cleaning out workspaces......Page 422
Integration Testing......Page 423
Source code analysis......Page 426
Setting version information with parameters......Page 427
Using containers in a pipeline......Page 430
Converting from a Jenkins Pipeline Project to a Jenkinsfile......Page 432
Approach......Page 435
Migrating parameter usage to Jenkinsfiles......Page 438
Final Steps......Page 441
Including a Jenkinsfile back into a native Jenkins project......Page 442
Converting from a Scripted Pipeline to a Declarative Pipeline......Page 443
Sample Pipeline......Page 444
The Conversion......Page 445
Adding the libraries directive......Page 446
Steps......Page 447
Post-build processing......Page 448
Completed Conversion......Page 449
General Guidance for Conversions......Page 450
Summary......Page 452
Using Shell Steps......Page 453
The sh Step......Page 454
set Options......Page 455
Executing shell scripts from shared libraries......Page 457
Executing batch scripts from shared libraries......Page 459
The powershell Step......Page 460
Working with Environment Variables......Page 461
The withEnv Step......Page 463
Working with Workspaces......Page 464
Creating a Custom Workspace......Page 465
Cleaning a Workspace......Page 466
Using other delete programs......Page 467
File and Directory Steps......Page 468
Checking for file existence......Page 469
pwd......Page 470
Doing More with Files and Directories......Page 471
Summary......Page 472
SonarQube Survey......Page 474
Working with Individual Rules......Page 475
Quality Gates and Profiles......Page 482
The Scanner......Page 483
Global Configuration......Page 484
Using SonarQube in a Freestyle Project......Page 485
Using SonarQube in a Pipeline Project......Page 486
Setting up the SonarQube webhook......Page 487
Processing the SonarQube webhook in the Jenkins DSL......Page 488
SonarQube Integration Output with Jenkins......Page 491
About JaCoCo......Page 492
Integrating JaCoCo with the Pipeline......Page 493
JaCoCo Output Integration with Jenkins......Page 495
Summary......Page 496
Publishing and Retrieving Artifacts......Page 498
Setup and Global Configuration......Page 499
Using Artifactory in a Scripted Pipeline......Page 500
Downloading Specific Files to Specific Locations......Page 505
Build Promotion......Page 506
Integration with a Declarative Pipeline......Page 507
Artifactory Integration with Jenkins Output......Page 508
Archiving Artifacts and Fingerprinting......Page 509
Summary......Page 515
Configured as a Cloud......Page 517
Global Configuration......Page 518
Using Docker Images as Agents......Page 521
Setting up a Docker cloud template—basic options......Page 523
Container settings......Page 524
Using Cloud Images in a Pipeline......Page 525
Troubleshooting......Page 527
Agent Created on the Fly for a Declarative Pipeline......Page 530
Docker Pipeline Global Variable......Page 532
Global Variables......Page 533
Docker Application Global Variable Methods......Page 534
withServer......Page 535
withRegistry......Page 536
withTool......Page 537
Docker Image Global Variable Methods......Page 540
The inside method......Page 543
Docker Container Global Variable Methods......Page 544
Running Docker via the Shell......Page 545
Summary......Page 546
15. Other Interfaces......Page 548
Using the Direct SSH Interface......Page 549
Using the CLI Client......Page 552
HTTP mode......Page 553
Remoting mode......Page 554
Filtering Results......Page 555
Initiating Builds......Page 557
Obtaining crumbs......Page 558
Using the Script Console......Page 559
Summary......Page 562
Diving into Pipeline Steps......Page 563
Serializing Pipelines......Page 567
NotSerializableException......Page 568
Handling Nonserializable Errors......Page 569
Identifying the Line in Your Script that Caused an Error......Page 571
Using Nondeclarative Code Within a Declarative Pipeline......Page 572
Unapproved Code (Script and Method Approval)......Page 576
Unsupported Operations......Page 577
System Logs......Page 578
Timestamps......Page 580
Pipeline Durability Settings......Page 581
Summary......Page 583
Index......Page 584