
The MightywombleJan 30, 2018 · 5 min read
The following document is designed to explain what Jenkins Pipelining is, and provide examples of a Jenkins pipeline which runs a basic script on a Jenkins slave.
I spent a while searching around google and couldn’t find a good simple concise guide, so I help this helps someone get up and going.
The Management platform run a more advanced version of this to create machines using Terraform.
- Step-by-step guide
- What is Jenkins
- What is a Jenkins Pipeline
- The Jenkinsfile
- Pipeline
- Stages
- Note about brackets
- The Jenkinsfile in a git/Bitbucket project
- The Pipeline plugin
- Creating a new pipeline
- Running a Pipeline build
- Error checking
- Related articles
Step-by-step guide
What is Jenkins
Jenkins is an open source automation server written in Java. Jenkins helps to automate the non-human part of software development process, with continuous integration and facilitating technical aspects of continuous delivery. It is a server-based system that runs in servlet containers such as Apache Tomcat. It can execute Apache Ant, Apache Maven and sbt based projects as well as arbitrary shell scripts and Windows batch commands.
What is a Jenkins Pipeline
According to Jenkins it’s..
Jenkins Pipeline (or simply “Pipeline” with a capital “P”) is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.
A continuous delivery pipeline is an automated expression of your process for getting software from version control right through to your users and customers. Every change to your software (committed in source control) goes through a complex process on its way to being released. This process involves building the software in a reliable and repeatable manner, as well as the progression of the built software (called a “build”) through multiple stages of testing and deployment.
In real English,
A pipeline is a script which gives Jenkins a series of jobs to do in a pipeline like manner (one after the other) these jobs can be as Jenkins code, scripts, build languages etc.
The Jenkinsfile
The core of the pipeline is the file called Jenkinsfile. It’s the list of jobs which the pipeline will perform and can be held on the Jenkins server itself as part of the pipeline or at the root of a linked git/Bitbucket repository
An example Jenkinsfile looks like this:
pipeline {
environment {
BUILD_SCRIPTS_GIT="http://10.100.100.10:7990/scm/~myname/mypipeline.git"
BUILD_SCRIPTS='mypipeline'
BUILD_HOME='/var/lib/jenkins/workspace'
}
agent any
stages {
stage('Checkout: Code') {
steps {
sh "mkdir -p $WORKSPACE/repo;
git config --global user.email 'email@address.com';
git config --global user.name 'myname';
git config --global push.default simple;
git clone $BUILD_SCRIPTS_GIT repo/$BUILD_SCRIPTS"
sh "chmod -R +x $WORKSPACE/repo/$BUILD_SCRIPTS"
}
}
stage('Yum: Updates') {
steps {
sh "sudo chmod +x $WORKSPACE/repo/$BUILD_SCRIPTS/scripts/update.sh"
sh "sudo $WORKSPACE/repo/$BUILD_SCRIPTS/scripts/update.sh"
}
}
}
post {
always {
cleanWs()
}
}
}
This pipeline file
- sets up environment variables
- pulls data down from a git repo
- sets it up in a Jenkins workspace
- runs a script under scripts/
- once completes by cleaning up the workspace (successful or not)
In this example we break this information down to the following chunks with an explanation
Pipeline
We define the pipeline using the pipeline{} section
Our first stage is an option but recommended one where we setup environment variables using an environment{} section
In this example I’m not restricting the agent the pipeline can run on, so I’ve included agent any
pipeline {
environment {
BUILD_SCRIPTS_GIT="http://10.100.100.10:7990/scm/~myhome/mypipeline.git"
BUILD_SCRIPTS='mypipeline'
BUILD_HOME='/var/lib/jenkins/workspace'
}
agent any
Stages
Checkout: Code
The core of pipelines are stages{} and steps{}, stages can have multiple steps in them
In this example our first stage is called Checkout: code, this label will also be shown as we will see later in the Jenkins interface so making it relevent is useful
Our steps are to create a workspace on the Jenkins slave and then setup and pull down the data from the Jenkins repo we linked the pipeline to when we set it up (see later)
stage('Checkout: Code') {
steps {
sh "mkdir -p $WORKSPACE/repo;
git config --global user.email 'email@address.com';
git config --global user.name 'myname';
git config --global push.default simple;
git clone $BUILD_SCRIPTS_GIT repo/$BUILD_SCRIPTS"
sh "chmod -R +x $WORKSPACE/repo/$BUILD_SCRIPTS"
}
}
Yum: updates
Our second stage is an example of using the data pulled down from the git repo, which includes a script (in this instance it just runs yum install -y wget nano nmap traceroute)
You can see we have to do some linux maintenance by making the script executable and running as sudo, (I’ve got a user setup on the Jenkins client with sudo nopasswd for this example)
stage('Yum: Updates') {
steps {
sh "sudo chmod +x $WORKSPACE/repo/$BUILD_SCRIPTS/scripts/update.sh"
sh "sudo $WORKSPACE/repo/$BUILD_SCRIPTS/scripts/update.sh"
}
}
Post
The post section comes after the stages and will execute even if the stages failed. In this example I’ve used it to clean up the workspace folder; if you don’t do this Jenkins will fail the next time the pipeline is run
post {
always {
cleanWs()
}
}
Note about brackets
Every open bracket { must have a close } and vice versa, if there are too many of either bracket then the pipeline will fail with various errors.
The Jenkinsfile in a git/Bitbucket project
The above Jenkinsfile along with the scripts/update.sh folder are held in a git/Bitbucket repository. The Jenkinsfile should be held in the root of the repository; if it is it will be automatically picked up when you setup the git repo in the Jenkins pipeline.
The Pipeline plugin
The pipeline needs to be added to Jenkins for this to work. This isn’t always added by default and can be added via the Manage Jenkins interface.
Creating a new pipeline
With our Jenkinsfile and script in git/Bitbucket the pipeline needs to be setup in Jenkins. We do this using the Jenkins server web interface.
Login to Jenkins
Click on New Job
We need to create a pipeline, so we
- Give the pipeline a name
- Select pipeline in the list
- Click on OK
Next we scroll down to Advanced Project Options.
- Select — Pipeline script from SCM (if you don’t see this option on your Jenkins server see the plugins note below)
- Add your git/Bitbucket repo link
- Add the credentials for connecting to your git repo
Note: because we have put the Jenkinsfile in the root of the Git repo path, it will be picked up and used by Jenkins automatically.
Note Additional Jenkins plugins
On a default installation of Jenkins most of the plugins you needed are installed, however to check we should have at least the following plugins:
Manage Jenkins -> Manage Plugins
- Bitbucket Plugin
- Pipeline
- Git
- Pipeline SCM Step
Running a Pipeline build
Once the pipeline has been created we can build the plugin. On the first build you will see a screen which can see a breakdown of the pipeline
Note that the heading we used in our stages of the above Jenkinsfile are now displayed within the pipeline display of Jenkins
As each stage of the pipeline runs through, the Jenkins interface will display a simple Red failed, Green worked status for the stage.
Error checking
If the pipeline fails (or even works in the interface but has unexpected results) the Jenkins interface provides logs for each stage of the pipeline
Clicking on the Logfile will explain what happened during the run of that stage
These instructions were written using a docker image of Bitbucket as the git repo and of Jenkins.
Related articles
https://jenkins.io/solutions/pipeline/
https://jenkins.io/doc/book/pipeline/jenkinsfile/
https://wiki.jenkins.io/display/JENKINS/Build+Pipeline+Plugin
Jenkins — Pipeline (Beginners guide)