Scalable Jenkins on Kubernetes Cluster & Pipeline for Automated CI CD (Jenkins - Github Integration)
Setting up scalable Jenkins on Kubernetes can be a daunting task. This article demonstrates not only How to setup Scalable Jenkins on Kubernetes but also how to integrate Jenkins into Github so that whenever any Pull Request is requested on a branch, build is automatically triggered and results are made available to the branch admin to review before merging the Pull Request.
Let us first get the scalable Jenkins up before setting up a pipeline for automated CI CD.
Before we go through the steps, here is the brief on the Master and Slave node docker images
Jenkins Master Node
Jenkins master uses the image
This image has many plugins required for smooth CD on Kubernetes. To know the image content, Dockerfile (./master/Dockerfile) is kept here. Following plugins are made available as part of this image to save time:
Jenkins Slave Node
Jenkins slave uses the image
This image is based on openjdk11 containing maven, docker runtime so that spring boot project can be checked out, maven built including docker image building & deployment.
Steps - Scalable Jenkins on Kubernetes Cluster
Below is the step-by-step guide to making it happen.
Launch Jenkins master
Run below Kubernetes configurations
Secure the master
Once the Jenkins master server is deployed, it would take few minutes to get the UI up and running To access UI, run below
After login, come back to this URL again and select tickmark to Agent – Controller Security. This option gets visible after setting up a username and password.
Configure the Kubernetes plugin as shown in the snapshots below
Setup Global credentials
To test the docker commands especially login, first set up the global credentials as shown in below
Create Build Job
Create docker-test job as Pipeline, click OK and insert below text in the pipeline block
Test the pipeline
Run the docker-test job and you would see that a pod will be launched by Jenkins master to run the docker-test build job and the pod will be terminated immediately build completes.
Once build job completes, then build pod is terminated
Steps - Jenkins Pipeline for Automated CI CD
To illustrate, automated CI CD, I have set up an example GitHub project Weather Service Also created a branch named Jenkins and whenever any contributions are made to this branch via pull request, branch/project admin can review the pull request, view the build, and test status before merging into Jenkins branch
Also, install below plugin
- GitHub pull request builder
Now generate an access token using https://github.com/settings/tokens/new (select repo:status)
Create a webhook using your Github repo settings
- Insert https:///ghprbhook/
- Event trigger – Select individual events (Pull requests, Issue comments) So that whenever any pull request is created, Github can notify your Jenkins URL
Scroll down and configure Github Pull Request Builder and choose the right credentials
Create a new job say weather-service, select pipeline, and click OK
Select Github Pull Request Builder, see the below snapshot
Click on advanced and insert whitelist target branch say Jenkins. You should also select Build every pull request automatically without asking to ensure that every pull request is good enough in terms of build and test quality before merging
Test the setup
Now, you can raise a pull request on the selected branch (Jenkins in my case) and you would notice that the build gets triggered automatically and its results will be shown on the Pull request page so that the branch admin can review the outcome and decide on the merge.
To show the build status on your main Github page, install the plugin
embeddable-build-status and goto your build job and click on this plugin link to generate URL that would look similar to the below
Kubernetes Cluster Info
Below is the info on the tool used to setup the Kubernetes cluster and its version
- Kubernetes Server 1.19.7
- Cluster Management Tool – kops
- Cloud Provider AWS
This article demonstrated How to setup Jenkins Pipeline to achieve automated CI CD including Github and Jenkins integration. The moment Git Pull request is raised on a particular branch, build is triggered and results are made avaialble to the branch admin to review before merging the pull request.
The author, Rajesh G, is The Chief Architect @ Brain Upgrade Academy where he has designed the IoT-based Fleet Management Platform that runs on a Kubernetes Cluster on AWS Amazon. He is also a certified Kubernetes Administrator and TOGAF certified Enterprise Architect. Rajesh led various digital transformation initiatives for Fortune 500 Fin Tech companies. Over the last 20+ years, he has been part of many successful technology startups.
About Brain Upgrade Academy
We, at Brain Upgrade, offer Kubernetes Consulting services to our clients including Up Skilling (training) of clients teams thus facilitate efficient utilization of Kubernetes Platform. To know more on the Kubernetes please visit www.brainupgrade.in/blog and register on www.brainupgrade.in/enroll to equip yourself with Kubernetes skills.
Why Brain Upgrade
We at Brain Upgrade, partner with our customers in the digital transformation of their businesses by providing:
Technology Consulting in product development, IoT, DevOps, Cloud, Containerization, Big Data Analysis with a heavy focus on Open source technologies. Training the IT workforce on the latest cloud technologies such as Kubernetes, Docker, AI, ML, etc.
You may want to register for the upcoming trainings on https://brainupgrade.in/enroll