Prerequisites:
1. Source Code Management (SCM) :
Tool: GitHub, GitLab, Bitbucket
- A repository with the application code.
- Branching strategy (e.g., GitFlow, trunk-based development).
- Access control configured (e.g., using CODEOWNERS).
- Webhooks enabled for CI/CD triggers.
2. Continuous Integration (CI) :
Tool: Jenkins, GitHub Actions, GitLab CI/CD
- Installed and configured Jenkins or equivalent CI server.• Required plugins installed (e.g., Git, Docker, Slack, SonarQube,
Anchore). - Pipeline scripts (e.g., Jenkinsfile) in the repositories.
- Credentials in Jenkins (e.g., DockerHub, SonarQube, Slack, Kubernetes).
- Automated build and test scripts (e.g., Maven, Gradle, npm).
- Static code analysis tools integrated (e.g., SonarQube).
3. Containerization :
Tool: Docker
- Docker installed and configured on build servers.
- Dockerfile created and tested for the application.
- Docker Hub or private Docker registry account ready.
- Permissions for image push and pull configured.
4. Continuous Delivery (CD) :
Tool: Helm, Kubernetes
- Kubernetes cluster set up and accessible.
- Helm installed and Tiller configured (if applicable).
- Kubernetes namespaces created for each application environment.
- Helm charts created for each application.
- ImagePullSecrets configured in Kubernetes.
5. Notifications :
Tool: Slack, Email
- Notification channels configured.
- Integrations added in CI/CD tools (e.g., Slack token in Jenkins).
6. Security and Compliance :
Tool: Anchore, Vault
- Vault installed and initialized for secret management.
- Vault integrated with Jenkins/Kubernetes for secret retrieval.
- Anchore set up for container image scanning.
- Quality gates defined for code and security.
7. Documentation :
- README files for repositories.
- Configuration files and guides for developers and operators.
End to End CI/CD Deployment using Following steps:
a) SonarQube:
SonarQube is an open-source platform for continuous inspection of code
quality. It is widely used in DevOps pipelines to improve code health and
maintainability by analyzing source code for potential bugs, vulnerabilities, and
code smells.
Key Features of SonarQube:
i) Code Analysis:
ii) Bug Detection:
iii) Security Analysis:
iv) Quality Gates
b) Dependency track:
Dependency-Track is an open-source platform designed to help
organizations manage and mitigate the risks associated with using third-party
and open-source software components. Here’s a breakdown of its primary uses
and benefits:
firstly create the API key in dependency track UI and add this in jenkins credentials
I) Generate BOM :
Dependency-Track, BOM stands for Bill of Materials. It refers to a detailed
inventory of all the dependencies, libraries, and components used in a softwareproject …. Dependency-Track uses the BOM to analyze and manage the security and
licensing of the components in your software supply chain.
Tools like CycloneDX plugins for Maven, Gradle, npm, or Yarn can generate a
BOM for your project.
Run the plugin to generate a BOM file in project Directory
mvn cyclonedx:makeAggregateBom
II) Upload this BOM.xml file in Dependency track:
Dependency-Track relies on CycloneDX BOMs. Use tools like Maven, Gradle, or CycloneDX CLI to generate BOM files for your projects. For non-standard environments, you can use cURL to upload the BOM directly via API.
stages of dependency track in jenkins pipeline is
stage('Generate BOM') {
steps {
dir('testhello') {
// Generate BOM file using CycloneDX Maven plugin
sh 'mvn org.cyclonedx:cyclonedx-maven-plugin:makeAggregateBom'
}
}
}
stage('Download Dependency-Track CLI') {
steps {
sh 'curl -L -o dependency-track-cli.jar https://example.com/path/to/dependency-track-cli.jar'
}
}
stage('Upload BOM to Dependency-Track') {
steps {
sh ''' curl -X POST "${DEP_TRACK_SERVER_URL}/api/v1/bom" \
-H "X-Api-Key: ${DEP_TRACK_API_KEY}" \
-H "Content-Type: multipart/form-data" \
-F "project=${DEP_TRACK_PROJECT_ID}" \
-F "bom=@/home/ubuntu/Desktop/testhello/target/bom.xml"
'''
}
}
c) Anchore:
Anchore is an open-source container security platform designed to analyze
and manage the security and compliance of container images. It integrates
seamlessly into DevOps workflows, enabling developers and security teams to
identify vulnerabilities, ensure compliance, and enforce security policies for
containerized applications.
Key Features of Anchore :
- Container Image Analysis:
- Policy Enforcement:
- Vulnerability Management:
- Compliance Checks:
refer the follows:
stages of Anchore scanning in Jenkins is
// Install Anchore CLI if not already installed
stage('Install Anchore CLI') {
steps {
sh '''
if ! command -v anchore-cli > /dev/null; then
pip install --user anchorecli
fi
'''
}
}
// Set Anchore CLI Path
stage('Set Anchore CLI Path') {
steps {
sh '''
export PATH=$PATH:/home/ubuntu/.local/bin
echo "Anchore CLI Path set to: $PATH"
'''
}
}
stage('Analyze Image with Anchore') {
steps {
withCredentials([usernamePassword(credentialsId: 'anchor_id', usernameVariable: 'ANCHORE_USER', passwordVariable: 'ANCHORE_PASS')]) {
sh """
export PATH=$PATH:/home/ubuntu/.local/bin
anchore-cli --url ${ANCHORE_URL} --u ${ANCHORE_CREDENTIALS_USR} --p ${ANCHORE_CREDENTIALS_PSW} image add ${params.DOCKERHUB_USERNAME}/${params.JAVA_IMAGE_NAME}:${currentBuild.number}
anchore-cli --url ${ANCHORE_URL} --u ${ANCHORE_CREDENTIALS_USR} --p ${ANCHORE_CREDENTIALS_PSW} image wait ${params.DOCKERHUB_USERNAME}/${params.JAVA_IMAGE_NAME}:${currentBuild.number}
anchore-cli --url ${ANCHORE_URL} --u ${ANCHORE_CREDENTIALS_USR} --p ${ANCHORE_CREDENTIALS_PSW} image vuln ${params.DOCKERHUB_USERNAME}/${params.JAVA_IMAGE_NAME}:${currentBuild.number} all
"""
}
}
}
End to End CI/CD pipeline using SonarQube Anlysis, Dependenct_track, Anchore Scanning is as follows:
pipeline {
agent any
environment {
DOCKERHUB_CREDENTIALS = credentials('dockerhubpwd')
SLACK_CREDENTIALS = credentials('b3ee302b-e782-4d8e-ba83-7fa591d43205')
DEP_TRACK_API_KEY = credentials('d_track_id')
BOM_FILE_PATH = 'Desktop/testhello/target'
DEP_TRACK_SERVER_URL = 'http://localhost:8025'
DEP_TRACK_PROJECT_ID = '33d794ea-e8d3-49fd-b8f8-0c6cbce382cf'
MAVEN_HOME = 'Maven 3.8.4'
SONARQUBE_CREDENTIALS = credentials('sonar_d_token')
SONARQUBE_SERVER = 'http://localhost:9000'
ANCHORE_URL = 'http://localhost:8228'
ANCHORE_CREDENTIALS = credentials('anchor_id')
ANCHORE_CREDENTIALS_USR = 'admin'
ANCHORE_CREDENTIALS_PSW = 'foobar'
}
parameters {
string(name: 'JAVA_REPO', defaultValue: 'Your project repo URL', description: 'Java Application Repository')
string(name: 'DOCKERHUB_USERNAME', defaultValue: 'your Dockerhub username', description: 'DockerHub Username')
string(name: 'JAVA_IMAGE_NAME', defaultValue: 'testhello', description: 'Java Docker Image Name')
string(name: 'JAVA_NAMESPACE', defaultValue: 'test', description: 'Kubernetes Namespace for Java Application')
}
stages {
stage('Clone Repository') {
steps {
git url: params.JAVA_REPO, branch: 'main'
}
}
stage('SonarQube Analysis') {
steps {
dir('testhello') {
withSonarQubeEnv('SonarQube') {
sh '''
mvn sonar:sonar \
-Dsonar.projectKey=testhello \
-Dsonar.host.url=${SONARQUBE_SERVER} \
-Dsonar.login=${SONARQUBE_CREDENTIALS}
'''
}
}
}
}
stage('Build and Push Docker Image') {
steps {
dir('testhello') {
sh 'mvn clean install'
script {
def image = docker.build("${params.DOCKERHUB_USERNAME}/${params.JAVA_IMAGE_NAME}:${currentBuild.number}")
docker.withRegistry('', 'dockerhubpwd') {
image.push()
}
}
}
}
}
stage('Generate BOM') {
steps {
dir('testhello') {
sh 'mvn org.cyclonedx:cyclonedx-maven-plugin:makeAggregateBom'
}
}
}
stage('Upload BOM to Dependency-Track') {
steps {
sh '''
curl -X POST "${DEP_TRACK_SERVER_URL}/api/v1/bom" \
-H "X-Api-Key: ${DEP_TRACK_API_KEY}" \
-H "Content-Type: multipart/form-data" \
-F "project=${DEP_TRACK_PROJECT_ID}" \
-F "bom=@${BOM_FILE_PATH}/bom.xml"
'''
}
}
stage('Install Anchore CLI') {
steps {
sh '''
if ! command -v anchore-cli > /dev/null; then
pip install --user anchorecli
fi
'''
}
}
stage('Set Anchore CLI Path') {
steps {
sh '''
export PATH=$PATH:/home/ubuntu/.local/bin
echo "Anchore CLI Path set to: $PATH"
'''
}
}
stage('Analyze Image with Anchore') {
steps {
withCredentials([usernamePassword(credentialsId: 'anchor_id', usernameVariable: 'ANCHORE_USER', passwordVariable: 'ANCHORE_PASS')]) {
sh """
export PATH=$PATH:/home/ubuntu/.local/bin
anchore-cli --url ${ANCHORE_URL} --u ${ANCHORE_USER} --p ${ANCHORE_PASS} image add ${params.DOCKERHUB_USERNAME}/${params.JAVA_IMAGE_NAME}:${currentBuild.number}
anchore-cli --url ${ANCHORE_URL} --u ${ANCHORE_USER} --p ${ANCHORE_PASS} image wait ${params.DOCKERHUB_USERNAME}/${params.JAVA_IMAGE_NAME}:${currentBuild.number}
anchore-cli --url ${ANCHORE_URL} --u ${ANCHORE_USER} --p ${ANCHORE_PASS} image vuln ${params.DOCKERHUB_USERNAME}/${params.JAVA_IMAGE_NAME}:${currentBuild.number} all
"""
}
}
}
stage('Get Approval') {
steps {
script {
input message: 'Do you approve this deployment?', ok: 'Yes, deploy'
}
}
}
stage('Install yq') {
steps {
sh """
wget https://github.com/mikefarah/yq/releases/download/v4.6.1/yq_linux_amd64 -O "${WORKSPACE}/yq"
chmod +x "${WORKSPACE}/yq"
export PATH="${WORKSPACE}:$PATH"
"""
}
}
stage('Build and Package Java Helm Chart') {
steps {
dir('testhello') {
sh """
"${WORKSPACE}/yq" e -i '.image.tag = "latest"' ./myspringbootchart/values.yaml
helm template ./myspringbootchart
helm lint ./myspringbootchart
helm package ./myspringbootchart --version "1.0.0"
"""
}
}
}
stage('Deploy Java Application to Kubernetes') {
steps {
script {
kubernetesDeploy(
configs: 'Build and Deploy Java and Python Applications',
kubeconfigId: 'kubeconfig1pwd'
)
}
}
}
}
post {
always {
script {
def slackBaseUrl = 'https://slack.com/api/'
def slackChannel = '#builds'
def slackColor = currentBuild.currentResult == 'SUCCESS' ? 'good' : 'danger'
def slackMessage = "Build ${currentBuild.fullDisplayName} finished with status: ${currentBuild.currentResult}"
echo "Sending Slack notification to ${slackChannel} with message: ${slackMessage}"
slackSend(
baseUrl: 'https://yourteam.slack.com/api/',
teamDomain: 'StarAppleInfotech',
channel: '#builds',
color: slackColor,
botUser: true,
tokenCredentialId: SLACK_CREDENTIALS,
notifyCommitters: false,
message: "Build end to end pipeline #${env.BUILD_NUMBER} finished with status: ${currentBuild.currentResult}"
)
}
echo 'Pipeline completed.'
emailext(
to: 'your email@gmail.com',
subject: "Jenkins Build ${env.JOB_NAME} #${env.BUILD_NUMBER} ${currentBuild.currentResult}",
body: """<p>Build ${env.JOB_NAME} #${env.BUILD_NUMBER} finished with status: ${currentBuild.currentResult}</p>
<p>Check console output at ${env.BUILD_URL}</p>""",
mimeType: 'text/html'
)
}
failure {
emailext(
to: 'your email@gmail.com',
subject: "Jenkins Build ${env.JOB_NAME} #${env.BUILD_NUMBER} Failed",
body: """<p>Build ${env.JOB_NAME} #${env.BUILD_NUMBER} failed.</p>
<p>Check console output at ${env.BUILD_URL}</p>""",
mimeType: 'text/html'
)
}
success {
emailext(
to: 'your email@gmail.com',
subject: "Jenkins Build ${env.JOB_NAME} #${env.BUILD_NUMBER} Succeeded",
body: """<p>Build ${env.JOB_NAME} #${env.BUILD_NUMBER} succeeded.</p>
<p>Check console output at ${env.BUILD_URL}</p>""",
mimeType: 'text/html'
)
}
}
}
Build this,
Admin at DevOpsLover.com
Mahesh Wabale is the admin & author of this blog, Technology enthusiast with interest in DevOps, CICD, Spring Boot, Microservices Architecture He has more than 7 years of experience in IT industry, programming language and development stuff. He loves to share his experience with his blog.
Latest posts by Mahesh Wabale (see all)
- Dependency Track – End To End CI/CD Pipeline - November 29, 2024
- Dependency-track Jenkins Integration - November 27, 2024
- Jenkins Setup for PyTest + Selenium Automation Testing - November 27, 2024