# Jenkins Pipeline Analysis Report This report summarizes the pipeline logic extracted from the provided Jenkins XML configuration files (.xml format). The Jenkins setup utilizes a combination of Pipeline jobs (Groovy scripts) for orchestration and parameterized Freestyle jobs for specific build/deploy steps. ## Key Focus: The common Folder Copy Mechanism The requirement to copy the common folder to every deployed subdirectory is implemented not in the main Groovy pipeline, but within the shell script of the dependent `lambda-bulk-asset-ingester-docker-to-ecr` job. This step is critical as it makes shared utility code available in the build context of each individual Lambda function before its Docker image is built. ### Source: docker-to-ecr.xml (Shell Builder) The exact shell commands used to handle the copy are: ```bash if [ -d "./common/" ]; then echo "Copying ./common to ./lambda/$LAMBDA_DIR" cp -Rf ./common ./lambda/$LAMBDA_DIR fi ls -al ./lambda/$LAMBDA_DIR/common ``` ## Extracted Groovy Pipeline Scripts The following are the full Groovy scripts defining the pipeline stages for the main CI/CD flow and the triggering mechanism. ### 1. Main CD Pipeline Script (pipeline.xml) This script defines the multi-stage Continuous Delivery process for a single Lambda function, taking the Git commit and Lambda directory as parameters. ```groovy node { stage('Get Git Commit') { checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '577cbc72-7d9e-4eba-924c-ecbbe6de9805', url: 'git@github.com:theorchard/lambda-bulk-asset-ingester.git']]]) GIT_COMMIT = sh(returnStdout: true, script: 'git rev-parse HEAD').trim() script{ currentBuild.description = LAMBDA_DIR } } stage ('Unit Tests & Lint'){ build job: 'lambda-bulk-asset-ingester-unit-lint', parameters: [string(name: 'GIT_COMMIT', value: GIT_COMMIT), string(name: 'LAMBDA_DIR', value: LAMBDA_DIR)] } stage ('Sonar Scan and Analysis'){ build job: 'lambda-bulk-asset-ingester-sonar-scan', parameters: [string(name: 'GIT_COMMIT', value: GIT_COMMIT), string(name: 'LAMBDA_DIR', value: LAMBDA_DIR)] } stage ('Create a release'){ // This sub-job contains the 'common' folder copy logic build job: 'lambda-bulk-asset-ingester-docker-to-ecr', parameters: [string(name: 'GIT_COMMIT', value: GIT_COMMIT), string(name: 'LAMBDA_DIR', value: LAMBDA_DIR)] } stage ('Deploy to QA'){ build job: 'lambda-bulk-asset-ingester-container-deploy', parameters: [string(name: 'GIT_COMMIT', value: GIT_COMMIT), string(name: 'LAMBDA_DIR', value: LAMBDA_DIR), string(name: 'ENV', value: 'qa')] } stage('Integration Tests'){ build job: 'lambda-bulk-asset-ingester-integration-tests', parameters: [string(name: 'GIT_COMMIT', value: GIT_COMMIT), string(name: 'LAMBDA_DIR', value: LAMBDA_DIR)] } stage('Deploy to Prod'){ script { if(params.DEPLOY_PROD.toBoolean()) { echo 'deploying prod' echo DEPLOY_PROD build job: 'lambda-bulk-asset-ingester-container-deploy', parameters: [string(name: 'GIT_COMMIT', value: GIT_COMMIT), string(name: 'LAMBDA_DIR', value: LAMBDA_DIR), string(name: 'ENV', value: 'prod')] } } } } ``` ### 2. Pipeline Trigger Script (pipeline-trigger.xml) This script detects changes between the current commit and HEAD~1 on master, identifies which Lambda subdirectories were modified, and non-blockingly triggers the main pipeline for each of them. ```groovy node() { stage('Get Git Commit') { checkout([$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '577cbc72-7d9e-4eba-924c-ecbbe6de9805', url: 'git@github.com:theorchard/lambda-bulk-asset-ingester.git']]]) GIT_COMMIT = sh(returnStdout: true, script: 'git rev-parse HEAD').trim() MASTER_COMMIT = sh(returnStdout: true, script: 'git rev-parse HEAD~1').trim() DIFF_FILES = sh(returnStdout: true, script: "git diff --name-only --diff-filter=ACMR ${MASTER_COMMIT} ${GIT_COMMIT}") script{ LAMBDA_DIRS = [] for (item in DIFF_FILES.split('\n')){ if (item.contains('lambda/')){ lambda_dir = item.split('/')[1] if (!LAMBDA_DIRS.contains(lambda_dir)){ LAMBDA_DIRS.add(lambda_dir) } } } } } stage('Trigger Lambda Pipelines') { script{ // pipelines are non-blocking by default, can spawn multiple for (LAMBDA_DIR in LAMBDA_DIRS){ echo "git commit: ${GIT_COMMIT}" echo "lambda dir: ${LAMBDA_DIR}" build job: 'lambda-bulk-asset-ingester-pipeline', parameters: [string(name: 'GIT_COMMIT', value: GIT_COMMIT), string(name: 'LAMBDA_DIR', value: LAMBDA_DIR)], wait: false } } } } ``` ### 3. Pull Request CI Script (pull-request.xml) This script runs on a Pull Request event, using `ghprbActualCommit` to checkout the feature branch. It performs the same change detection as the trigger job, but instead of calling a sub-job, it executes the local Docker-based lint/test check directly within the workspace of each changed Lambda directory. ```groovy node() { stage('Get Git Commit') { GIT_COMMIT = "${ghprbActualCommit}" GIT_VERSION = sh(returnStdout: true, script: "git --version") echo "git version: $GIT_VERSION" checkout([$class: 'GitSCM', branches: [[name: "${GIT_COMMIT}"]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '577cbc72-7d9e-4eba-924c-ecbbe6de9805', url: 'git@github.com:theorchard/lambda-bulk-asset-ingester.git', refspec: '+refs/pull/*:refs/remotes/origin/pr/*']]]) MASTER_COMMIT = sh(returnStdout: true, script: 'git rev-parse origin/master').trim() DIFF_FILES = sh(returnStdout: true, script: "git diff --name-only --diff-filter=ACMR ${MASTER_COMMIT} ${GIT_COMMIT}") script{ LAMBDA_DIRS = [] for (item in DIFF_FILES.split('\n')){ if (item.contains('lambda/')){ lambda_dir = item.split('/')[1] if (!LAMBDA_DIRS.contains(lambda_dir)){ LAMBDA_DIRS.add(lambda_dir) } } } } echo "lambda_dirs: $LAMBDA_DIRS" } stage('Trigger Pull Request') { checkout([$class: 'GitSCM', branches: [[name: "${GIT_COMMIT}"]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: '577cbc72-7d9e-4eba-924c-ecbbe6de9805', url: 'git@github.com:theorchard/lambda-bulk-asset-ingester.git']]]) script{ for (LAMBDA_DIR in LAMBDA_DIRS){ dir("lambda/${LAMBDA_DIR}") { sh "docker compose up --exit-code-from lint-and-test --abort-on-container-exit --build lint-and-test" } } } } } ```