Codebuild Step Failure continues onto next stage in pipeline

[rizvfe00]

Multi-branch pipeline with declarative syntax on Jenkins 2.121 and the latest codebuild plugin 0.24

Relevant Code:
buildspec.yml

phases:
  build:
    commands:
      - fail

Jenkinsfile

String version = "1.0.${env.BUILD_NUMBER}"
String awsCredentialsId = '<redacted>'

pipeline {
    agent any
    options {
        gitLabConnection('gitlab')
    }

    stages {
        stage('SCM Checkout') {
            steps {
                echo "CURRENT RESULT: ${currentBuild.result}"
                checkout scm
            }
        }

        stage('Build') {
            steps {
                echo "CURRENT RESULT: ${currentBuild.result}"
                step([$class                   : 'CodeBuilder', artifactLocationOverride: 'artifacts',
                      artifactNameOverride     : "backend-${version}.zip",
                      artifactPackagingOverride: 'ZIP', 
                      artifactPathOverride: 'demo', 
                      artifactTypeOverride: 'S3',
                      credentialsId            : awsCredentialsId, 
                      credentialsType: 'jenkins',
                      projectName              : 'jenkins', 
                      region: 'us-east-1',
                      sourceControlType        : 'jenkins',
                      buildSpecFile            : 'buildspec.yml' ])
                echo "CURRENT RESULT: ${currentBuild.result}"
            }
        }

        stage('Integration test') {
            steps {
                echo "CURRENT RESULT: ${currentBuild.result}"
                echo "Integration test"
            }
        }
    }
}

Expect:
Pipeline to report a failure and skip the rest of the stages

Actual:

 > git config core.sparsecheckout # timeout=10
 > git checkout -f b232234dcfaa524c3eabc45860031de0bdc797ba
Commit message: "severe cleanup to attempt force fail"
 > git rev-list --no-walk b7dd56d723fc1c0130fc0250628610817d738e51 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (SCM Checkout)
[Pipeline] echo
CURRENT RESULT: null
[Pipeline] checkout
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url <-redacted-> # timeout=10
Fetching without tags
Fetching upstream changes from <-redacted->
 > git --version # timeout=10
using GIT_ASKPASS to set credentials 
 > git fetch --no-tags --progress <-redacted->
 +refs/heads/*:refs/remotes/origin/*
Checking out Revision b232234dcfaa524c3eabc45860031de0bdc797ba (test-branch)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f b232234dcfaa524c3eabc45860031de0bdc797ba
Commit message: "severe cleanup to attempt force fail"
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] echo
CURRENT RESULT: null
[Pipeline] step
[AWS CodeBuild Plugin] Using given AWS access and secret key for authorization (provided from Jenkins credentials <-redacted->)
[AWS CodeBuild Plugin] Uploading code to S3 at location <redacted>/demo. MD5 checksum is b/1IWR0hvVjVtk3ZrC2ynw==
[AWS CodeBuild Plugin] S3 object version id for uploaded source is p0FloUnL6y631_g3as759DfbPJQRx8fE
[AWS CodeBuild Plugin] Starting build with 
        > project name: jenkins
        > source version: p0FloUnL6y631_g3as759DfbPJQRx8fE
        > artifact type: S3
        > artifact location: artifacts
        > artifact name: backend-1.0.12.zip
        > artifact packaging: ZIP
        > artifact path: demo
        > build spec: buildspec.yml
[AWS CodeBuild Plugin] Build id: jenkins:027a1720-bb82-4967-9568-d8e8df0ce0eb
[AWS CodeBuild Plugin] CodeBuild dashboard: https://us-east-1.console.aws.amazon.com/codebuild/home?region=us-east-1#builds/jenkins:027a1720-bb82-4967-9568-d8e8df0ce0eb/view/new
[AWS CodeBuild Plugin] CloudWatch dashboard: https://console.aws.amazon.com/cloudwatch/home?region=us-east-1#logEvent:group=/aws/codebuild/jenkins;stream=027a1720-bb82-4967-9568-d8e8df0ce0eb
[AWS CodeBuild Plugin] 2018/05/16 18:25:32 Waiting for agent ping
[AWS CodeBuild Plugin] 2018/05/16 18:25:33 Waiting for DOWNLOAD_SOURCE
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase is DOWNLOAD_SOURCE
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 CODEBUILD_SRC_DIR=/codebuild/output/src340291698/src
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 YAML location is /codebuild/output/src340291698/src/buildspec.yml
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Processing environment variables
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Moving to directory /codebuild/output/src340291698/src
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Registering with agent
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phases found in YAML: 1
[AWS CodeBuild Plugin] 2018/05/16 18:25:36  BUILD: 1 commands
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: DOWNLOAD_SOURCE Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase INSTALL
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: INSTALL Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase PRE_BUILD
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: PRE_BUILD Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase BUILD
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Running command fail
[AWS CodeBuild Plugin] /codebuild/output/tmp/script.sh: 4: /codebuild/output/tmp/script.sh: fail: not found
[AWS CodeBuild Plugin] 
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Command did not exit successfully fail exit status 127
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: BUILD Success: false
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code: COMMAND_EXECUTION_ERROR Message: Error while executing command: fail. Reason: exit status 127
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Entering phase POST_BUILD
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase complete: POST_BUILD Success: true
[AWS CodeBuild Plugin] 2018/05/16 18:25:36 Phase context status code:  Message: 
[AWS CodeBuild Plugin] Build jenkins:027a1720-bb82-4967-9568-d8e8df0ce0eb failed
        > Error while executing command: fail. Reason: exit status 127 (status code: COMMAND_EXECUTION_ERROR)
[Pipeline] echo
CURRENT RESULT: FAILURE
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Integration test) _<---- should not execute_
[Pipeline] echo
CURRENT RESULT: FAILURE _<---- should not execute_
[Pipeline] echo
Integration test  _<---- should not execute_
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: FAILURE

When replacing the codebuild step with a sh 'FAIL' with the same setup, works as expected and the integration stage is skipped due to failure.

[leoherran-aws]

Thanks for letting us know about this. I'll attempt to reproduce it and then diagnose the root cause.

[leoherran-aws]

The issue here is that the plugin does not recognize the step([$class : 'CodeBuilder' syntax as a pipeline step. One workaround is to replace that with awsCodeBuild (see the pipeline snippet generator). Another option is to check the build status after the build and throw an exception if the status is failed:

script {    
  if(currentBuild.result == "FAILURE") {
    throw new Exception()
  }
}

[vedarthk]

I am facing same issue with following configuration:

stage('package') {
    steps {
        withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws']]) {
            awsCodeBuild credentialsId: 'CodeBuild', credentialsType: 'jenkins', projectName: 'Package', region: 'us-east-1', sourceControlType: 'project', sourceVersion: "${params.COMMIT_HASH}", envVariables: "[{BUILD_SCRIPT, docker/package/pack.sh}, {JENKINS_BUILD_NUMBER, ${env.BUILD_NUMBER}}, {AWS_SECRET_ACCESS_KEY, ${env.AWS_SECRET_ACCESS_KEY}}, {AWS_ACCESS_KEY_ID, ${env.AWS_ACCESS_KEY_ID}}, {ARTIFACTS_BUCKET, ${env.ARTIFACTS_BUCKET}}, {BRANCH, ${env.BRANCH}}]"
        }
    }
}

Codebuild Plugin Version - 0.17

The build continued even though the codebuild run failed in DOWNLOAD_SOURCE stage.

[leoherran-aws]

Can you confirm that you're still having problems getting your use case working?

[shawnpetros]

this has NOTHING to do with this particular issue which remains open, but...anyone know how to access the ENV vars in the CodeBuild buildspec.yml when they are supplied by this plugin? @vedarthk has envVariables listed in his Jenkins Pipeline file, I want to know how to access these vars in the CodeBuild environment...sorry if it derails the convo...

[lkolchin]

Hi,

If using try { cbResult = awsCodeBuild ....} catch {} (as advised here https://issues.jenkins-ci.org/browse/JENKINS-61259?focusedCommentId=386419&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel)

Even if one of the CodeBuild phases fails in the previous stages the other stages continue

So to work around it I have to add:

                        if(currentBuild.result == "FAILURE") {
                            throw new Exception()
                        }

Is that something that can be fixed?