Multiple execution of newman in a single stage on Jenkins-CodePudding

I have some collections made with Postman and want to execute them using Newman in the same stage on a Jenkins environment.

My Jennkinsfile looks like below:

sh 'newman run /xxx/collection1.json'
sh 'newman run /xxx/collection2.json'
sh 'newman run /xxx/collection3.json'

However, if any one of a test in the first collection fails, the shell exits with 1 so neither collection 2 nor 3 will be run.

I tried adding || true at the end of each shell execution, but this makes the resulting exit code always be 0, so I can't see the error from my dashboard even if some of the tests fail.

sh 'newman run /xxx/collection1.json || true'
sh 'newman run /xxx/collection2.json || true'
sh 'newman run /xxx/collection3.json || true'

Is there any way to know the final test result while letting all the shells run?

CodePudding user response：

The only approach that comes to my mind would be to wrap the testing stage in certain sub-stages that runs within a parallel block. The following approach should work:

pipeline {
  agent any
    options {
      buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '5')
    }
  stages {
    stage('First') {
      steps {
        sh 'echo "This stage is going to run very first"'
      }
    }
    stage('Hello') {
      failFast false
      parallel {
        stage('one') {
          steps {
            sh 'echo one; exit 1'
          }
        }
        stage('two') {
          steps {
            sh 'echo two'
          }         
        }
        stage('three') {
          steps {
            sh 'echo three'
          }         
        }
      }
    }
  }
}

Stage "First" is supposed to run first, the 3 other stages will run in parallel, with stage "two" and "three" running successfully although stage "one" failed. The whole build is marked as failed, but stage "two" and "three" are marked as green in the dashboard.