Fail Jenkins build/stage if any errors in JMeter-CodePudding

Jenkins pipeline exec -> sh jmeter -n -t /var/19072022.jmx

Console output on Jenkins even when there are errors.

summary =    117 in 00:00:08 =   13.9/s Avg:    58 Min:     0 Max:   492 Err:    59 (50.43%)
Tidying up ...    @ Wed Jul 20 17:42:49 CEST 2022 (1658331769098)
... end of run
Finished: SUCCESS

Using the Jmeter performance plugin with an error threshold flag is not an option due to vulnerabilities.

Tried JSR223 listener

if (!prev.isSuccessful()) {
    System.exit(1)
}

Jenkins still passes the build even though there was an error and the test was terminated.

Waiting for possible Shutdown/StopTestNow/HeapDump/ThreadDump message on port 4445
summary =      1 in 00:00:01 =    1.5/s Avg:   458 Min:   458 Max:   458 Err:     1 (100.00%)
Tidying up ...    @ Thu Jul 21 16:13:48 CEST 2022 (1658412828482)
... end of run
Finished: SUCCESS

CodePudding user response：

Your "not working" stanza is very informative, do you have any evidences by any chance? Because it's "working" for me.

So maybe the Sampler(s) which is failing is not in the JSR223 Listener's scope?

Also be aware that the listener will immediately terminate the JVM so your test will end as soon as the first failure occurs and you may loose a portion of your test results because JMeter flushes them periodically.

If you're unwilling to use Performance Plugin you can consider using Taurus tool as the wrapper for your test, it has Pass/Fail Criteria subsystem which is capable of returning non-zero exit status code if the criteria are not met so it might be a viable alternative for your use case.

More information:

CodePudding user response：

So, what worked for me was a BeanShell Listener containing the code below.

if (!prev.isSuccessful()) {
    System.exit(1);
}

The plus with this is that whenever there is an error, the process is terminated, forcing the Jenkins pipeline to fail. The downside is, request samples after the error is not executed.

In my case, I expect all scripts to pass because 1 fail means the whole process is not valid.