I have set of JUnit5 tests which are run and reported with gradle by Jenkins. Some of them are unstable without my control (let's say "internal server error" from external service) and in random way. I can re-run them with gradle "test retry" plugin or by some other way but then initial failures are present in test summary and build is reported as unstable in Jenkins. But the need is to have "Success" build if failed tests are good after second try. Therefore does anybody know the way to re-run tests without reporting of initial failure in test results?

My current idea is to try the following:

• hide specific exceptions with TestExecutionExceptionHandler to have initial failure reported as success
• implement @TestFactory to return failed tests dynamically for second run
• implement ClassOrderer to run mentioned test factory as the last test and other complications. So looks pretty ugly. Therefore hope for existing of better solution.

Update: Tried to use https://github.com/artsok/rerunner-jupiter library. Since I don't want to replace @Test annotations with custom @ReplaceIfExceptions... for all test methods in the project, I tried to override ...Extension class of the library and use it with "@ExtendWith" at class level of some basic test class. Then understood that replacing of @Test with @TestTemplate at method level is still needed for this approach to work. Therefore now I'm thinking about replacing of @Test with some my custom annotation to have ability quickly add to it anything like @ReplaceIfExceptions... or @TestTemplate or whatever would be needed.

CodePudding user response：

Finally I used https://github.com/artsok/rerunner-jupiter library. It throws TestAbortedException before repetition of failed test and such failure is marked as ignored test. The result of build is marked as successful in this case. It was needed to replace @Test annotation to my custom annotation and add @TestTemplate to it, also I extended from Extension class of the library and applied my implementation of extension to class of my basic test using @ExtendWith. Looks like the only way to use the library without using of @ReplaceIfExceptions... annotation on each test method.