Given a Maven module with multiple test source roots, how can we limit maven-surefire-plugin to run tests only from a specific one?

The use case is as follows: I would like to have multiple test directories, and pick them based on maven profiles, e.g. -P only-tests-in-directory-A or -P only-tests-in-directory-B.

I am aware about include and exclude configuration options, but they only work on package/class names, which is not great, because in all test roots the tests might be named similarly.

There is also a testSourceDirectory configuration option, but according to documentation it only relates to testNG reports.

CodePudding user response：

The most important thing in Maven is to follow conventions over configuration. This means to name integration tests like *IT.java instead of something else. If you have a single module setup they belong within the same module if you have multi module setup it sometimes make sense to have a separte module for integration tests.

Furthermore it means integration tests have to be executed by maven-failsafe-plugin which has to be activated by using it like this:

<plugins>
  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <executions>
      <execution>
        <goals>
          <goal>integration-test</goal>
          <goal>verify</goal>
        </goals>
      </execution>
    </executions>
  </plugin>
</plugins>

It is of course necessary to define the plugin version via an appropriate pluginManagement section in your pom file or in your common parent.

A pluginManagement configuration could looke like this:

    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>3.0.0-M7</version>
      <configuration>
        <skipTests>${skipUTs}</skipTests>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-failsafe-plugin</artifactId>
      <version>3.0.0-M7</version>
      <configuration>
        <skipTests>${skipITs}</skipTests>
      </configuration>
    </plugin>

In combination that give the opportunity to run unit tests via:

mvn test

If you want to run integration tests as well (incl. unit tests):

mvn verify 

Sometimes it makes sense not to run integration tests during a release or alike which can simply be achieved by using (sometimes people use profiles for such things):

mvn verfiy -DskipITs

To make it possible to run supplemental tests (whatever we call them) you could do that via test suites. That needs the following dependencies in JUnit Jupiter like this:

  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>5.9.1</version>
        <scope>import</scope>
        <type>pom</type>
      </dependency>
    ....
    </dependencies>
  </dependencyManagement>

And really using the following dependencies (And yes you need both of them):

  <dependencies>
    <dependency>
      <groupId>org.junit.platform</groupId>
      <artifactId>junit-platform-suite-engine</artifactId>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter-engine</artifactId>
      <scope>test</scope>
    </dependency>

You can define a test suite like the following:

import org.junit.platform.suite.api.IncludeClassNamePatterns;
import org.junit.platform.suite.api.SelectPackages;
import org.junit.platform.suite.api.Suite;
import org.junit.platform.suite.api.SuiteDisplayName;

@Suite
@SuiteDisplayName("Suite Demo - First Suite")
@SelectPackages("packageName.suite1")
@IncludeClassNamePatterns(".*SD")
class FirstSuite {

}

By using a particular package for say your System-Integration Tests Part one etc. Those test will be included there... The naming pattern here as defined by the annotation *.*SD or whatever naming pattern you like (but be carefull not to interfere with the usual ones of surefire/failsafe plugin)

A particular suite class could look like this:

class SuiteOneSD {

  @Test
  void first_add() {
    Fraction sum1 = new Fraction(5);
    Fraction sum2 = new Fraction(2);

    assertThat(sum1.add(sum2)).isEqualTo(new Fraction(7));
  }

You can activate a suite simply on command line via:

mvn test -Dtest=FirstSuite

This will only those tests defined by the FirstSuite. If you want to combine those suite with your unit tests you can simply do that via (be careful about the quoting it is necessary):

mvn test -Dtest='FirstSuite,*Test'

You can also execute the suites in a different phase (at the time of the integration tests). This will execute the unit tests including the suites.

mvn verify -Dit.test='FirstSuite'

If you want to combine the suites with the usual integration tests:

mvn verify -Dit.test='FirstSuite,*IT'

The given properties give you the opportunity to control if unit- and/or integration tests will be running:

mvn verify -DskipUTs 

only integration tests will run but not unit tests.

mvn verify -DskipITs 

only unit tests will run but no integration tests.

All without any profile nor separate test source directory etc.

CodePudding user response：

I think @khmarbaise's question is incredibly relevant. OP, you should consider the complexity of your project structure. Why do you have multiple test source roots and not just multiple packages? Are you trying to separate IT and other tests or something?

Regardless, you could use the Build Helper plugin's add-test-source goal:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <version>3.3.0</version>
  <executions>
    <execution>
      <goals>
        <goal>add-test-source</goal>
      </goals>
      <configuration>
        <sources>
          <source>src/it/java</source>
        </sources>
      </configuration>
    </execution>
  </executions>
</plugin>

You can also use the add-test-resource goal to add resources.

More about the plugin here: https://www.mojohaus.org/build-helper-maven-plugin/

Hope this helped!