Jqwik: How to check if all possibilities is covered?-CodePudding

How to check if all possibilities (cartesian product of arguments) is covered in summary by N properties? Some of them can be tested few times by different properties.

CodePudding user response：

There's no way in jqwik to assert exhaustive parameter generation across property methods. What you can do, though, is to check all property conditions in a single property method:

@Property(generation = GenerationMode.EXHAUSTIVE)
void fullCartesianProductProperty(
    @ForAll @IntRange(min = 0, max = 1) int p1,
    @ForAll @IntRange(min = 0, max = 3) int p2,
    @ForAll @IntRange(min = 1, max = 3) int p3
) {
    MyEnum result = toCheck(p1, p2, p3);
    if (p1 > 0) {
        // Will only be executed in 12 cases
        assertThat(result)...;
    }
    if (p2 <= 2) {
        // Will only be executed in 18 cases
        assertThat(result)...;
    }
    if (p3 > 1) {
        // Will only be executed in 16 cases
        assertThat(result)...;
    }
    // Will be executed in all 24 cases
    assertThat(result);
}

In this case, all 24 combinations will be executed.

Mind that generation = GenerationMode.EXHAUSTIVE is only necessary if the size of the cartesian product exceeds a property's number of tries (1000 by default).

CodePudding user response：

I found a solution with AfterContainer, but I have a problem with disappearing Statistics after every Property.

private static final StatisticsCollectorImpl CLASS_STATISTICS_COLLECTOR = new StatisticsCollectorImpl("allCases");

Actually I am using manually created StatisticsCollectorImpl and updating it additionally in every Property. So, in result, I have statistics per property and per class.

Statistics.collect(p1, p2, p3);
CLASS_STATISTICS_COLLECTOR.collect(p1, p2, p3);

Fake property allCasesGenerator generates cartesian product.

@Property(generation = GenerationMode.EXHAUSTIVE)
void allCasesGenerator (
    @ForAll @IntRange(min = 0, max = 1) int p1,
    @ForAll @IntRange(min = 0, max = 3) int p2,
    @ForAll @IntRange(min = 1, max = 3) int p3
) {
    CLASS_STATISTICS_COLLECTOR.collect(p1, p2, p3);
}

In AfterContainer I am checking cases existing only once - only from allCasesGenerator.

@AfterContainer
    static void afterAll() {
        final List<String> presentOnlyInAllCasesGeneratorProperty = CLASS_STATISTICS_COLLECTOR.statisticsEntries()
                .stream()
                .filter(entry -> entry.count() <= 1)
                .map(StatisticsEntryImpl::name)
                .collect(Collectors.toList());
        assertThat(presentOnlyInAllCasesGeneratorProperty)
                .isEmpty();
    }