ERROR JacksonVersion - Version '2.9.4' of package 'jackson-databind' is not supp-CodePudding

i am using azure sdk(azure-storage-file-datalake, azure-core, azure-identity) for writing the avro data to azure datalake gen 2 from kafka(using connectors), but i am getting very weird errors as below

2022-05-12 10:46:10 ERROR JacksonVersion:506 - Version '2.9.4' of package 'jackson-annotations' is not supported (older than earliest supported version - 2.10.0), please upgrade.

2022-05-12 10:46:10 ERROR JacksonVersion:506 - Version '2.9.4' of package 'jackson-core' is not supported (older than earliest supported version - 2.10.0), please upgrade.

2022-05-12 10:46:10 ERROR JacksonVersion:506 - Version '2.9.4' of package 'jackson-databind' is not supported (older than earliest supported version - 2.10.0), please upgrade.

java.lang.LinkageError: Package versions: jackson-annotations=2.9.4, jackson-core=2.9.4, jackson-databind=2.9.4, jackson-dataformat-xml=unknown, jackson-datatype-jsr310=unknown, azure-core=1.28.0, Troubleshooting version conflicts: https://aka.ms/azsdk/java/dependency/troubleshoot\n\tat com.azure.core.implementation

I have tried excluding the jackson dependencies and tried adding explicitly, still i am getting the errors.

please some one help on this

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-annotations</artifactId>
    <version>2.13.2</version>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.13.2</version>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.13.2</version>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.dataformat</groupId>
    <artifactId>jackson-dataformat-xml</artifactId>
    <version>2.13.2</version>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.datatype</groupId>
    <artifactId>jackson-datatype-jsr310</artifactId>
    <version>2.13.2</version>
    <scope>runtime</scope>
</dependency>

    <dependencies>


    <dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>connect-api</artifactId>
        <version>2.0.1</version>
        <scope>provided</scope>
    </dependency>
    <!--<dependency>
        <groupId>org.apache.kafka</groupId>
        <artifactId>kafka-clients</artifactId>
        <version>1.1.0</version>
        <scope>provided</scope>
    </dependency>-->
    <dependency>
        <groupId>log4j</groupId>
        <artifactId>log4j</artifactId>
        <version>1.2.17</version>
    </dependency>
    <!-- Apache avro dependency -->
    <dependency>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro</artifactId>
        <version>1.8.2</version>
    </dependency>
    <!-- Newer versions of the storage account jar is not working. -->
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-storage-file-datalake</artifactId>
        <version>12.9.0</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <!-- https://mvnrepository.com/artifact/com.azure/azure-core -->
    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-core</artifactId>
        <version>1.28.0</version>
        <exclusions>
            <!-- Collision with Kafka Jackson dependencies -->
            <exclusion>
                <groupId>com.fasterxml.jackson.core</groupId>
                <artifactId>jackson-core</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.fasterxml.jackson.core</groupId>
                <artifactId>jackson-annotations</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.fasterxml.jackson.core</groupId>
                <artifactId>jackson-databind</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.fasterxml.jackson.dataformat</groupId>
                <artifactId>jackson-dataformat-xml</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.fasterxml.jackson.datatype</groupId>
                <artifactId>jackson-datatype-jsr310</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>com.azure</groupId>
        <artifactId>azure-identity</artifactId>
        <version>1.5.1</version>
        <exclusions>
            <exclusion>
                <groupId>com.azure</groupId>
                <artifactId>azure-core</artifactId>
            </exclusion>
            <exclusion>
                <groupId>com.microsoft.azure</groupId>
                <artifactId>msal4j</artifactId>
            </exclusion>
            <exclusion>
                <groupId>org.linguafranca.pwdb</groupId>
                <artifactId>KeePassJava2</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>com.microsoft.azure</groupId>
        <artifactId>msal4j</artifactId>
        <version>1.9.1</version> <!-- {x-version-update;com.microsoft.azure:msal4j;external_dependency} -->
        <exclusions>
            <exclusion>
                <groupId>com.fasterxml.jackson.core</groupId>
                <artifactId>jackson-databind</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- Schema registry dependency -->

    <dependency>
        <groupId>io.confluent</groupId>
        <artifactId>kafka-connect-avro-converter</artifactId>
        <version>3.3.0</version>
        <exclusions>
            <exclusion>
            <groupId>io.confluent</groupId>
            <artifactId>kafka-schema-registry-client</artifactId>
            </exclusion>
        </exclusions>
    </dependency>

    <dependency>
        <groupId>io.confluent</groupId>
        <artifactId>kafka-schema-registry-client</artifactId>
        <version>3.3.0</version>
        <exclusions>
            <exclusion>
                <groupId>com.fasterxml.jackson.core</groupId>
                <artifactId>jackson-databind</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.mockito</groupId>
        <artifactId>mockito-core</artifactId>
        <version>2.21.0</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.13.2</version>
        <scope>test</scope>
    </dependency>






</dependencies>

CodePudding user response：

Dependencies with scope "runtime" are not used for compilation, but only for execution. A dependency with this scope is not put on the compile classpath, so you could remove the runtime scope.

However a better way to overwrite transitive dependency versions that you don't use yourself is to use depependency management like this:

<project>
  [...]
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>group-a</groupId>
        <artifactId>artifact-a</artifactId>
        <version>1.0</version>
      </dependency>
    [...]
    </dependencies>
  </dependencyManagement>
</project>

CodePudding user response：

This is a challenge with Azure-SDK (here is a similar issue). Azure-SDK has a dependency on Jackson. The specific version of Jackson depends on the azure-sdk version you are using.

The latest release supports Jackson 2.10-2.13. What it means is you may have other dependencies on Jackson, and the SDK will still work. Provided Jackson version fetched is from 2.10-2.13. Azure-SDK is compatible with all these versions.

If you wish to use Jackson 2.9 (or if that's coming as a dependency from other dependency), then there is no way around it. You would necessarily need to upgrade your dependencies so that it fetches jackson 2.10 and above.

In your case, there would be some dependency that is fetching 2.9.x series.

This is what I would do:

Based on all non-azure-sdk dependencies. Decide which jackson version to hold on to. Explicitly add this jackson dependency in the pom.xml
Use azure-sdk BOM
You dont have to specify exclusion. Specifying a version in pom overrides other dependency versions