sbt unable to import apache library-CodePudding

i am trying to setup a simple kafka consumer app , which consumes messages from secure HTTPS kafka cluster .

here is my sbt build file .


version := "0.1"
//libraryDependencies  = "org.apache.spark" % "spark-core_2.11" % "2.0.0"
// https://mvnrepository.com/artifact/org.apache.spark/spark-streaming
libraryDependencies  = "org.apache.spark" %% "spark-streaming" % "3.2.0" % "provided"
// https://mvnrepository.com/artifact/org.apache.kafka/kafka
libraryDependencies  = "org.apache.kafka" %% "kafka" % "6.1.0-ccs"
d
scalaVersion := "2.13.6"

my actual consumer code is ..

package main.scala.kafka
import java.util
import java.util.Properties
import org.apache.kafka.clients.consumer.KafkaConsumer
object consumer extends App {




  val TOPIC="amg-dev-time"

  val  props = new Properties()
  props.put("bootstrap.servers", "kafka-localhost.net:9093")

  props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
  props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer")
  props.put("group.id", "amlng-dev-realtime")

  val consumer = new KafkaConsumer[String, String](props)

  consumer.subscribe(util.Collections.singletonList(TOPIC))

  while(true){
    val records=consumer.poll(100)
    for (record<-records.asScala){
      println(record)
    }
  }
}

when i run the above setup i am getting object apache is not a member of package org import org.apache.kafka.clients.consumer.KafkaConsumer

help me fix this and help me with an example on how to connect to secure kafka cluster to consume messages in scala.

upon refreshing sbt build file.. i am getting ..

[error]   Not found
[error]   Not found
[error]   not found: /Users/h0j020h/.ivy2/localorg.apache.kafka/kafka_2.13/6.1.0-ccs/ivys/ivy.xml
[error]   download error: Caught javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error]   not found: https://repository.com/content/repositories/pangaea_releases/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] (ssExtractDependencies) sbt.librarymanagement.ResolveException: Error downloading org.apache.kafka:kafka_2.13:6.1.0-ccs
[error]   Not found
[error]   Not found
[error]   not found: /Users/h0j020h/.ivy2/localorg.apache.kafka/kafka_2.13/6.1.0-ccs/ivys/ivy.xml
[error]   download error: Caught javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target) while downloading https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error]   not found: https://repository.com/content/repositories/pangaea_releases/org/apache/kafka/kafka_2.13/6.1.0-ccs/kafka_2.13-6.1.0-ccs.pom
[error] Total time: 30 s, completed 19-Oct-2021, 6:21:47 pm
[info] shutting down sbt server```

CodePudding user response：

Kafka 6.1.0-ccs is hosted on Confluent.

Try to add:

resolvers  = "confluent" at "https://packages.confluent.io/maven/"

It should solve the problem (example on scastie)