Java Kafka console-consumer running out of heap space-CodePudding

I am running the following command in Console. kafka-console-consumer and receiving out of memory error: java.lang.OutOfMemoryError: Java heap space. How can I resolve this? Trying to apply this resource, except I do not have a topics file.

Kafka Connect running out of heap space

Console:

kafka-console-consumer --bootstrap-server localhost:9092 --topic encounter.personEvent

Application.Yml

kafka:
  topic: encounter
  event: personEvent

Error Message below:

ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
java.lang.OutOfMemoryError: Java heap space
    at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:64)
    at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:363)
    at org.apache.kafka.common.memory.MemoryPool$1.tryAllocate(MemoryPool.java:30)
    at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:113)
    at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452)
    at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402)
    at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674)
    at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576)
    at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
    at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:551)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
    at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
    at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:246)
    at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:480)
    at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1262)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1231)
    at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1211)
    at kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:454)
    at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:101)
    at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:75)
    at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:52)
    at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)

CodePudding user response：

KAFKA_HEAP_OPTS is a shared shell variable across all Kafka CLI scripts, so the answer you're referring to in the linked post is correct.

e.g

KAFKA_HEAP_OPTS='-Xmx1G' kafka-console-consumer --bootstrap-server localhost:9092 --topic encounter.personEvent

my apis are running on localhost:8080/swagger-ui.html. and call Kafka producer event

Then you want an HTTP client to POST an HTTP-request to localhost:8080. That, in turn, generates a Kafka message as you say, which is sent to localhost:9092 (by default). You need to consume from Kafka, not an HTTP server