1

I am running the following command in Console. kafka-console-consumer and receiving out of memory error: java.lang.OutOfMemoryError: Java heap space. How can I resolve this? Trying to apply this resource, except I do not have a topics file.

Kafka Connect running out of heap space

Console:

kafka-console-consumer --bootstrap-server localhost:9092 --topic encounter.personEvent

Application.Yml

kafka: topic: encounter event: personEvent

Error Message below:

ERROR Error processing message, terminating consumer process: (kafka.tools.ConsoleConsumer$) java.lang.OutOfMemoryError: Java heap space at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:64) at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:363) at org.apache.kafka.common.memory.MemoryPool$1.tryAllocate(MemoryPool.java:30) at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:113) at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:452) at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:402) at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:674) at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:576) at org.apache.kafka.common.network.Selector.poll(Selector.java:481) at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:551) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236) at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215) at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:246) at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:480) at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1262) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1231) at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1211) at kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:454) at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:101) at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:75) at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:52) at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
Improve this question

1 Answer 1

Reset to default
5

KAFKA_HEAP_OPTS is a shared shell variable across all Kafka CLI scripts, so the answer you're referring to in the linked post is correct.

e.g

KAFKA_HEAP_OPTS='-Xmx1G' kafka-console-consumer --bootstrap-server localhost:9092 --topic encounter.personEvent

my apis are running on localhost:8080/swagger-ui.html. and call Kafka producer event

Then you want an HTTP client to POST an HTTP-request to localhost:8080. That, in turn, generates a Kafka message as you say, which is sent to localhost:9092 (by default). You need to consume from Kafka, not an HTTP server

Improve this answer
Sign up to request clarification or add additional context in comments.

4 Comments

@mattsmith5 Because 9092 is the default Kafka port. Why did you change it for your server?
You cannot consume from an HTTP server. You would use curl localhost:8080 for that
You can use the Swagger UI, as well. Doesn't really matter, but I suspect you will get the data you want if you use the correct port in your consumer CLI
kafka-console-consumer.sh is just a wrapper around Java/Scala code. You don't need to write it in your own app unless you do need that data there as well. Yes, it's for testing/scripting, but you still need to point it at the correct address (which is not an HTTP server)

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.