6

I'm playing around with Kafka and using my own local single instance of zookeeper + kafka and running into this error that I don't seem to understand how to resolve.

I started a simple server per the Apache Kafka Quickstart Guide

$ bin/zookeeper-server-start.sh config/zookeeper.properties $ bin/kafka-server-start.sh config/server.properties

Then utilizing kafkacat (installed via Homebrew) I started a Producer that will just echo messages that I type into the console

$ kafkacat -P -b localhost:9092 -t TestTopic -T test1 test1

But when I try to consume those messages I get an error:

$ kafkacat -C -b localhost:9092 -t TestTopic % ERROR: Topic TestTopic error: Broker: Leader not available

And similarly when I try to list its' metadata

$ kafkacat -L -b localhost:9092 -t TestTopic Metadata for TestTopic (from broker -1: localhost:9092/bootstrap): 0 brokers: 1 topics: topic "TestTopic" with 0 partitions: Broker: Leader not available (try again)

My questions:

  1. Is this an issue with my running instance of zookeeper and/or kafkacat - I ask this because I've been constantly shutting them down and restarting them, after deleting the /tmp/zookeeper and /tmp/kafka-logs directories
  2. Is there some simple setting that I need to try? I tried adding auto.leader.rebalance.enable=true in Kafka's server.properties settings file, but that didn't fix this particular issue
  3. How do I do a fresh restart of zookeeper/kafka. Is shutting them down, deleting the /tmp/zookeeper and /tmp/kafka-logs directories and then restarting zookeeper and then kafka the way to go? (Well maybe the way to go is to build a docker container that I can stand-up and tear down, I was going to use the spotify/docker-kafka container but that is not on Kafka 0.9.0.0 and I haven't taking the time to build my own)
Improve this question
4
  • I am having the same issue with the same tutorial. have you found a solution for this? I am not using Kafkakat though. Commented Apr 13, 2016 at 18:41
  • my solution was to stand up a docker container that I can just tear down and re-run as needed. See: github.com/spotify/docker-kafka you can just change the version of Kafka you want to run in the Dockerfile and build the image Commented Apr 27, 2016 at 17:31
  • Please have a look here: stackoverflow.com/a/52209029/1545425 In the situation reported in the other question the problem was seen at production time, but i believe it might be the same error at consumption time as in your case Commented Aug 23, 2021 at 13:53
  • Can you try bin/kafka-topics.sh --list --bootstrap-server localhost:9092. Try debugging with the built in tools in the bin/ dir. I doubt if your broker is running, you can post your broker logs here to aid in debugging. Commented Aug 26, 2021 at 14:39

1 Answer 1

Reset to default
5
  1. It might be, but probably is not. My guess is the topic isn't created, so kafkacat echoes the massage on screen but doesn't really send it to kafka. All the topics are probably deleted after you delete the /tmp/kafka-logs
  2. No. I don't think this is the way to look for a solution.
  3. Having a docker container is definitely the way to go - you'll soon end up running kafka on multiple brokers, examining the replication behavior, high availability scenarios etc.. Having it dockerised helps a lot.
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.