12

I am trying to set up Kafka rest proxy using a docker. But the topic I am providing in the configuration isn't creating.

I am checking Kafka Topic with API: curl "http://metrics-kafka-rest:38082/topics" and I am getting this response: ["__confluent.support.metrics","_schemas"]

Below in the config that I have used in docker-compose:

 image: confluentinc/cp-zookeeper:5.3.0 container_name: 'metrics-zookeeper' restart: always ports: - "32181:32181" environment: ZOOKEEPER_CLIENT_PORT: 32181 ZOOKEEPER_TICK_TIME: 2000 ZOOKEEPER_SYNC_LIMIT: 2 ZOOKEEPER_SASL_ENABLED: "FALSE" metrics-kafka: image: confluentinc/cp-kafka:5.3.0 container_name: 'metrics-kafka' restart: always ports: - "29092:29092" depends_on: - metrics-zookeeper environment: KAFKA_BROKER_ID: 1 KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true" KAFKA_ZOOKEEPER_CONNECT: metrics-zookeeper:32181 KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://metrics-kafka:29092 KAFKA_CREATE_TOPICS: "Notification:1:1" KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 CONFLUENT_METRICS_ENABLE: 'false' metrics-schema-registry: image: confluentinc/cp-schema-registry:5.3.0 container_name: 'metrics-schema-registry' restart: always ports: - "38081:38081" depends_on: - metrics-kafka environment: SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: metrics-zookeeper:32181 SCHEMA_REGISTRY_HOST_NAME: metrics-schema-registry SCHEMA_REGISTRY_LISTENERS: "http://metrics-schema-registry:38081" SCHEMA_REGISTRY_DEBUG: "true" metrics-kafka-rest: image: confluentinc/cp-kafka-rest:5.3.0 container_name: 'metrics-kafka-rest' restart: always ports: - "38082:38082" depends_on: - metrics-schema-registry environment: KAFKA_REST_ZOOKEEPER_CONNECT: metrics-zookeeper:32181 KAFKA_REST_SCHEMA_REGISTRY_URL: "http://metrics-schema-registry:38081" KAFKA_REST_HOST_NAME: metrics-kafka-rest KAFKA_REST_LISTENERS: "http://metrics-kafka-rest:38082" KAFKA_REST_DEBUG: "true"

I expect that when I hit API of getting the list of topics then it should contain topic Notification.

Improve this question

2 Answers 2

Reset to default
17

KAFKA_CREATE_TOPICS is not a supported Environment variable for the cp-kafka image that you're using.

Since you already have KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true" then you can just start using the broker and topics will be created as and when they're first referenced by the producer or consumer.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

So how should we create default topics with cp-kafka at container startup time?
But is it possible to define a name previously? [Edited] I think that it will answer my question environment: KAFKA_CREATE_TOPICS: "Topic1:1:3,Topic2:1:1:compact" See: registry.hub.docker.com/r/wurstmeister/kafka
5

Agree with @Robin, and give some config references because some guys mix the env variables KAFKA_CREATE_TOPICS for images wurstmeister/kafka: https://registry.hub.docker.com/r/wurstmeister/kafka and confluentinc/cp-kafka:https://docs.confluent.io/platform/current/installation/docker/config-reference.html#optional-confluent-enterprise-ak-settings This is to say KAFKA_CREATE_TOPICS is not a supported Environment variable for the cp-kafka, it's for wurstmeister/kafka

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.