4

1) Can any any one explain how KAFKA consumer knows about partition and consume the data from specific partition?.

What is consumer group?

2) I have 3 partitions and data stored in all partitions,

for example ( msg1, msg2 , msg3 and msg4). Msg1 -> partition 2, msg2 -> partition 3, msg3 -> partition 1, msg4 -> partition 4.

So how can I get sequential order of messages (msg) from consumer.

Improve this question

1 Answer 1

Reset to default
3
  1. Broker partitions are registered in Zookeeper so by querying ZK, consumers can find out what the available partitions are. The high-level consumer is doing this automatically for you.

A consumer group is a common identifier for a group of consumer instances which makes it possible to parallelize consumption for one or more topics. The number of consumers consuming a topic in parallel can be less than or equal to the number of partitions for that topic and partitions will be balanced across consumer instances with the same consumer group id.

  1. Kafka guarantees ordered consumption within a partition but not across partitions so if this is needed, it has to be handled at consumer side.
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.