2

Suppose:

  1. Kakfa consumer reads a message M from its assigned partition P
  2. It gets network-partitioned away from the broker
  3. Kafka detects this and reassigns P to a consumer on another machine.
  4. Network-partition is healed, and the first consumer tries to commit the offset for message M

Will there be any exception thrown in step 4? Is there a check which detects that the first consumer is no longer assigned partition P, and so shouldn't be committing offsets for it?

Improve this question

2 Answers 2

Reset to default
0

In this case, the first consumer can not commit off set anymore. It's assigned partition will be revoked and when it join consumer group again, a rebalancing process will be trigger. So the answer is yes.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

So what will happen when it will rejoin consumer group again, but tries to commit an offset from its previously assigned partition? Will Kafka throw an exception?
yeah, it will throw an exception, run time exception :D.
0

https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol

The offsets for a given consumer group are maintained by a specific broker called the group coordinator. i.e., a consumer needs to issue its offset commit and fetch requests to this specific broker

Since one group coordinator is responsible and it know which consumers have partition assigned and which consumers have no longer the partition assigned

When they rejoin, they are treated in the same way as a brand new consumer joining the consumer group. They will get one or more partitions assigned by rebalancing (which may be totally different from the partitions they were reading from last time) and the consumer_offset topic will inform them where to start reading from.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.