0

We are working on Confluent Platform and we are still getting to know the internals. But we have implemented generic use cases . We are trying to optimizing our cluster

In my use case, I need to increase the number of partitions of a topic . What is the impact of it ? Can you please share of it

Improve this question
1

1 Answer 1

Reset to default
1

Sure, you can increase partitions.

However,

  1. Increasing partitions does not move existing data. If using Confluent Enterprise, you could use confluent-rebalancer, or if not, then kafka-reassign-partitions CLI tool. So, you'll definitely want to rebalance a topic to "optimize" the cluster.
  2. During the retention period of the topic (read: for the existing data), if you previously had a producer sending data to partition N, and now had N+1 partitions, then you lose ordering of those messages that solely existed in partition N. New messages could be spread across new partitions that a new producer calculates with the DefaultPartitioner. If you don't send keys with messages, then this isn't a problem.
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.