Is a partition always on the same physical machine as a broker or can a partition reside on a machine which is not also a broker?
I am pretty sure a partition or multiple partitions can reside on a broker node but I am not sure if a partition can reside on a non broker node?
Partition is just a structure / object which resides inside the broker. Without running Kafka broker, there is no partition. They cannot exist outside of broker.
The Kafka brokers run in clusters - a Kafka cluster can consist only from one broker, but it can be 1000s of brokers. When you create a topic with a defined number of partitions, they will be distributed (either automatically, or you can specify this distribution) across the brokers in the cluster. So if you want to use multiple machines for your topics / partitions, you will need to run a Kafka broker on each of these machines and connect them into a cluster.
a partition reside in multiple brokers (Servers) ?
No a partition cannot reside on multiple machine in kafka ... Partitions cannot be split between multiple brokers and not even between multiple disks on the same broker .....In other words you can say that the size of the partition is limited by the space in the disk mount.
