Download as PDF, PPTX
Uploaded byDiego Pacheco
Apache Kafka
Apache Kafka is a high-throughput distributed messaging system that allows for both streaming and offline log processing. It uses Apache Zookeeper for coordination and supports activity stream processing and real-time pub/sub messaging. Kafka bridges the gaps between pure offline log processing and traditional messaging systems by providing features like batching, transactions, persistence, and support for multiple consumers.
In this document
Powered by AI
Apache Kafka is introduced as a high-throughput distributed messaging system, emphasizing its architecture and playful comparison to JMS.
Kafka supports activity stream processing, real-time pub/sub messaging, and uses Apache Zookeeper. It addresses common messaging issues with high performance and persistence.
Kafka bridges gaps between various messaging issues, supporting both real-time processing and offline aggregation with frameworks like Hadoop. References to producer and consumer testing, and further emphasis on Kafka's capabilities in large data handling and stream processing.
The presentation concludes with gratitude and reiteration of Kafka's advantages in messaging and data processing.
Apache Kafka
- 1. Apache Kafka A high-throughput distributed messaging system. @diego_pacheco Software Architect | Agile Coach
- 2. Apache Kafka A high-throughputdistributed messaging system.
- 3. Apache Kafka A high-throughputdistributed messaging system. No JMS Stuff
- 4. Apache Kafka A high-throughput distributed messaging system. Activity Stream Processing / Offline log processing + Real-time Pub/Sub Messaging
- 5. Apache Kafka A high-throughputdistributed messaging system. Uses Apache Zookerper
- 6. Apache Kafka A high-throughputdistributed messaging system. “Distributed Publish-Subiscribe messaging system, Hight Troughtput, Persistent, Partitioning Messages, Parallel data lod into Hadoop.”
- 7. Apache Kafka A high-throughput distributed messaging system. Pure offline log processing are: Real-time, High Performance, Hight Troughtput Lightweight business logic(and not lots) to deliver that.
- 8. Apache Kafka A high-throughput distributed messaging system. Pure Messasing issues(ActiveMQ/RabbitMQ): NO API for Batching, Transactional, No persistence means, multiple consumers are limited by arch.
- 9. Apache Kafka A high-throughputdistributed messaging system. Kafka bridges the gap beetween both issues!
- 10. Apache Kafka A high-throughputdistributed messaging system. Producer Test
- 11. Apache Kafka A high-throughputdistributed messaging system. Consumer Test
- 12. Apache Kafka A high-throughput distributed messaging system. Supports *Activity Stream Processing*, like: Facebook/Scribe and Apache Flume. *Activity Stream Processing* => Collecting, Aggregating, larges ammout on data, very present on social business. Later you often do offline analysys with hadoop. A.K.A Offline log Aggregation.
- 13. Apache Kafka A high-throughputdistributed messaging system.
- 14. Apache Kafka A high-throughputdistributed messaging system.
- 15. Apache Kafka A high-throughputdistributed messaging system.
- 16. Apache Kafka A high-throughput distributed messaging system. Thank You ! @diego_pacheco Software Architect | Agile Coach