![Confluents Business Value per Use Case Improve Customer Experience (CX) Increase Revenue (make money) Business Value Decrease Costs (save money) Core Business Platform Increase Operational Efficiency Migrate to Cloud Mitigate Risk (protect money) Key Drivers Strategic Objectives (sample) Fraud Detection IoT sensor ingestion Digital replatforming/ Mainframe Offload Connected Car: Navigation & improved in-car experience: Audi Customer 360 Simplifying Omni-channel Retail at Scale: Target Faster transactional processing / analysis incl. Machine Learning / AI Mainframe Offload: RBC Microservices Architecture Online Fraud Detection Online Security (syslog, log aggregation, Splunk replacement) Middleware replacement Regulatory Digital Transformation Application Modernization: Multiple Examples Website / Core Operations (Central Nervous System) The [Silicon Valley] Digital Natives; LinkedIn, Netflix, Uber, Yelp... Predictive Maintenance: Audi Streaming Platform in a regulated environment (e.g. Electronic Medical Records): Celmatix Real-time app updates Real Time Streaming Platform for Communications and Beyond: Capital One Developer Velocity - Building Stateful Financial Applications with Kafka Streams: Funding Circle Detect Fraud & Prevent Fraud in Real Time: PayPal Kafka as a Service - A Tale of Security and Multi-Tenancy: Apple Example Use Cases $↑ $↓ $ Example Case Studies (of many)](https://image.slidesharecdn.com/201811praguekafkameetupeventdrivenarchitectureswithapachekafkaandconfluent-181127151438/75/Apache-Kafka-as-Event-Driven-Open-Source-Streaming-Platform-Prague-Meetup-26-2048.jpg)
Uploaded byKai Wähner
Apache Kafka as Event-Driven Open Source Streaming Platform (Prague Meetup)
The document serves as an introduction to Apache Kafka, highlighting its role as an event-driven open-source streaming platform for handling various business events and data processes. Key features include event stream processing, integration with microservices, and applications in various industries such as retail, fraud detection, and cloud migration. The content also emphasizes the advantages of leveraging Kafka for operational efficiency and improving customer experiences.
Apache Kafka as Event-Driven Open Source Streaming Platform (Prague Meetup)
- 1. 1Confidential Introduction to ApacheKafka as Event-Driven Open Source Streaming Platform Kai Waehner Technology Evangelist kontakt@kai-waehner.de LinkedIn @KaiWaehner www.confluent.io www.kai-waehner.de Apache Kafka, Kafka Streams, KSQL, Schema Registry, and more…
- 6. Events What is anevent?
- 7.
- 8. Events A Sale AnInvoice A Trade A Customer Experience
- 9. Events and State User_data= ‘old_value’ User_data = ‘new_value’
- 10.
- 11. All Your Datais Streams of Events
- 12. Events have twofunctions 1. Triggers 2. Data
- 13. Where are they? Eventshaven’t had a proper home in infrastructure or in code. They are implicit. Here!
- 14. Events are everywhere Abusiness is a series of events and the reactions to those events.
- 15.
- 16. What do wehave today? Data Stores
- 17. You need boththese things
- 18. Events and BusinessServices Business Service Business Service Sales Events New Orders to place Processing and Inventory
- 19. Events and DataPipelines Databases Databases Customer Data Updates Unified 360 Merged Customer Profiles
- 20. These are equivalent Bothare Event Stream Processing
- 21.
- 22. Haven’t we seenall this before?
- 23. What’s different thistime around? Published in 2009 Published in 2004
- 24. What’s different thistime around?
- 25. Microservices Mobile Machine Learning Internetof Things The World has Changed
- 26. Confluents Business Valueper Use Case Improve Customer Experience (CX) Increase Revenue (make money) Business Value Decrease Costs (save money) Core Business Platform Increase Operational Efficiency Migrate to Cloud Mitigate Risk (protect money) Key Drivers Strategic Objectives (sample) Fraud Detection IoT sensor ingestion Digital replatforming/ Mainframe Offload Connected Car: Navigation & improved in-car experience: Audi Customer 360 Simplifying Omni-channel Retail at Scale: Target Faster transactional processing / analysis incl. Machine Learning / AI Mainframe Offload: RBC Microservices Architecture Online Fraud Detection Online Security (syslog, log aggregation, Splunk replacement) Middleware replacement Regulatory Digital Transformation Application Modernization: Multiple Examples Website / Core Operations (Central Nervous System) The [Silicon Valley] Digital Natives; LinkedIn, Netflix, Uber, Yelp... Predictive Maintenance: Audi Streaming Platform in a regulated environment (e.g. Electronic Medical Records): Celmatix Real-time app updates Real Time Streaming Platform for Communications and Beyond: Capital One Developer Velocity - Building Stateful Financial Applications with Kafka Streams: Funding Circle Detect Fraud & Prevent Fraud in Real Time: PayPal Kafka as a Service - A Tale of Security and Multi-Tenancy: Apple Example Use Cases $↑ $↓ $ Example Case Studies (of many)
- 27. How do weget there? Why Apache Kafka and Confluent?
- 33. à It isNOT a Queue! J
- 46. • Robustness andscalability • Slow consumers • Reprocessing
- 48. • Strict Ordering •Persistence • Exactly-Once Processing
- 58.
- 66. Apache Kafka +Kafka Connect + Kafka Streams + many additional components
- 68. 6969 Kafka Clients Apache KafkaNative Clients Confluent Native Clients Community Supported Clients Proxy http/REST stdin/stdout
- 69. 7070 Confluent REST Proxy RESTProxy Non-Java Applications Native Kafka Java Applications Schema Registry REST / HTTP(S) Talking to Non-native Kafka Apps and Outside the Firewall
- 70. 7171 Confluent Schema Registry Elastic HDFS ExampleConsumers Serializer App 1 Serializer App 2 ! Kafka Topic! Schema Registry Deserializer Deserializer
- 71. KSQL – AStreaming SQL Engine for Apache Kafka
- 72. KSQL – AStreaming SQL Engine for Apache Kafka CREATE STREAM vip_actions AS SELECT userid, page, action FROM clickstream c LEFT JOIN users u ON c.userid = u.user_id WHERE u.level = 'Platinum'; CREATE TABLE possible_fraud AS SELECT card_number, count(*) FROM authorization_attempts WINDOW TUMBLING (SIZE 5 MINUTES) GROUP BY card_number HAVING count(*) > 3;
- 74. Apache Kafka: TheRise of a Streaming Platform The Log ConnectorsConnectors Producer Consumer Streaming Engine Add-ons Schema Registry REST Proxy KSQL Etc.
- 75. Event-Streaming Maturity Model/ Adoption Journey Pre-Streaming Streaming Awareness and Pilot Early Production Streaming Mission Critical, Integrated Streaming Global Streaming Central Nervous System
- 76. Free material forManagers, Project lead, Architects, Developers
- 77. 85 Kai Waehner Technology Evangelist kontakt@kai-waehner.de @KaiWaehner www.confluent.io www.kai-waehner.de LinkedIn Questions?Feedback? Please contact me! Download Apache Kafka & Confluent Platform confluent.io/download