Download as PDF, PPTX
Uploaded bySafe Software
Introduction to Data Stream Processing
The document discusses the growing importance of data stream processing, highlighting an expected increase in the global streaming analytics market from USD 12.5 billion in 2020 to USD 38.6 billion by 2025. It outlines characteristics of data streams, the benefits of stream processing for various industries, and the role of cloud computing in enhancing these processes. Organizations are encouraged to adopt streaming data integration for improved decision-making and operational efficiency.
Introduction to Data Stream Processing
- 1. Introduction to Data StreamProcessing
- 2. Meet the Presenters DonMurray Stewart Harper
- 3. Agenda ● Market trends ●Overview of data streams ● Overview of stream processing ● Cloud computing and stream processing
- 4.
- 5. It’s time toadopt Stream Data Integration ● The global streaming analytics market size is expected to grow from USD 12.5 billion in 2020 to USD 38.6 billion by 2025. ● More than half of major new business systems will incorporate real-time data and continuous intelligence to improve decisions. ● More and more organizations will use streaming data to support their data integration use cases.
- 6. Why are more businessesleveraging stream processing? ● Explosive growth of Sensors and hardware that collect data. ● Technology to capture, store, and process data continues to improve. ● Actionable information delivered faster to decision makers is a competitive advantage.
- 7. Overview of DataStreams
- 8. Let’s start withthe data ● Bounded data is finite and has a discrete beginning and end. It is associated with batch processing. ● Unbounded data is infinite, having no discrete beginning or end. It is associated with stream processing.
- 9. Characteristics of Data Streams ●Data records are small in size. ● Data volumes can be extremely high. ● Data distribution can be inconsistent with quiet and busy periods. ● Data can arrive out of sequence compared to when the event happened.
- 10. Source: Amazon WebServices Diminishing Value of Data
- 11. What are thesources of Data Streams?
- 12. Common Event Streams BusinessApplications Customer orders Airline reservations Insurance claims Bank transactions Telco call detail records Digital Information Clickstreams Social computing Customer call logs News, weather feeds IT, network logs Market data Email Internet of Things Radio-frequency Identification Telemetry SCADA Geolcoation Machine logs
- 13. Industries benefiting from streamingdata: ● Utilities ● Energy ● Telecommunications ● Transportation ● Commercial ● Government ● Healthcare ● ...
- 14. Business Improvements from Steaming Data: ●Operational efficiency ● Asset management ● Customer experience ● Situational awareness ● Medical care
- 15.
- 16. Poll Question: Is yourorganization leveraging or planning to leverage streaming data?
- 17. Overview of StreamProcessing
- 18. What is StreamProcessing? Stream processing is a term that groups together the collection, integration, and analysis of unbounded data. Stream Processing delivers insights to organizations on a continuous basis.
- 19. High and LowVolume Message Streams There are three ways organizations work with unbounded data. 1. Batch Processing of Stored Data - Unbounded data is stored and processed at specified intervals. 2. Event Processing - Each event in the unbounded stream is handled separately with connections between events being stored in persistent storage. 3. Stream Processing - Continuous processing of high-volume data streams. Data is processed in memory before storing. Real-Time Any Velocity Near Real-time Real-Time Low Velocity
- 20. Batch Processing vsStreaming Processing
- 21. Breaking Up DataStreams for Processing
- 22. Overview of Core StreamProcessing Workflows ● Filtering ● Enriching ● Aggregating ● Event detection
- 23. Filtering Reduce data volumesin memory before committing data to disk.
- 24. Enriching Join the unboundeddata to other datasets (databases, APIs) before committing data to disk.
- 25. Aggregating Summarize the unboundeddata by calculating time-windowed aggregations before committing data to disk.
- 26. Event Detection Detect patternsin memory and then trigger an event when certain criteria are met.
- 27. Geospatial Data andAnalysis in Stream Processing Leveraging real-time geospatial or location data expands the insights delivered to your organization.
- 28. Geospatial Processing ● Filteringwith Geofences ● Proximity Analysis ● Snapping Data to a network ● Calculate Distance
- 29. The Role ofCloud Computing in Stream Processing
- 30. Cloud Services Related toStream Processing ● Data Warehouses ● Data Lakes ● Stream Processing ● IoT Device Connection ● Machine Learning ● Scalable Compute
- 31. Summary Stream Processing deliversreal-time insights to organizations. There is no better time to get started than now! ● Sensors and data collection are exploding. ● Technologies continue to improve. ● All industries can benefit. ● Customer satisfaction will skyrocket.
- 32. See you onApril 8th for part 2. Empowering Real-Time Decision Making with Data Streaming ● Common use cases ● Customer Stories ● How FME supports streaming data ● Demos Register: safe.com/webinars
- 33. Come one, comeall from May 4 - 14! Discover tips & tricks, the newest updates, and innovative ways to use FME. FME World Fair 2021 Register: safe.com/world-fair
- 34. Thank You! Read ourData Streaming blog at safe.com/blog Connect with us for more FME