Download to read offline
![Sample Distributed Siddhi App @source(type = ‘kafka’, …, @map(type = ‘json’)) define stream ProductionStream (name string, amount double, factoryId int); @dist(parallel = ‘4’, execGroup = ‘gp1’) from ProductionStream[amount > 100] select * insert into HighProductionStream ; @dist(parallel = ‘2’, execGroup = ‘gp2’) partition with (factoryId of HighProductionStream) begin from HighProductionStream#window.timeBatch(1 min) select factoryId, sum(amount) as amount group by factoryId insert into ProdRateStream ; end; Filter Source FilterFilterFilter PartitionPartition](https://image.slidesharecdn.com/webinarwso2streamprocessorgraphicaleditorhttpmessagetraceanalyticsandmore-180711063658/75/WSO2-Stream-Processor-Graphical-Editor-HTTP-Message-Trace-Analytics-and-more-34-2048.jpg)
WSO2 Stream Processor: Graphical Editor, HTTP & Message Trace Analytics and more.
- 1. WSO2 Stream Processor: GraphicalEditor, HTTP & Message Trace Analytics and More Sriskandarajah Suhothayan Director, WSO2
- 2. WSO2 Stream Processor Anopen source, cloud native analytics product optimized to create real-time, actionable insights for agile digital businesses. 2
- 3. Stream Processing 3 ● Needto write code ● Complex deployment (5 - 6 nodes) ● Inability to change fast Challenges Source : http://cdn.business2community.com/wp-content/uploads/2012/08/Invest_Money_Photoxpress_2938045.jpg
- 4. WSO2 Stream Processor 4 ●Need to write code ○ Streaming SQL + Graphical Editor ● Complex deployment (5 - 6 nodes) ○ 2 node minimum HA deployment (scale beyond with Kafka) ● Inability to change fast ○ Query templates, editor The solution
- 5.
- 6. • Lightweight andlean • Easy-to-learn streaming SQL with graphical editor • High performance analytics with just 2 nodes (HA) • Native support for streaming machine learning • Long-term aggregations from seconds to years • Highly scalable deployment with exactly-once processing • Tools for development and monitoring • Tools for business users to write their own rules Overview of WSO2 Stream Processor
- 7. Market Recognition ● Namedas a Strong Performer in The Forrester Wave™: Big Data Streaming Analytics, Q1 2016 ● Highest score possible in 'Acquisition and Pricing' criteria, and among second highest scores in 'Ability to Execute' criteria ● The Forrester report notes: “WSO2 is an open source middleware provider that includes a full spectrum of architected-as-one components such as application servers, message brokers, enterprise service bus, and many others. Its streaming analytics solution follows the complex event processor architectural approach, so it provides very low-latency analytics. Enterprises that already use WSO2 middleware can add CEP seamlessly. Enterprises looking for a full middleware stack that includes streaming analytics will find a place for WSO2 on their shortlist as well.” 7
- 8.
- 9.
- 10.
- 11. Graphical Query Editor Drag& drop query builder
- 12. 1. Streaming datapreprocessing and transformation 2. Data store integration 3. Streaming data summarization from seconds to years 4. KPI analysis and alerts 5. Event correlation and trend analysis 6. Real-time prediction and streaming machine learning 7. Service integration 8. Improved JSON processing support Supported Streaming Analytics Patterns
- 13. • Source andSinks – HTTP, Kafka, TCP, Email, JMS, File, RabbitMQ, MQTT, Web-Socket, Twitter • Message Formats – JSON, XML, Text, Binary, Key-value, CSV • Data Stores – RDBMS, Solr, MongoDB, HBase, Cassandra, Elasticsearch Supported connectors
- 14.
- 15.
- 16. • Generate dashboardand widgets • Fine grained permissions – Dashboard level – Widget level – Data level • Localization support • Inter-widget communication • Shareable dashboards with widget state persistence Dashboards
- 17.
- 18.
- 19.
- 20. • Understand systemperformance via – Throughput – Latency – CPU, memory utilizations • Monitor in various scales – Node level – Siddhi app level – Siddhi query level Status Dashboard Monitor resource nodes and Siddhi apps
- 21.
- 22. Self Service &Omniscience
- 23. • Define andmodify rules using config forms • Generate templates with Editor Business Rules for Non Technical Users
- 24. • Build templateusing Graphical UI or JSON schema Create Business Rules Templates
- 25.
- 26. HTTP Analytics Request-Response and erroranalysis over time
- 27. Distributed Message Tracing OpenTracing Support
- 28. Twitter Analytics Word cloudand sentiment analytics
- 29.
- 30. • High performance –Process around 100k events/sec – While most other stream processing systems need around 5+ nodes • Zero downtime • Zero event loss • Simple deployment with RDBMS coordination (no ZooKeeper, Kafka, etc.) • Multi data center support Minimum HA with 2 Nodes Stream Processor Stream Processor Event Sources Dashboard Notification Invocation Data Source Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Event Store
- 31. Distributed Deployment withKafka • Exactly-once processing • Fault tolerance • Highly scalable • No back pressure • Distributed development configurations via annotations • Pluggable distribution options (YARN, K8, etc.)
- 32.
- 33. Distributed Deployment withKafka Data base Event Source Event Sink Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Siddhi App Kafka Topic Kafka Topic Kafka Topic Kafka Topic Kafka Topic
- 34. Sample Distributed SiddhiApp @source(type = ‘kafka’, …, @map(type = ‘json’)) define stream ProductionStream (name string, amount double, factoryId int); @dist(parallel = ‘4’, execGroup = ‘gp1’) from ProductionStream[amount > 100] select * insert into HighProductionStream ; @dist(parallel = ‘2’, execGroup = ‘gp2’) partition with (factoryId of HighProductionStream) begin from HighProductionStream#window.timeBatch(1 min) select factoryId, sum(amount) as amount group by factoryId insert into ProdRateStream ; end; Filter Source FilterFilterFilter PartitionPartition
- 35.
- 36. Success Stories 37 a TFL usedWSO2 real time streaming to create next generation transport systems. Uber detected fraud in real time, processing over 400K events per second CSI uses WSO2 streaming capabilities to integrate people, systems, and things. State of Arizona monitors and manages their Private PaaS in real time to improve efficiencies and trim costs.
- 37. Download and Tryout 38 WSO2Stream Processor http://wso2.com/analytics/ Documentation https://docs.wso2.com/display/SP420
- 38.