Downloaded 32 times
Uploaded byStratio
“A Distributed Operational and Informational Technological Stack”
This document describes a distributed operational and informational technological stack. It provides a unique datacentric suite that includes a multidatastore for operational and analytical applications, data fusion and intelligence layers, and the Stratio EOS platform. The roadmap focuses on further developing the multidatastore, data intelligence capabilities like artificial intelligence, and security and governance functions.
“A Distributed Operational and Informational Technological Stack”
- 1. A DISTRIBUTED OPERATIONAL & INFORMATIONAL TECHNOLOGICALSTACK JUNE 28, 2017 Our transformational product uses the third generation of Big Data technologies to execute the most comprehensive form of Digital Transformation
- 2. STRATIO DATACENTRIC ● UniqueData ● Datacentric Suite 1 INDEX 4 VALUE PROPOSITION ● Multidatastore ● Operations & Analytics ● Stratio EOS 2 5 3 ROADMAP - BACKLOG ● Data Fusion Layer ● Multidatastore MODULES ● Stratio EOS ● EOS Universe ● Security & Governance ARTIFICIAL INTELLIGENCE ● Distributed VS Non-Distributed ● Benchmarks ● Architecture
- 3.
- 4. STRATIO DATACENTRIC Data isbrought together in a central place, ridding companies of unnecessary duplication and inconsistencies across departments, systems and applications. Individual applications no longer own the data. It is unique and connected to all the applications around it. DATA Mobile APP Campaign Management Digital Marketing Legacy Applications Call center ERP ATG TPV APP CRM E-commerce Unique data 4
- 5. DATA Mobile APP Campaign Management Digital Marketing Legacy Applications Callcenter ERP ATG TPV APP CRM E-commerce Microservices of the Data Intelligence layer New Applications are developed through microservice orchestration reducing code by half Unique data at the center and applications around it using it in real time with maximum intelligence Data intelligence Api Daas Operational Applications use the microservices of the Data as a Service layer Microservices Data-centric STRATIO DATACENTRIC 5 Microservices
- 6. DATA VISUALIZATION DATAINTELLIGENCE DATA AS A SERVICE DATA FUSION LAYER MULTIDATASTORE MULTIPROCESSING SECURITY & GOVERNANCE STRATIO EOS (DC/OS PAAS) EOS UNIVERSE Turn difficult concepts into something simple Become a truly data-driven company Easily access your data from a centralized spot Gain a centralized vision of all your data and easily govern its access and management Merge your operational and informational business intelligence Manage your resources as if on a single super-giant computer Protect your data against security breaches and maintain compliance Deploy all your services with a single click
- 7.
- 8. STRATIO DATA CENTRICIS MULTIDATASTORE ….. VALUE PROPOSITION: MULTIDATASTORE 8 + Lucene Index To cover operational applications or Use Cases you need operational Data Stores To cover Analytical Applications or Use cases you need an analytical Data Stores Stratio takes advantage of the specialization of Big Data and NoSQL DataStores to provide the best data store for each use case
- 9. VALUE PROPOSITION: StratioEOS (DC/OS-Mesos based PAAS) 9 ● DC/OS-Mesos based ● Microservices management ● Manage any process, job, or service ● Manage any application type: monolitics, microservices, dockers
- 10.
- 11.
- 12. Crossdata EOS Data Intelligence & Real -time StratioEOS Multidatastore
- 13. Crossdata EOS MARATHON CONSUL DOCKER ANSIBLE MESOS TERRAFORM DC/OS METRONOME ZOOKEEPER EOS BACK Data Intelligence & Real-time Stratio EOS Multidatastore
- 14. Crossdata EOS MARATHON CONSUL DOCKER ANSIBLE MESOS TERRAFORM DC/OS METRONOME ZOOKEEPER EOS BACK Data Intelligence & Real-time Multidatastore Stratio EOS
- 15. Crossdata EOS MARATHON CONSUL DOCKER ANSIBLE MESOS TERRAFORM DC/OS METRONOME ZOOKEEPER EOS BACK Data Intelligence & Real-time Multidatastore Stratio EOS
- 16. Crossdata EOS MARATHON CONSUL DOCKER ANSIBLE MESOS TERRAFORM DC/OS METRONOME ZOOKEEPER EOS BACK Data Intelligence & Real-time Multidatastore Stratio EOS
- 17. Crossdata EOS MARATHON CONSUL DOCKER ANSIBLE MESOS TERRAFORM DC/OS METRONOME ZOOKEEPER EOS BACK Data Intelligence & Real-time Multidatastore Stratio EOS
- 18. Crossdata EOS MARATHON CONSUL DOCKER ANSIBLE MESOS TERRAFORM DC/OS METRONOME ZOOKEEPER EOS BACK Data Intelligence & Real-time Multidatastore Stratio EOS
- 19.
- 20.
- 21. EOS Universe: DataDiscovery & Viewer Data Intelligence & Real -time Crossdata EOS Multidatastore
- 22. EOS Universe: DataDiscovery & Viewer Data Intelligence & Real -time Crossdata EOS Multidatastore
- 23. EOS Universe: DataDiscovery & Viewer Data Intelligence & Real -time Crossdata EOS Multidatastore
- 24. EOS Universe: Sparta Data Intelligence &Real -time Crossdata EOS Sparta Sparta user interface is ready to let you work without writing any line of code. Ready-to-use Inputs and Outputs Extract, Transform and Load (ETL) Powerful Aggregation Operators Batch Data + Streaming Data and Simple Event Processing Full SQL language mixing streaming data and batch data with triggers. Cluster Deployment (Mesos, YARN, Spark Standalone) Fully secured dataflow end-to-end Multidatastore
- 25. EOS Universe: Sparta Data Intelligence &Real -time Crossdata EOS Sparta Multidatastore
- 26. EOS Universe: Sparta Data Intelligence &Real -time Crossdata EOS Sparta Multidatastore
- 27. EOS Universe: Sparta Data Intelligence &Real -time Crossdata EOS Sparta Multidatastore
- 28. EOS Universe: Intelligence Crossdata EOS Intelligence StratioIntelligence is the Data Intelligence layer of the Data-centric architecture. The main milestones of Stratio Intelligence are: Big Data Science development environment for every data scientist (Scala, R and Python) Integration of Open Source distributed ML libraries Real-time decision making with the trained models Integration with the Stratio Platform Deep Learning Data Intelligence & Real -time Multidatastore
- 29. EOS Universe: Intelligence Crossdata EOS Intelligence StratioIntelligence is the Data Intelligence layer of the Data-centric architecture. The main milestones of Stratio Intelligence are: Big Data Science development environment for every datascientist (Scala, R and Python) Integration of Open Source distributed ML libraries Real-time decision making with the trained models Integration with the Stratio Platform Deep Learning Data Intelligence & Real -time Multidatastore
- 30. EOS Universe: Crossdata CrossdataPowered by Crossdata EOS Data Intelligence & Real -time Multidatastore Stratio Crossdata is a distributed framework and a fast and general-purpose computing system powered by Apache Spark: Database federation Data as a Service with SQL and RESTful
- 31. EOS Universe: Crossdata CrossdataPowered by Crossdata EOS Data Intelligence & Real -time Multidatastore
- 32. EOS Universe: Multidatastore PostgresBig Data Transactional Multi master SQL compliant Basic automatic fail over Continuous monitoring Basic elasticity Multidatastore Crossdata EOS Data Intelligence & Real -time POSTGRES BD CASSANDRA ELASTIC HDFS MONGO DB ….
- 33.
- 34.
- 35. Gosec Security Authentication Authorization Audit logsreal-time processing Security GOSEC SECURITY 35
- 36. SECURITY MANAGEMENT APP Settings Integrationwith existing LDAP Users and Groups Directory Security Policies management SECURITY PLUGINS ON CORE SERVICES Pluggable mechanism involving AAA actions. Several key technologies (HDFS, KAFKA, CROSSDATA…) Security policies rely on distributed backend LIGHTWEIGHT AND DISTRIBUTED BACKEND Based on Apache Zookeeper Secured and fault tolerant storage (long-term) SINGLE SIGN ON Single sign on across web modules in the platform Consistent authentication and session management GOSEC SECURITY
- 37. Gosec Governance Data governancefor Stratio DataCentric Metadata repository Business glossary Lineage Governance GOSEC GOVERNANCE
- 38. BUSINESS GLOSSARY Possibility toassign Business Glossary terms to Metadata. Enhance business-IT collaboration with a common business vocabulary and built-in collaboration GOSEC GOVERNANCE
- 39. BUSINESS GLOSSARY All termsin the Business Glossary are updated by the business and IT Departments. Each term passes through approval flows and contain governance information, Data Quality assurance, Data Steward etc.. GOSEC GOVERNANCE
- 40. LINEAGE Data traceability. Whatis the source, how is it processed, where is it stored? GOSEC GOVERNANCE
- 41.
- 42. GOVERNANCE CROSSDATA SERVER BUSINESS GLOSSARY PostgreSQL HDFS ElasticSearch Kafka TechnicalMetadata Business Metadata Entity Metadata Repository Technical View Akka HTTP/S CROSSDATA SERVER Business View Akka HTTP/S RESTFul APPS DISCOVERY Data Engineer ODBCJDBC Data Fusion Layer R2 R3 R4
- 43.
- 44. DATA CENTER OPERATING SYSTEMMESOS SERVICE ORCHESTATION CONTAINERS NODE PROVISIONING TERRAFORM Kafka Zookeeper VAULT BAREMETAL PUBLIC CLOUD SQL HYBRID CLOUD PRIVATE CLOUD Docker DaaS Apps Apps Microservices Docker Data Intelligence as a Service Microservices Standalone Applications A P P S SERVICE DISCOVERY Metadata (API & Storage) STRATIO EOS (Enterprise Operating System) Microservices Apps with Docker Docker Docker + MARATHON CONSUL DOCKER StratioDataCentric INFRAS
- 45.
- 46. Distributed Query Distributed Query ANALYTICENVIRONMENT Non-Distributed Data Non-Distributed Machine Learning Algorithm DataImpala DataImpala IMPALA IMPALA Machine Learning Algorithm Distributed Query and Non-Distributed Algorithm
- 47. Data Scientist Crossdata lib DataSpark DataSpark CROSSDATASERVER Entity Metadata Repository STRATIO INTELLIGENCE Distributed Query Distributed Query Distributed Machine Learning Algorithm JDBC Distributed Query and Distributed Algorithm
- 48. ... Multiuser Environment Manage users andprovision of notebooks Analytic Environment User 1 front-end User N front-end Kernel Code interpreter Analytic Environment Notebook server User 1 back-end User N back-end ... User code STRATIO INTELLIGENCE ARCHITECTURE
- 49. Version 1 Version 2 Version3 user 1 user 2 user3 Kernels Kernels Kernels STRATIO INTELLIGENCE: MULTI-USER AND MULTI-LANGUAGE
- 50. PostgreSQL HDFS ElasticSearch Kafka STRATIO INTELLIGENCE (unified analyticenvironment) Spark TensorFlow Streaming Low Latency distributed engine Model Repository Spark Machine Learning Models TensorFlow Deep Learning Models Real Time Models Unified Model Unified Model Container API - Model as a Service Data Intelligence R2 R3 R4 MICROSERVICE ARCHITECTURE
- 51.