Running Persistent Data in a Multi-Cloud Architecture

Running Persistent Data in a Multi-Cloud Architecture

• 1.
  Confidential │ ©2019VMware, Inc. Running Persistent Data in a Multi-Cloud Architecture Considerations for data modernization Judy Wang Product Manager, VMware Tanzu SQL Aditya Tripathi Product Manager, VMware Tanzu Data Services
• 2.
  Confidential │ ©2019VMware, Inc. 2 Agenda Confidential │ ©2020 VMware, Inc. Defining Multi-Cloud Multi-Cloud Today How to approach running data on multi-cloud Examples
• 3.
  Confidential │ ©2019VMware, Inc. 3Confidential │ ©2020 VMware, Inc. A Multi-cloud Architecture distributes applications and workloads across multiple cloud computing environments.
• 4.
  Confidential │ ©2019VMware, Inc. 4 Running stateless workloads on multi-cloud is a tractable problem Multi-Cloud Today LB GKE (us-east1) GCP (us-west1) GCP (asia-east1) On-Premise
• 5.
  Confidential │ ©2019VMware, Inc. 5 Stateful workloads are left behind Multi-Cloud Today LB GCP (us-west1) GCP (asia-east1) iSeries DB2 On-Premise Oracle GKE (us-east1)
• 6.
  Confidential │ ©2019VMware, Inc. 6 Moving stateful workloads to multi-cloud is complicated Multi-Cloud Today LB GCP (us-west1) GCP (asia-east1)GKE (us-east1)On-Premise ??? ??? ??? ??? ???
• 7.
  Confidential │ ©2019VMware, Inc. 7 Identify Limitations Identify Business Outcomes Brainstorm Solutions Prioritize Tradeoffs Confidential │ ©2020 VMware, Inc. Theory of constraints approach How to Approach Running Data on Multi-Cloud
• 8.
  Confidential │ ©2019VMware, Inc. 8 ● Disaster Recovery - My apps can recover quickly when a hurricane hits ● High Availability - My apps are always available and serving customers ● Geo-query response time - My apps must achieve a certain level of performance ● Handling bursty traffic load - My apps must efficiently handle high traffic periods ● Data sovereignty - My apps must process data in accordance with laws ● … Why does my business need multi-cloud? Identify Business Outcomes
• 9.
  Confidential │ ©2019VMware, Inc. 9 ● Business or Organizational Factors ○ Budget - What can I spend on infrastructure/platform? ○ Influence factors - Leaders, lines of business, organisations that have influence? ○ Skills and Expertise - Does my team know how to use/manage Kafka? ○ Compliance - Legal obligations that apps must abide by? ● Technical Factors ○ Networking limitations - No connectivity between my data centers ○ Hardware limitations - We don’t have data centers in the right regions ○ Heterogeneous app requirements - App 1 needs a document store, App 2 needs transactional data, App 3 needs to handle large queries, … ○ Managing legacy data platforms - All data still needs to be eventually persisted in mainframe. Consider starting with more greenfield things? What factors am I bound by? Identify Limitations
• 10.
  Confidential │ ©2019VMware, Inc. 10 ● Data consistency vs. Data availability - quick recovery or continuity? ● Maintenance & management trade-offs ○ Complex data solutions == more management woes ○ Capacity planning requirements go away in public cloud ○ Performing failovers by hand (active-passive) or automatically (active-active) ● Vendor trade-offs ○ Services marketplace selection in public clouds ○ Redundancy ■ do you need this for all your apps? ○ Lock-in ■ switching cost and data ownership What do I want to optimize for? Prioritize Trade-offs
• 11.
  Confidential │ ©2019VMware, Inc. 11 ● Data replication options ○ Active-active vs active-passive ○ Near-near-far ○ Sync vs async ● Cloud footprint ○ All on-prem vs. all public cloud vs. public-private hybrid ● Data technologies ○ DB caching and replication (Gemfire) ○ Messaging - inherently async ○ Event logs (Kafka) - good for data transformation, heterogeneous systems ○ Hosted highly distributed data solutions (Spanner, Aurora) What combination of tools and services help me achieve my goals? Brainstorm Solutions
• 12.
  Confidential │ ©2019VMware, Inc. 12 ● Primary Business Outcome: Disaster Recovery (When there is a datacenter-wide outage, the system must recover within 20 min and lose no more than 4hrs of data.) ● Limitations: ○ Team experience - Fairly inexperienced, want to minimize operational responsibility ○ Hardware limitations - Only have one private data center ● Trade-offs 1. Data consistency above all else 2. Minimize management complexity/overhead 3. Availability -- 20 min of downtime is OK 4. Query performance -- sub-second query response is not necessary Business outcomes, Limitations, and Tradeoffs Example #1 - Traditional Disaster Recovery
• 13.
  Confidential │ ©2019VMware, Inc. 13 Potential Architecture Example #1 - Traditional DR On-Prem Public Cloud mysql mysql mysql mysql mysql mysql data replication
• 14.
  Confidential │ ©2019VMware, Inc. 14 ● Primary Business Outcome: Apps need to be geographically co-located with the point of sale location. ● Limitations: ○ Hardware limitations - PoS locations need to run on lighter, bare-metal environments ○ Network connectivity - PoS locations may periodically lose network connectivity to central environment ○ Inertia of legacy data platforms - All transactions must be stored in the System of Record mainframe ● Trade-offs 1. Availability -- the system must not be unavailable, as customers are waiting at the cash register 2. Query performance -- sub-second query responses are required, as customers are waiting at the cash register 3. Data consistency -- fairly important but OK as long as transactions eventually make it to the centralized system Business outcomes, limitations, and tradeoffs Example #2 - Point of Sale
• 15.
  Confidential │ ©2019VMware, Inc. 15 Nightly cron batch jobs Central prioritise business outcome + tradeoffs + limitations Example #2 - Point of Sale SoR Edge 1 Edge 2 Edge 3
• 16.
  Confidential │ ©2019VMware, Inc. 16 Public Cloud (AWS, GCP, Azure,...) Central prioritise business outcome + tradeoffs + limitations Example #2 - Point of Sale SoR Edge 1 Edge 2 Edge 3
• 17.
  Confidential │ ©2019VMware, Inc. 17 Public Cloud (AWS, GCP, Azure,...) Central prioritise business outcome + tradeoffs + limitations Example #2 - Point of Sale SoR Edge 2 Edge 3 Edge 1
• 18.
  Confidential │ ©2019VMware, Inc. Please consult an expert if necessary!☺
• 19.
  Confidential │ ©2019VMware, Inc. Thank You Judy Wang juwang@vmware.com Aditya Tripathi tripathiad@vmware.com