Downloaded 54 times
Uploaded byHazelcast
OrientDB & Hazelcast: In-Memory Distributed Graph Database
The document discusses OrientDB's transition from a master/slave replication architecture to a more scalable multi-master, sharded architecture, highlighting the need for distributed databases to handle both read and write throughput. It emphasizes the evaluation of different technologies, ultimately selecting Hazelcast for its robust features, including auto-discovery and easy API integration. The future roadmap includes performance improvements and enhanced conflict resolution strategies, aimed at facilitating better scalability and reliability.
OrientDB & Hazelcast: In-Memory Distributed Graph Database
- 1. + Copyright (c)- Orient Technologies LTD 1
- 2. Agenda What isOrientDB Master-Slave replication New Distributed Architecture goals Why we choose Hazelcast Multi-Master replication features Future roadmap
- 3. What is ? Copyright (c) - Orient Technologies LTD 3
- 4. Best of bothworlds Graph Document Performance Flexibility Copyright (c) - Orient Technologies LTD 4
- 5. Complex Domains Person name surname phone email Customer Provider type delivery discount True Polymorphism Copyright (c) - Orient Technologies LTD 5
- 6. ACID Transactions db.begin(); try { … db.commit(); } catch( Exception e ) { db.rollback(); } Eventually consistency is cool, but many times you need full ACID transactions distributed across servers Copyright (c) - Orient Technologies LTD 6
- 7. SQL + extensions Everybody knows SQL We’ve just added a few new operators and functions to support Tree and Graphs concepts
- 8. HTTP + JSONnative support HTTP + JSON OrientDB Server (C)lient { “@class”: “VIP”, “name”: “Jay”, “surname”: “Miner”, “invented”: “Amiga” } Copyright (c) - Orient Technologies LTD 8
- 9. Commercial Friendly Apachelicense FREE for any usage ! We like to keep it simple for users: no (A)GPL Copyright (c) - Orient Technologies LTD 9
- 10. Is OrientDB a Distributed DBMS ? Copyright (c) - Orient Technologies LTD 10
- 11. Short history ! In 2012, we had a Master/Slave replication ! While it scaled up well on reads, users complained of a single Master node bottleneck It’s quite easy to scale up reads, the hard part is to scale up both reads and writes Copyright (c) - Orient Technologies LTD 11
- 12. How Master/Slave works Copyright (c) - Orient Technologies LTD 12 C C C Master Node Slave Node Slave Node Writes Master node is the bottleneck
- 13. Master/Slave ! PROS: - Relatively easy to develop ! CONS: - The master is the bottleneck for writes - No matter how many servers you have, the throughput is limited by the Master node Copyright (c) - Orient Technologies LTD 13
- 14. What happened toOrientDB's M/S architecture? This is the old MASTER/SLAVE replication Copyright (c) - Orient Technologies LTD 14
- 15. 2012: new architecturalgoals Multi-Master: all the nodes must accept writes Sharding: split data in multiple partitions Better Fail-Over Simplified configuration with Auto-Discovery Copyright (c) - Orient Technologies LTD 15
- 16. How to achievethis? We evaluated different approaches such as: Akka (actor model) JBoss Infinispan Queue Engines: ActiveMQ, RabbitMQ, ZeroMQ and Hazelcast, the winner! Copyright (c) - Orient Technologies LTD 16
- 17. Why ? ! - It has everything you need to build distributed software: - Auto-discovery - RPC - Synchronization - Queues and Topics - Embeddable - Easy API, good documentation - Stable in terms of: - Run-time - API changes - Mature product with many users in production - Same license as OrientDB: Apache2 Copyright (c) - Orient Technologies LTD 17
- 18. Auto-Discovery C Master Node I’m the only one! Copyright (c) - Orient Technologies LTD 18
- 19. Auto-Discovery Connected! C Master Node Master Node Copyright (c) - Orient Technologies LTD 19
- 20. Clients see thedistributed configuration C Master Node updated distributed configuration is broadcasted to all the connected clients Master Node Copyright (c) - Orient Technologies LTD 20
- 21. Auto-reconnect in caseof failure In case of failure, the clients auto-reconnect to C C the available nodes Master Node Master Node Copyright (c) - Orient Technologies LTD 21
- 22. Auto-deploy of databases automatically deployed C to the new joining Master Node C Master Node DB are nodes C C DB DB Copyright (c) - Orient Technologies LTD 22
- 23. DEMO Copyright (c)- Orient Technologies LTD 23
- 24. Classes rely onCluster to store records 1 class -> 1 cluster Class Customer customer By default Cluster Copyright (c) - Orient Technologies LTD 24
- 25. Classes can besplit into more clusters Customer customer_usa Class multiple clusters and assign them to customer_china Define each node Cluster Cluster customer_europe Cluster Copyright (c) - Orient Technologies LTD 25
- 26. Assign 1 clusterper Node Master Node Customer Master Node Master Node customer_usa customer_europe customer_china Copyright (c) - Orient Technologies LTD 26
- 27. Copyright (c) -Orient Technologies LTD What about sharing + replication? ! We used a solution similar to RAID for HardDrives 27
- 28. RAID for databases Replica factor = 2 Master Node Customer Master Node Master Node customer_usa customer_europe customer_china customer_china customer_usa customer_europe Copyright (c) - Orient Technologies LTD 28
- 29. RAID for databases Replica factor = 3 Master Node Master Node Each node owns all customers Master Node customer_usa customer_europe customer_china customer_customer_china usa customer_europe customer_europe customer_china customer_usa Copyright (c) - Orient Technologies LTD 29
- 30. Replication: under thehood Client sends an INSERT request HZ Queue Requests Master Node HZ Queue Master Node HZ Queue Master Node C INSERT Copyright (c) - Orient Technologies LTD 30
- 31. Replication: under thehood HZ Queue Response handling Requests Master Node HZ Queue Master Node HZ Queue WriteQuorum = 2 Sends OK Master Node C HZ Queue HZ Queue HZ Queue OK Responses Copyright (c) - Orient Technologies LTD 31
- 32. Replication: under thehood Fix the unaligned node HZ Queue Requests Master Node HZ Queue Master Node HZ Queue Master Node HZ Queue HZ Queue HZ Queue Responses Fix Copyright (c) - Orient Technologies LTD 32
- 33. DEMO Copyright (c)- Orient Technologies LTD 33
- 34. Linear and Elasticscalability C Master Node C on both read & writes! Master Node C C Master Node C C C C Master Node C C C C Master Node C C C Master Node C C C Master Node C C Copyright (c) - Orient Technologies LTD 34
- 35. Hazelcast’s role Auto-Discovering(Multicast/TCP-IP/Amazon) Queues for requests and responses Store metadata in distributed Maps Distributed Locks Copyright (c) - Orient Technologies LTD 35
- 36. OrientDB’s Future Roadmap OrientDB 2.0 (Sept 2014) has even better performance: +300% improvement on all the distributed operations Pluggable conflict resolution strategy Auto-discovery also by Clients Copyright (c) - Orient Technologies LTD 36
- 37. Copyright (c) -Orient Technologies LTD 37
- 38. Thanks! Luca Garulli CEO Orient Technologies LTD @lgarulli Any Questions? Copyright (c) - Orient Technologies LTD 38