Mastering MongoDB on Kubernetes, the power of operators Arek Borucki Beamery
Agenda • MongoDB architecture • Kubernetes Operators • MongoDB Community K8s Operator • MongoDB Enterprise K8s Operator • MongoDB Atlas K8s Operator • Demo
OM java application • consistent backup • rolling upgrades • automation • monitoring
Agenda • MongoDB clusters • Serverless • Data Lake • Atlas Search • The Data API (HTTPS) • Advanced security • and more
Why to use MongoDB K8s Operators ? Capabilities like Replication, Sharding, Failover or Scalability can help more easily deal with the uncertainty sidecar pattern- MongoDB Agent handles configuring, stopping, restarting and upgrading the mongod process, readiness and liveness probes Provides out of the box security features like TLS, authentication, RBAC Reduce complexity - uses the best human knowledge of creating and operating a MongoDB products Extending Kubernetes API by MongoDB specific objects, like OpsManager or ShardedCluster Uses K8s best practice - Affinity, Pod Disruption Budget, Resource management and K8s reliability features like auto healing
The Operator tell K8s how to configure MDB cluster including provisioning storage, setting up the network connections, and configuring other resources
The Kubernetes operator watches Custom Resources _ Observe the actual state _ Analyse difference from desired state _ Take action _ Drive the actual state towards the desired stare _ Repeat! Observe Analyse Act my-replica set has 2 running pods Differs from desired state of 3 replicas (pods) Action: - Start a pod - Add pod to the cluster - Waiting for replication
• The Operator is Apache open source • Currently only Replica Set • Suited for experimentation and testing • Automated cluster provisioning • Client-to-server, server-to-server with TLS • Scale replica sets up and down • Automated rolling updates • Provides metrics target that can be used with Prometheus MongoDB Community Kubernetes Operator
1 1 custom resource • 1 1 --- apiVersion: mongodbcommunity.mongodb.com/v1 kind: MongoDBCommunity metadata: name: mongodb-replica-set namespace: mongodb spec: members: 3 type: ReplicaSet version: “5.0.5" security: authentication: modes: ["SCRAM"]
• Replica Sets, Sharded Clusters, Standalones • AAA, TLS for all traffic between servers and between clients and servers • Ops Manager - monitoring and management platform for MongoDB • Backup Infrastructure (PIT Restore, consistent backup of shard clusters) • User management support • Encryption at Rest, • Multi-Cluster Capabilities MongoDB Enterprise Kubernetes Operator
MongoDB Custom Resource types 1 4
1 5 custom resources • 1 5 --- apiVersion: mongodb.com/v1 kind: MongoDBOpsManager metadata: name: ops-manager spec: version: 5.0.5 externalConnectivity: type: LoadBalancer applicationDatabase: members: 3 version: 4.0.5-ent --- apiVersion: mongodb.com/v1 kind: MongoDB metadata: name: sharded-cluster spec: shardCount: 2 mongodsPerShardCount: 3 version: 5.0.5 type: ShardedCluster
MongoDBUser custom resource 1 6
1 7 • 1 7 --- apiVersion: mongodb.com/v1 kind: MongoDBMulti metadata: name: multi-replica-set spec: version: 5.0.5 type: ReplicaSet clusterSpecList: clusterSpecs: clusterName: - cluster1.mongokubernetes.com members: 2 - cluster2.mongokubernetes.com members: 1 - cluster2.mongokubernetes.com members: 1
1 8
• Manage Atlas clusters in directly K8s • Automated Atlas cluster provisioning • Integrate Atlas with Kubernetes-native CI/CD pipelines • Setup Cloud backup • Database users management Atlas Kubernetes Operator
2 0 custom resources • 2 0 --- apiVersion: atlas.mongodb.com/v1 kind: AtlasProject metadata: name: my-project spec: name: Test Atlas Operator Project projectIpAccessList: - ipAddress: "192.0.2.15” comment: "IP address for App Server A" --- apiVersion: atlas.mongodb.com/v1 kind: AtlasDeployment metadata: name: my-atlas-deployment spec: projectRef: name: my-project deploymentSpec: name: test-deployment providerSettings: instanceSizeName: M10
2 1 custom resources • 2 1 --- apiVersion: atlas.mongodb.com/v1 kind: AtlasBackupPolicy metadata: name: atlas-backup-policy spec: items: frequencyType: "weekly" frequencyInterval: 1 retentionUnit: "days" retentionValue: 7 --- apiVersion: atlas.mongodb.com/v1 kind: AtlasDatabaseUser metadata: name: my-database -user spec: roles: - roleName: "readWriteAnyDatabase” databaseName: "admin" projectRef: name: my-project username: the-user
• Operators reduce complexity • Extends K8s API by MongoDB objects • Improve consistency • Automate at the level you need • The operator creates persistent volumes and STS for MongoDB Replica Set • Recommendedn method of running databases on K8s Summary

Mastering MongoDB on Kubernetes, the power of operators

  • 1.
    Mastering MongoDB on Kubernetes,the power of operators Arek Borucki Beamery
  • 2.
    Agenda • MongoDB architecture •Kubernetes Operators • MongoDB Community K8s Operator • MongoDB Enterprise K8s Operator • MongoDB Atlas K8s Operator • Demo
  • 4.
    OM java application •consistent backup • rolling upgrades • automation • monitoring
  • 5.
    Agenda • MongoDB clusters •Serverless • Data Lake • Atlas Search • The Data API (HTTPS) • Advanced security • and more
  • 7.
    Why to useMongoDB K8s Operators ? Capabilities like Replication, Sharding, Failover or Scalability can help more easily deal with the uncertainty sidecar pattern- MongoDB Agent handles configuring, stopping, restarting and upgrading the mongod process, readiness and liveness probes Provides out of the box security features like TLS, authentication, RBAC Reduce complexity - uses the best human knowledge of creating and operating a MongoDB products Extending Kubernetes API by MongoDB specific objects, like OpsManager or ShardedCluster Uses K8s best practice - Affinity, Pod Disruption Budget, Resource management and K8s reliability features like auto healing
  • 8.
    The Operator tellK8s how to configure MDB cluster including provisioning storage, setting up the network connections, and configuring other resources
  • 9.
    The Kubernetes operatorwatches Custom Resources _ Observe the actual state _ Analyse difference from desired state _ Take action _ Drive the actual state towards the desired stare _ Repeat! Observe Analyse Act my-replica set has 2 running pods Differs from desired state of 3 replicas (pods) Action: - Start a pod - Add pod to the cluster - Waiting for replication
  • 10.
    • The Operatoris Apache open source • Currently only Replica Set • Suited for experimentation and testing • Automated cluster provisioning • Client-to-server, server-to-server with TLS • Scale replica sets up and down • Automated rolling updates • Provides metrics target that can be used with Prometheus MongoDB Community Kubernetes Operator
  • 11.
    1 1 custom resource • 1 1 --- apiVersion:mongodbcommunity.mongodb.com/v1 kind: MongoDBCommunity metadata: name: mongodb-replica-set namespace: mongodb spec: members: 3 type: ReplicaSet version: “5.0.5" security: authentication: modes: ["SCRAM"]
  • 13.
    • Replica Sets,Sharded Clusters, Standalones • AAA, TLS for all traffic between servers and between clients and servers • Ops Manager - monitoring and management platform for MongoDB • Backup Infrastructure (PIT Restore, consistent backup of shard clusters) • User management support • Encryption at Rest, • Multi-Cluster Capabilities MongoDB Enterprise Kubernetes Operator
  • 14.
  • 15.
    1 5 custom resources • 1 5 --- apiVersion:mongodb.com/v1 kind: MongoDBOpsManager metadata: name: ops-manager spec: version: 5.0.5 externalConnectivity: type: LoadBalancer applicationDatabase: members: 3 version: 4.0.5-ent --- apiVersion: mongodb.com/v1 kind: MongoDB metadata: name: sharded-cluster spec: shardCount: 2 mongodsPerShardCount: 3 version: 5.0.5 type: ShardedCluster
  • 16.
  • 17.
    1 7 • 1 7 --- apiVersion: mongodb.com/v1 kind:MongoDBMulti metadata: name: multi-replica-set spec: version: 5.0.5 type: ReplicaSet clusterSpecList: clusterSpecs: clusterName: - cluster1.mongokubernetes.com members: 2 - cluster2.mongokubernetes.com members: 1 - cluster2.mongokubernetes.com members: 1
  • 18.
  • 19.
    • Manage Atlasclusters in directly K8s • Automated Atlas cluster provisioning • Integrate Atlas with Kubernetes-native CI/CD pipelines • Setup Cloud backup • Database users management Atlas Kubernetes Operator
  • 20.
    2 0 custom resources • 2 0 --- apiVersion:atlas.mongodb.com/v1 kind: AtlasProject metadata: name: my-project spec: name: Test Atlas Operator Project projectIpAccessList: - ipAddress: "192.0.2.15” comment: "IP address for App Server A" --- apiVersion: atlas.mongodb.com/v1 kind: AtlasDeployment metadata: name: my-atlas-deployment spec: projectRef: name: my-project deploymentSpec: name: test-deployment providerSettings: instanceSizeName: M10
  • 21.
    2 1 custom resources • 2 1 --- apiVersion:atlas.mongodb.com/v1 kind: AtlasBackupPolicy metadata: name: atlas-backup-policy spec: items: frequencyType: "weekly" frequencyInterval: 1 retentionUnit: "days" retentionValue: 7 --- apiVersion: atlas.mongodb.com/v1 kind: AtlasDatabaseUser metadata: name: my-database -user spec: roles: - roleName: "readWriteAnyDatabase” databaseName: "admin" projectRef: name: my-project username: the-user
  • 22.
    • Operators reducecomplexity • Extends K8s API by MongoDB objects • Improve consistency • Automate at the level you need • The operator creates persistent volumes and STS for MongoDB Replica Set • Recommendedn method of running databases on K8s Summary

Editor's Notes

  • #10 All of this is made possible by our robust global operations. We designed, own and operate the world’s largest Data Processing center dedicated to travel. This world-class data center is high-performance and highly-resilient, capable of processing billions of transactions involving sensitive data, securely and reliably, every day. Our global operations are supported by more than 900 employees from more than 45 nationalities and is based on a follow-the-sun model, with specialist support groups in Germany, the US, Australia, India and the United Kingdom.