September 2018 Free monitoring for MySQL, MariaDB, PostgreSQL and MongoDB Bartłomiej Oleś Presenter bart@severalnines.com
Copyright 2017 Severalnines AB I'm Jean-Jérôme from the Severalnines Team and I'm your host for today's webinar! Feel free to ask any questions in the Questions section of this application or via the Chat box. You can also contact me directly via the chat box or via email: info@severalnines.com during or after the webinar. Your host & some logistics
Copyright 2017 Severalnines AB
Copyright 2017 Severalnines AB Free to download Initial 30 days Enterprise trial Converts into free Community Edition Enterprise / paid versions available
Copyright 2017 Severalnines AB Automation & Management Deployment (Free Community) ● Deploy a Cluster in Minutes ○ On-Prem ○ Cloud (AWS/Azure/Google) - paid Monitoring (Free Community) ● Systems View with 1 sec Resolution ● DB / OS stats & Performance Advisors ● Configurable Dashboards ● Query Analyzer ● Real-time / historical Management (Paid Features) ● Backup Management ● Upgrades & Patching ● Security & Compliance ● Operational Reports ● Automatic Recovery & Repair ● Performance Management ● Automatic Performance Advisors
Copyright 2017 Severalnines AB Supported Databases
Copyright 2017 Severalnines AB Our Customers
Poll - Which of the following free monitoring tools does your currently company use? Copyright 2018 Severalnines AB (choose many) ● Traditional (Nagios, Zabbix, ...) ● Saas based monitoring (MMS, ...) ● Specialized (PMM, MySQL OpsPack, ...) ● ClusterControl ● Other
Copyright 2017 Severalnines AB Copyright 2018 Severalnines AB Monitoring distributed database systems - Requirements
Most databases are somehow distributed today ● Services running on multiple hosts ○ Replication ○ Sharding ○ Clustering ● Multiple Data Centers ○ Cloud and/or On-prem ○ Disaster Recovery ● Load balancing and Single point of contact IP ○ For workload management, HA, query caching... ○ E.g., HAProxy, KeepAlived/VIP, ProxySQL, MaxScale
Why we need a database monitoring system ● Data is a key asset of the organisation ● Databases are important as they manage the source of truth ● Database is complex - IO, transaction engine, query optimizer, caches, locks, versioning,... ● Very dependent on OS, IO subsystems, network ● Distribution across multiple instances makes it even more complex ● Good database monitoring helps make sense of all that
A “good” monitoring system involves ● Collection, Correlation ● Presentation ● Interpretation/analysis ● Interactions between concurrently executing processes ● Integration with other platforms
Who uses database monitoring ● Managers ○ To improve project implementation ○ Keep track on internal or external SLA’s ○ Track team performance ○ Budgeting ○ Monitor KPIs ● Business ○ To inform and improve future programmes and projects ○ Inform stakeholders ● DBA, DevOps, SysOps ○ Prevent and event ○ Query caching ○ DB Firewall
Support for multiple DB platforms
Monitoring is not management
What is management? Included in the free ClusterControl Community version
Copyright 2017 Severalnines AB Copyright 2018 Severalnines AB Cloud-based vs On-prem monitoring solutions
Comparing cloud-based vs on-prem Cloud-based monitoring ● Benefits ○ Quick and easy configuration ○ Managing your solution over time is also easier ○ Remote access from any location or device with browsing capabilities ● Drawbacks ○ Security ○ Reliability and stability On-prem monitoring ● Benefits ○ A level of control that cloud-based solutions just can't provide ○ Performance reliability ○ Control over data, security ● Drawbacks ○ Installation and setup ○ Data access
Cloud monitoring
Poll - TBD Copyright 2018 Severalnines AB Which databases do you currently monitor? ● MySQL ● MariaDB ● PostgreSQL ● MongoDB ● Other
Copyright 2017 Severalnines AB Copyright 2018 Severalnines AB Agent-based vs Agentless
Agentless vs Agent-based monitoring Agentless Agent-based Deployment Software on ClusterControl host only. Up and running within minutes Agent installed on each monitored DB server Admin. Overhead. Only CC needs to be maintained Agents need to be maintained, updated, restarted in case of failures Configuration Centralized Decentralized Security Controller to SSH into managed DBs, requires certain privileges. More secure. Agent communication to DB/OS is internal to the server. No additional firewall rules. Network Overhead Additional traffic with more connections Local data collection, results processed and then shipped to the server Metrics resolution High resolution at the cost of many connections High resolution, a lot of data can be collected locally by the agent. No gaps in data if the network connection fails.
Copyright 2017 Severalnines AB Copyright 2018 Severalnines AB Deepdive into ClusterControl Community Edition
ClusterControl Requirements Copyright 2018 Severalnines AB ● Installed on a single machine ○ 2GB of RAM, 2CPU and 20GB disk space running on x86 architecture ● Can run on-prem or in the cloud ● Web-GUI or Command Line ● With or without agents ● Supported platforms: ○ RedHat/CentOS 6.x/7.x ○ Ubuntu 12.04/14.04/16.04 LTS ○ Debian 7.x/8.x
Monitoring is highly available Copyright 2018 Severalnines AB
3 simple steps to deploy ClusterControl Copyright 2018 Severalnines AB Download and execute installation script $ wget http://www.severalnines.com/downloads/cmon/install-cc $ chmod +x install-cc $ ./install-cc # as root or sudo user Generate SSH keys $ ssh-keygen -t rsa #generates ssh key Copy SSH keys (on-prem monitoring) ssh-copy-id root@clustercontrolhost # clustercontrol ssh-copy-id root@dbhost1 #your database host ...
Cluster Control Architecture Copyright 2018 Severalnines AB
Metrics Collection Copyright 2018 Severalnines AB SSH Host metrics collection (process, load balancers stats, resource usage and consumption, etc.) using SSH library Database client Database metrics collection (status, queries, variables, usage etc) using the respective database client library Advisor Mini programs written using ClusterControl Domain Specific Language (DSL) Agent based uses Prometheus exporters to capture metrics data from the system it monitors. Statistics are then stored in the time series DB
Technology specific dashboards Copyright 2018 Severalnines AB ● MySQL ○ standalone, MySQL Replication, NDB Cluster, InnoDB Cluster/Group Replication ● MariaDB ○ standalone, Replication, Galera Cluster ● Percona Server for MySQL ○ standalone, Percona XtraDB Cluster ● PostgreSQL ○ standalone, Streaming Replication ● MongoDB Inc. ○ standalone, ReplicaSet, Sharded Cluster ● Percona Server for MongoDB ○ standalone, ReplicaSet, Sharded Cluster
Query Monitoring Copyright 2018 Severalnines AB ● Top queries ● Running queries ● Query Outliers
Performance Advisors Copyright 2018 Severalnines AB ● Predefined advisors ● Custom advisors and developer studio
Other features available to Community users Copyright 2018 Severalnines AB ● Deployment ○ 15+ DB topologies ○ Standalone ○ Replication setups ○ Cluster setups ● ClusterControl CLI ● Topology view
Copyright 2017 Severalnines AB Copyright 2018 Severalnines AB Demo
Poll - TBD Copyright 2018 Severalnines AB Are you currently using ClusterControl for monitoring and management? ● Yes ● I’m going to download it ● Not convinced yet
Copyright 2017 Severalnines AB Copyright 2018 Severalnines AB Q & A
End of Year Promotion Get Three Months Free 25% In Savings Just Sign By December 20th! with an Annual Contract
Links & Resources ● Download / install ClusterControl ● ClusterControl Community Edition Page ● Contact us: info@severalnines.com

Webinar slides: Free Monitoring (on Steroids) for MySQL, MariaDB, PostgreSQL and MongoDB

  • 1.
    September 2018 Free monitoringfor MySQL, MariaDB, PostgreSQL and MongoDB Bartłomiej Oleś Presenter bart@severalnines.com
  • 2.
    Copyright 2017 SeveralninesAB I'm Jean-Jérôme from the Severalnines Team and I'm your host for today's webinar! Feel free to ask any questions in the Questions section of this application or via the Chat box. You can also contact me directly via the chat box or via email: info@severalnines.com during or after the webinar. Your host & some logistics
  • 4.
  • 5.
    Copyright 2017 SeveralninesAB Free to download Initial 30 days Enterprise trial Converts into free Community Edition Enterprise / paid versions available
  • 6.
    Copyright 2017 SeveralninesAB Automation & Management Deployment (Free Community) ● Deploy a Cluster in Minutes ○ On-Prem ○ Cloud (AWS/Azure/Google) - paid Monitoring (Free Community) ● Systems View with 1 sec Resolution ● DB / OS stats & Performance Advisors ● Configurable Dashboards ● Query Analyzer ● Real-time / historical Management (Paid Features) ● Backup Management ● Upgrades & Patching ● Security & Compliance ● Operational Reports ● Automatic Recovery & Repair ● Performance Management ● Automatic Performance Advisors
  • 7.
    Copyright 2017 SeveralninesAB Supported Databases
  • 8.
  • 9.
    Poll - Whichof the following free monitoring tools does your currently company use? Copyright 2018 Severalnines AB (choose many) ● Traditional (Nagios, Zabbix, ...) ● Saas based monitoring (MMS, ...) ● Specialized (PMM, MySQL OpsPack, ...) ● ClusterControl ● Other
  • 10.
    Copyright 2017 SeveralninesAB Copyright 2018 Severalnines AB Monitoring distributed database systems - Requirements
  • 11.
    Most databases aresomehow distributed today ● Services running on multiple hosts ○ Replication ○ Sharding ○ Clustering ● Multiple Data Centers ○ Cloud and/or On-prem ○ Disaster Recovery ● Load balancing and Single point of contact IP ○ For workload management, HA, query caching... ○ E.g., HAProxy, KeepAlived/VIP, ProxySQL, MaxScale
  • 12.
    Why we needa database monitoring system ● Data is a key asset of the organisation ● Databases are important as they manage the source of truth ● Database is complex - IO, transaction engine, query optimizer, caches, locks, versioning,... ● Very dependent on OS, IO subsystems, network ● Distribution across multiple instances makes it even more complex ● Good database monitoring helps make sense of all that
  • 13.
    A “good” monitoringsystem involves ● Collection, Correlation ● Presentation ● Interpretation/analysis ● Interactions between concurrently executing processes ● Integration with other platforms
  • 14.
    Who uses databasemonitoring ● Managers ○ To improve project implementation ○ Keep track on internal or external SLA’s ○ Track team performance ○ Budgeting ○ Monitor KPIs ● Business ○ To inform and improve future programmes and projects ○ Inform stakeholders ● DBA, DevOps, SysOps ○ Prevent and event ○ Query caching ○ DB Firewall
  • 15.
  • 16.
  • 17.
    What is management? Includedin the free ClusterControl Community version
  • 18.
    Copyright 2017 SeveralninesAB Copyright 2018 Severalnines AB Cloud-based vs On-prem monitoring solutions
  • 19.
    Comparing cloud-based vson-prem Cloud-based monitoring ● Benefits ○ Quick and easy configuration ○ Managing your solution over time is also easier ○ Remote access from any location or device with browsing capabilities ● Drawbacks ○ Security ○ Reliability and stability On-prem monitoring ● Benefits ○ A level of control that cloud-based solutions just can't provide ○ Performance reliability ○ Control over data, security ● Drawbacks ○ Installation and setup ○ Data access
  • 20.
  • 21.
    Poll - TBD Copyright2018 Severalnines AB Which databases do you currently monitor? ● MySQL ● MariaDB ● PostgreSQL ● MongoDB ● Other
  • 22.
    Copyright 2017 SeveralninesAB Copyright 2018 Severalnines AB Agent-based vs Agentless
  • 23.
    Agentless vs Agent-basedmonitoring Agentless Agent-based Deployment Software on ClusterControl host only. Up and running within minutes Agent installed on each monitored DB server Admin. Overhead. Only CC needs to be maintained Agents need to be maintained, updated, restarted in case of failures Configuration Centralized Decentralized Security Controller to SSH into managed DBs, requires certain privileges. More secure. Agent communication to DB/OS is internal to the server. No additional firewall rules. Network Overhead Additional traffic with more connections Local data collection, results processed and then shipped to the server Metrics resolution High resolution at the cost of many connections High resolution, a lot of data can be collected locally by the agent. No gaps in data if the network connection fails.
  • 24.
    Copyright 2017 SeveralninesAB Copyright 2018 Severalnines AB Deepdive into ClusterControl Community Edition
  • 25.
    ClusterControl Requirements Copyright 2018Severalnines AB ● Installed on a single machine ○ 2GB of RAM, 2CPU and 20GB disk space running on x86 architecture ● Can run on-prem or in the cloud ● Web-GUI or Command Line ● With or without agents ● Supported platforms: ○ RedHat/CentOS 6.x/7.x ○ Ubuntu 12.04/14.04/16.04 LTS ○ Debian 7.x/8.x
  • 26.
    Monitoring is highlyavailable Copyright 2018 Severalnines AB
  • 27.
    3 simple stepsto deploy ClusterControl Copyright 2018 Severalnines AB Download and execute installation script $ wget http://www.severalnines.com/downloads/cmon/install-cc $ chmod +x install-cc $ ./install-cc # as root or sudo user Generate SSH keys $ ssh-keygen -t rsa #generates ssh key Copy SSH keys (on-prem monitoring) ssh-copy-id root@clustercontrolhost # clustercontrol ssh-copy-id root@dbhost1 #your database host ...
  • 28.
  • 29.
    Metrics Collection Copyright 2018Severalnines AB SSH Host metrics collection (process, load balancers stats, resource usage and consumption, etc.) using SSH library Database client Database metrics collection (status, queries, variables, usage etc) using the respective database client library Advisor Mini programs written using ClusterControl Domain Specific Language (DSL) Agent based uses Prometheus exporters to capture metrics data from the system it monitors. Statistics are then stored in the time series DB
  • 30.
    Technology specific dashboards Copyright2018 Severalnines AB ● MySQL ○ standalone, MySQL Replication, NDB Cluster, InnoDB Cluster/Group Replication ● MariaDB ○ standalone, Replication, Galera Cluster ● Percona Server for MySQL ○ standalone, Percona XtraDB Cluster ● PostgreSQL ○ standalone, Streaming Replication ● MongoDB Inc. ○ standalone, ReplicaSet, Sharded Cluster ● Percona Server for MongoDB ○ standalone, ReplicaSet, Sharded Cluster
  • 31.
    Query Monitoring Copyright 2018Severalnines AB ● Top queries ● Running queries ● Query Outliers
  • 32.
    Performance Advisors Copyright 2018Severalnines AB ● Predefined advisors ● Custom advisors and developer studio
  • 33.
    Other features availableto Community users Copyright 2018 Severalnines AB ● Deployment ○ 15+ DB topologies ○ Standalone ○ Replication setups ○ Cluster setups ● ClusterControl CLI ● Topology view
  • 34.
    Copyright 2017 SeveralninesAB Copyright 2018 Severalnines AB Demo
  • 35.
    Poll - TBD Copyright2018 Severalnines AB Are you currently using ClusterControl for monitoring and management? ● Yes ● I’m going to download it ● Not convinced yet
  • 36.
    Copyright 2017 SeveralninesAB Copyright 2018 Severalnines AB Q & A
  • 37.
    End of YearPromotion Get Three Months Free 25% In Savings Just Sign By December 20th! with an Annual Contract
  • 38.
    Links & Resources ●Download / install ClusterControl ● ClusterControl Community Edition Page ● Contact us: info@severalnines.com