MongoDB & Analytics Building solutions with the MongoDB BI Connector Patrick Sheehan – Sr. Solutions Architect, MongoDB - Palo Alto, CA
3 Context – Business Intelligence (BI) Business intelligence (BI) is a technology- driven process for analyzing data and presenting actionable information to help corporate executives, business managers and other end users make more informed business decisions. BI encompasses a variety of tools, applications and methodologies that enable organizations to collect data from internal systems and external sources, prepare it for analysis, develop and run queries against the data, and create reports, dashboards and data visualizations to make the analytical results available to corporate decision makers as well as operational workers. Porta Ultricies Commodo Porta
4 BI is a BIG Deal! Gartner Says Worldwide Business Intelligence and Analytics Market to Reach $16.9 Billion in 2016 Global revenue in the business intelligence (BI) and analytics market is forecast to reach $16.9 billion in 2016, an increase of 5.2 percent from 2015, according to the latest forecast from Gartner, Inc. According to Gartner, the BI and analytics market is in the final stages of a multiyear shift from IT-led, system-of-record reporting to business-led, self-service analytics. As a result, the modern business intelligence and analytics (BI&A) platform has emerged to meet new organizational requirements for accessibility, agility and deeper analytical insight. Source: Gartner NewsRoom, February, 2016: http://www.gartner.com/newsroom/id/3198917
5 Context - MongoDB RANK DBMS MODEL SCORE GROWTH (20 MO) 1. Oracle Relational DBMS 1,442 -5% 2. MySQL Relational DBMS 1,294 2% 3. Microsoft SQL Server Relational DBMS 1,131 -10% 4. MongoDB Document Store 277 172% 5. PostgreSQL Relational DBMS 273 40% 6. DB2 Relational DBMS 201 11% 7. Microsoft Access Relational DBMS 146 -26% 8. Cassandra Wide Column 107 87% 9. SQLite Relational DBMS 105 19% Only non-relational in the top 5; 2.5x ahead of nearest NoSQL Competitor
6 { first_name: ‘Paul’, surname: ‘Miller’, city: ‘London’, location: [45.123,47.232], cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } ] } MongoDB Document Model with Flexible Schema RDBMS
7 Documents are Rich Data Structures { first_name: ‘Paul’, surname: ‘Miller’, cell: 447557505611, city: ‘London’, location: [45.123,47.232], Profession: [‘banking’, ‘finance’, ‘trader’], cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } ] } Fields can contain an array of sub-documents Fields Typed field values Fields can contain arrays Number
8 MongoDB Connector for BI Visualize and explore multi-dimensional documents using SQL-based BI tools. The connector does the following: • Provides the BI tool with the schema of the MongoDB collection to be visualized • Translates SQL statements issued by the BI tool into equivalent MongoDB queries that are sent to MongoDB for processing • Converts the results into the tabular format expected by the BI tool, which can then visualize the data based on user requirements
9 Location & Flow of Data MongoDB BI Connector Mapping meta-data Application data {name: “Andrew”, address: {street:… }} DocumentTableAnalytics & visualization
10 Defining Data Mapping mongodrdl --host 192.168.1.94 --port 27017 -d myDbName -o myDrdlFile.drdl mongobischema import myCollectionName myDrdlFile.drdl DRDL mongodrdl mongobischema PostgreSQL MongoDB- specific Foreign Data Wrapper
11 Optionally Manually Edit DRDL File • Redact attributes • Use more appropriate types (sampling can get it wrong) • Rename tables (v1.1+) • Rename columns (v1.1+) • Build new views using MongoDB Aggregation Framework • e.g., $lookup to join 2 tables - table: homesales collection: homeSales pipeline: [] columns: - name: _id mongotype: bson.ObjectId sqlname: _id sqltype: varchar - name: address.county mongotype: string sqlname: address_county sqltype: varchar - name: address.nameOrNumber mongotype: int sqlname: address_nameornumber sqltype: varchar
Data, Data Every WhereDemo
13 Next Steps  Download the MongoDB 3.2 Whitepaper https://www.mongodb.com/collateral/mongodb-3-2-whats-new  Try the MongoDB Connector for BI https://www.mongodb.com/lp/download/mongodb-enterprise  Review the docs! https://docs.mongodb.org/bi-connector/ https://docs.mongodb.org/bi-connector/installation/ https://docs.mongodb.org/bi-connector/schema-configuration/ https://docs.mongodb.org/bi-connector/components/  FAQ https://docs.mongodb.org/bi-connector/faq/  Developers Notebook https://github.com/farrell0/MongoDB-Developers-Notebook/blob/master/README.md
Questions?
Webinar: MongoDB and Analytics: Building Solutions with the MongoDB BI Connector

Webinar: MongoDB and Analytics: Building Solutions with the MongoDB BI Connector

  • 2.
    MongoDB & Analytics Buildingsolutions with the MongoDB BI Connector Patrick Sheehan – Sr. Solutions Architect, MongoDB - Palo Alto, CA
  • 3.
    3 Context – BusinessIntelligence (BI) Business intelligence (BI) is a technology- driven process for analyzing data and presenting actionable information to help corporate executives, business managers and other end users make more informed business decisions. BI encompasses a variety of tools, applications and methodologies that enable organizations to collect data from internal systems and external sources, prepare it for analysis, develop and run queries against the data, and create reports, dashboards and data visualizations to make the analytical results available to corporate decision makers as well as operational workers. Porta Ultricies Commodo Porta
  • 4.
    4 BI is aBIG Deal! Gartner Says Worldwide Business Intelligence and Analytics Market to Reach $16.9 Billion in 2016 Global revenue in the business intelligence (BI) and analytics market is forecast to reach $16.9 billion in 2016, an increase of 5.2 percent from 2015, according to the latest forecast from Gartner, Inc. According to Gartner, the BI and analytics market is in the final stages of a multiyear shift from IT-led, system-of-record reporting to business-led, self-service analytics. As a result, the modern business intelligence and analytics (BI&A) platform has emerged to meet new organizational requirements for accessibility, agility and deeper analytical insight. Source: Gartner NewsRoom, February, 2016: http://www.gartner.com/newsroom/id/3198917
  • 5.
    5 Context - MongoDB RANK DBMSMODEL SCORE GROWTH (20 MO) 1. Oracle Relational DBMS 1,442 -5% 2. MySQL Relational DBMS 1,294 2% 3. Microsoft SQL Server Relational DBMS 1,131 -10% 4. MongoDB Document Store 277 172% 5. PostgreSQL Relational DBMS 273 40% 6. DB2 Relational DBMS 201 11% 7. Microsoft Access Relational DBMS 146 -26% 8. Cassandra Wide Column 107 87% 9. SQLite Relational DBMS 105 19% Only non-relational in the top 5; 2.5x ahead of nearest NoSQL Competitor
  • 6.
    6 { first_name: ‘Paul’, surname: ‘Miller’, city:‘London’, location: [45.123,47.232], cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } ] } MongoDB Document Model with Flexible Schema RDBMS
  • 7.
    7 Documents are RichData Structures { first_name: ‘Paul’, surname: ‘Miller’, cell: 447557505611, city: ‘London’, location: [45.123,47.232], Profession: [‘banking’, ‘finance’, ‘trader’], cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } ] } Fields can contain an array of sub-documents Fields Typed field values Fields can contain arrays Number
  • 8.
    8 MongoDB Connector forBI Visualize and explore multi-dimensional documents using SQL-based BI tools. The connector does the following: • Provides the BI tool with the schema of the MongoDB collection to be visualized • Translates SQL statements issued by the BI tool into equivalent MongoDB queries that are sent to MongoDB for processing • Converts the results into the tabular format expected by the BI tool, which can then visualize the data based on user requirements
  • 9.
    9 Location & Flowof Data MongoDB BI Connector Mapping meta-data Application data {name: “Andrew”, address: {street:… }} DocumentTableAnalytics & visualization
  • 10.
    10 Defining Data Mapping mongodrdl--host 192.168.1.94 --port 27017 -d myDbName -o myDrdlFile.drdl mongobischema import myCollectionName myDrdlFile.drdl DRDL mongodrdl mongobischema PostgreSQL MongoDB- specific Foreign Data Wrapper
  • 11.
    11 Optionally Manually EditDRDL File • Redact attributes • Use more appropriate types (sampling can get it wrong) • Rename tables (v1.1+) • Rename columns (v1.1+) • Build new views using MongoDB Aggregation Framework • e.g., $lookup to join 2 tables - table: homesales collection: homeSales pipeline: [] columns: - name: _id mongotype: bson.ObjectId sqlname: _id sqltype: varchar - name: address.county mongotype: string sqlname: address_county sqltype: varchar - name: address.nameOrNumber mongotype: int sqlname: address_nameornumber sqltype: varchar
  • 12.
  • 13.
    13 Next Steps  Downloadthe MongoDB 3.2 Whitepaper https://www.mongodb.com/collateral/mongodb-3-2-whats-new  Try the MongoDB Connector for BI https://www.mongodb.com/lp/download/mongodb-enterprise  Review the docs! https://docs.mongodb.org/bi-connector/ https://docs.mongodb.org/bi-connector/installation/ https://docs.mongodb.org/bi-connector/schema-configuration/ https://docs.mongodb.org/bi-connector/components/  FAQ https://docs.mongodb.org/bi-connector/faq/  Developers Notebook https://github.com/farrell0/MongoDB-Developers-Notebook/blob/master/README.md
  • 14.