Exploring Google (Cloud) APIs with Python & JavaScript

Exploring Google (Cloud) APIs with Python & JavaScript Wesley Chun Developer Advocate Google Cloud

G Suite Dev Show goo.gl/JpBQ40 About the speaker ● Developer Advocate, Google Cloud ● Mission: enable current & future developers to be successful using Google Cloud and other Google developer tools, APIs, and platforms ● Videos: host of the G Suite Dev Show on YouTube ● Blogs: developers.googleblog.com & gsuite-developers.googleblog.com ● Twitters: @wescpy, @GCPcloud, @GoogleDevs, @GSuiteDevs ● Background ● Software engineer & architect for 20+ years ● One of the original Yahoo!Mail engineers ● Author of bestselling "Core Python" books (corepython.com) ● Teacher and technical instructor since 1983 (all ages) ● Fellow of the Python Software Foundation ● AB Mathematics & CMP Music, UC Berkeley; MSCS UC Santa Barbara

Agenda ● Introduction ● Getting started with Google API projects ● Google Cloud (HTTP-based) REST APIs ● Serverless (compute) platforms ● Miscellaneous & wrap-up Part I: Introduction

BUT ... wait, there’s more...

Part II: Getting Started with Google API projects

The first word on Security Authentication ("authn") vs authorization ("authz") ● authn: you are who you say you are ○ login & passwd ○ handprint authentication ○ retina scan ● authz: okay, you are who you say you are, but can you haz data? ○ OAuth2 - mostly authz, but some authn ○ Mostly about 3rd-party access to data ○ Users must give YOUR app access to THEIR data ○ Most of the time when you see "auth", it refers to authz Developers Console (devconsole)

OAuth2 scenarios ● Varies on application type (where apps located) ○ Web (server) applications ○ Installed (mobile & desktop computers) apps ○ Service accounts (cloud/server-to-server) ○ Client-side (JavaScript) apps ○ Limited-input devices (game consoles, printers, cameras, etc.) ○ developers.google.com/identity/protocols/OAuth2 ○ TODAY: command-line script == "Installed"

Google APIs client libraries for many languages; demos in developers.google.com/ api-client-library Accessing API client libraries ● Python ○ pip install -U google-api-python-client ○ developers.google.com/api-client-library/ python/start/get_started ● JavaScript ("browser"/client-side) ○ Use gapi.client.request ○ developers.google.com/api-client-library/ javascript/start/start-js ● JavaScript (Node.js/server-side) ○ npm install googleapis ○ googleapis.github.io/google-api-nodejs-client

SIMPLE AUTHORIZED Which do you choose? Simple API access

Google OAuth2 process ● Goal: valid access token == 3rd-party API data access ● How OAuth2 works (in general) ○ Step 1: Create OAuth2 credentials in DevConsole ○ Step 2: Send credentials to get access & refresh tokens ○ Step 3: Use access token to make authorized API calls ○ Step 4: Access tokens expire; use refresh to get new one ● developers.google.com/identity/protocols/OAuth2 ● Other APIs will be similar with some differences Authorized API access

OAuth2 HTTP-based REST APIs 1 HTTP 2 Google APIs request-response workflow 1. Application makes request 2. Request received by service 3. Process data, return response 4. Results sent to application a.k.a. client-server model Part III: Google Cloud REST APIs

Compute Big Data BigQuery Cloud Pub/Sub Cloud Dataproc Cloud Datalab Cloud Pub/Sub Genomics Cloud AI Cloud Machine Learning Engine Cloud Vision API Cloud Speech-to-Text Cloud Natural Language API Cloud Translation API Cloud Jobs API Cloud Dataprep Cloud Video Intelligence API Advanced Solutions Lab Compute Engine App Engine Kubernetes Engine GPU Cloud Functions Container- Optimized OS Identity & Security Cloud IAM Cloud Resource Manager Cloud Security Scanner Key Management Service BeyondCorp Data Loss Prevention API Identity-Aware Proxy Security Key Enforcement Cloud AutoML Cloud Text-to-Speech Cloud TPU Dialogflow Enterprise Edition Data Transfer Transfer Appliance Cloud Composer Cloud Security Command Center Networking Virtual Private Cloud Cloud Load Balancing Cloud CDN Dedicated Interconnect Cloud DNS Cloud Network Cloud External IP Addresses Cloud Firewall Rules Cloud Routes Cloud VPN Management Tools Stackdriver Monitoring Logging Error Reporting Trace Debugger Cloud Deployment Manager Cloud Console Cloud Shell Cloud Mobile App Cloud Billing API Cloud APIs Cloud Router Partner Interconnect Cloud Armor Standard Network Tier Premium Network Tier Profiler GCP products and services without a unique icon have a generic hexagon.

Developer Tools Cloud SDK Cloud Source Repositories Maven App Engine Plugin Cloud Tools for IntelliJ Cloud Tools for PowerShell Cloud Tools for Visual Studio Container Registry Cloud Tools for Eclipse Cloud Build API Platform & Ecosystems API Analytics API Monetization Apigee API Platform Apigee Sense Cloud Endpoints Developer Portal Gradle App Engine Plugin IDE plugins Internet of Things Cloud IoT Core Storage & Databases Cloud Storage Cloud Bigtable Cloud Datastore Cloud SQL Cloud Spanner Persistent Disk Cloud Memorystore Cloud Filestore Cloud Test Lab Cloud IoT Edge G Suite APIs

Proprietary + Confidential ● Launched API v3 (Dec 2015) ○ v2 not deprecated (yet) ● WhatsApp and Google BigQuery integrations (May 2016) ● Team Drives (Jun 2017) Drive Back up your file archives Write your own or see github.com/gsuitedevs/drive-zipextractor (JS)

Automate photo albums OR List (first 100) files/folders in Google Drive from __future__ import print_function from apiclient import discovery from httplib2 import Http from oauth2client import file, client, tools SCOPES = 'https://www.googleapis.com/auth/drive.readonly.metadata' store = file.Storage('storage.json') creds = store.get() if not creds or creds.invalid: flow = client.flow_from_clientsecrets('client_secret.json', SCOPES) creds = tools.run_flow(flow, store) DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http())) files = DRIVE.files().list().execute().get('files', []) for f in files: print(f['name'], f['mimeType']) Listing your files goo.gl/ZIgf8k

Proprietary + Confidential ● Launched API v4 (I/O 2016) ● Support text rotation (Mar 2017) ● Developer metadata (Sep 2017) ● Macro recorder (Apr 2018) Sheets Powerful API: expect UI features available programmatically ● Create new sheets within spreadsheets ● Add data to sheets ● Create new spreadsheets ● Set frozen rows ● Text+number formatting ● Enforce cell validation ● Adjust column sizes ● Apply formulas ● Build pivot tables ● Create charts ● … and more! BEFORE (<= v3) AFTER (>= v4)

Import/Export: Customized reports, “database,” or both! Try our Node.js customized reporting tool codelab: g.co/codelabs/sheets Migrate SQL data to a Sheet # read SQL data then create new spreadsheet & add rows into it FIELDS = ('ID', 'Customer Name', 'Product Code', 'Units Ordered', 'Unit Price', 'Status') cxn = sqlite3.connect('db.sqlite') cur = cxn.cursor() rows = cur.execute('SELECT * FROM orders').fetchall() cxn.close() rows.insert(0, FIELDS) DATA = {'properties': {'title': 'Customer orders'}} SHEET_ID = SHEETS.spreadsheets().create(body=DATA, fields='spreadsheetId').execute().get('spreadsheetId') SHEETS.spreadsheets().values().update(spreadsheetId=SHEET_ID, range='A1', body={'values': rows}, valueInputOption='RAW').execute() Migrate SQL data to Sheets goo.gl/N1RPwC

Format cells (frozen & bold row) requests = [ # freeze row 1 {'updateSheetProperties': { 'properties': {'gridProperties': {'frozenRowCount': 1}}, 'fields': 'gridProperties.frozenRowCount', }}, # bold row 1 {'repeatCell': { 'range': {'startRowIndex': 0, 'endRowIndex': 1}, 'cell': {'userEnteredFormat': {'textFormat': {'bold': True}}}, 'fields': 'userEnteredFormat.textFormat.bold', }}, ] SHEETS.spreadsheets().batchUpdate(body={'requests': requests}, spreadsheetId=SHEET_ID, fields='').execute() Formatting cells goo.gl/U6Oljn ● API preview (I/O 2016) ● Launched API v1 (Nov 2016) ● Enhancements “v1.1” (Feb 2017) ● Apps Script support (Sep 2017) Slides

Try our Node.js Markdown-to-Google-Slides generator: github.com/googlesamples/md2googleslides

Replace text & images from template deck requests = [ # (global) search-and-replace text {'replaceAllText': { 'findText': '{{TITLE}}', 'replaceText': 'Hello World!', }}, # replace text-based image placeholders (global) {'replaceAllShapesWithImage': { 'imageUrl': IMG_URL, # link to product logo 'replaceMethod': 'CENTER_INSIDE', 'containsText': {'text': '{{LOGO}}'}, }}, ] SLIDES.presentations().batchUpdate(body={'requests': requests}, presentationId=DECK_ID, fields='').execute() Replacing text and images goo.gl/o6EFwk Proprietary + Confidential ● Settings endpoints launch (Jul 2016) ● New metadata scope (Nov 2016) ● Gmail Add-ons (Oct 2017) ● Admin SDK Email Settings deprecation (Oct 2018) Gmail

Display Gmail threads (& # of messages) # get all "busy" mail threads & display (non-blank) Subject lines threads = GMAIL.users().threads().list(userId='me', fields='threads').execute().get('threads', []) for thread in threads: tdata = GMAIL.users().threads().get(userId='me', id=thread['id'], format='metadata', metadataHeaders=['subject'], fields='messages/payload/headers(name,value)').execute() if 'messages' not in tdata: continue if len(tdata['messages']) > 10: msg = tdata['messages'][0]['payload'] subject = '' for header in msg['headers']: if header['name'] == 'Subject': subject = header['value'] break if subject: print('%2d message(s) in thread: %s' % ( len(tdata['messages']), subject)) Gmail threads goo.gl/pFYUQ2 Proprietary + Confidential ● Events on Google Maps (Sep 2016) ● Find a time” on iOS (Sep 2016) ● New Calendar iOS app (Nov 2016) ● New Calendar UI (Oct 2017) ● Hangouts Meet integration in Calendar API (Dec 2017) Calendar

Create events in Calendar # define event data, then create event TIMEZONE = 'America/Los_Angeles' EVENT = { 'summary': 'Dinner with friends', 'start': {'dateTime': '2017-06-14T19:00:00', 'timeZone': TIMEZONE}, 'end': {'dateTime': '2017-06-14T22:00:00', 'timeZone': TIMEZONE}, 'attendees': [ {'email': 'friend1@example.com'}, {'email': 'friend2@example.com'}, ], } GCAL.events().insert(calendarId='primary', body=EVENT, sendNotifications=True, fields='').execute() Modifying and recurring events goo.gl/J2XkXc Creating events goo.gl/KuYMiq Proprietary + Confidential ● Launched API v1 EAP (NEXT 2018) ○ developers.google.com/docs Docs

Global search & replace from template doc requests = [ # (global) search-and-replace text {'replaceAllText': { 'containsText': {'text': '{{TITLE}}'}, 'replaceText': 'Hello World!', }}, ] DOCS.documents().batchUpdate(body={'requests': requests}, name='documents/'+DOC_ID, fields='').execute() Google Cloud Platform ● Added support for Standard SQL 2011 (Sep 2016) ● Launched BigQuery Data Transfer Service (Mar 2017) BigQuery

Top 10 most common Shakespeare words TITLE = "The top 10 most common words in all of Shakespeare's works" QUERY = ''' SELECT LOWER(word) AS word, sum(word_count) AS count FROM [bigquery-public-data:samples.shakespeare] GROUP BY word ORDER BY count DESC LIMIT 10 ''' rsp = BQ.query(body={'query': QUERY}, projectId=PROJ_ID).execute() print('n*** Results for %r:n' % TITLE) for col in rsp['schema']['fields']: # HEADERS print(col['name'].upper(), end='t') print() for row in rsp['rows']: # DATA for col in row['f']: print(col['v'], end='t') print() Google Cloud Platform Top 10 most common Shakespeare words $ python3 bq_shake.py *** Results for "The most common words in all of Shakespeare's works": WORD COUNT the 29801 and 27529 i 21029 to 20957 of 18514 a 15370 you 14010 my 12936 in 11722 that 11519 Google Cloud Platform

Google Cloud Platform ● API v1 launch (Nov 2016) ● Extended sentiment entity analysis & language support (May 2017) ● Automatic content classification (Sep 2017) ● AutoML Natural Language Beta release (Cloud NEXT 2018) Cloud Natural Language What is it, and how does it work? ● Cloud Natural Language API ○ ML algorithms: analyze unstructured text ○ Sentiment & Entity analysis: derive structure & meaning of text ○ Extract information about people, places, events, and more ○ cloud.google.com/natural-language ● Natural Language API requests & responses (JSON) ○ Request payload: content, type, language, encoding ○ Sentiment analysis score: -1.0 (negative) to 1.0 (positive) + magnitude ○ Entity analysis: entity types & metadata, salience, and mentions ○ Entity sentiment analysis: combination of above pair ○ Syntactic analysis: extract sentences and tokens from content ○ Content classification: identified content categories Google Cloud Platform

[simple API/API key sample] Simple sentiment & classification analysis text = ''' Google, headquartered in Mountain View, unveiled the new Android phone at the Consumer Electronics Show. Sundar Pichai said in his keynote that users love their new Android phones.''' data = {'type': 'PLAIN_TEXT', 'content': text} NL = discovery.build('language', 'v1', developerKey=API_KEY) sentiment = NL.documents().analyzeSentiment( body={'document': data}).execute().get('documentSentiment') print('TEXT:', text) print('nSENTIMENT: score (%s), magnitude (%s)' % ( sentiment['score'], sentiment['magnitude'])) print('nCATEGORIES:') categories = NL.documents().classifyText( body={'document': data}).execute().get('categories') for c in categories: print ('* %s (%s)' % (c['name'][1:], c['confidence'])) PY Sentiment & classification analysis output $ python3 nl_sent_class.py TEXT: Google, headquartered in Mountain View, unveiled the new Android phone at the Consumer Electronics Show. Sundar Pichai said in his keynote that users love their new Android phones. SENTIMENT: score (0.3), magnitude (0.6) CATEGORIES: * Internet & Telecom (0.76) * Computers & Electronics (0.64) * News (0.56)

Google Proprietary + Confidential ● YouTube Data API updates & bug fixes (Nov 2016) YouTube [simple API/API key sample] Search YouTube for videos from __future__ import print_function from apiclient import discovery from settings import API_KEY QUERY = 'python -snake' trim = lambda x, ct: ('%s%s' % (x[:ct], '...' if len(x)>ct else '')).ljust(ct+3) print('n** Searching for %r videos...' % QUERY) YOUTUBE = discovery.build('youtube', 'v3', developerKey=API_KEY) res = YOUTUBE.search().list(q=QUERY, type='video', part='id,snippet').execute().get('items', []) for item in res: print('http://youtu.be/%st%s' % ( trim(item['id']['videoId'], 24), trim(item['snippet']['title'], 48)))

Can use multiple APIs in same app Simple, authorized, or a mix of both API types; re-use HTTP client from __future__ import print_function from apiclient import discovery from httplib2 import Http from oauth2client import file, client, tools SCOPES = ( # iterable or space-delimited string 'https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/spreadsheets.readonly', 'https://www.googleapis.com/auth/presentations', ) store = file.Storage('storage.json') creds = store.get() if not creds or creds.invalid: flow = client.flow_from_clientsecrets('client_secret.json', SCOPES) creds = tools.run_flow(flow, store) HTTP = creds.authorize(Http()) DRIVE = discovery.build('drive', 'v3', http=HTTP) SHEETS = discovery.build('sheets', 'v4', http=HTTP) SLIDES = discovery.build('slides', 'v1', http=HTTP) Part IV: Serverless (compute) platforms

Proprietary + Confidential ● Quizzes in Forms support (Apr 2017) ● Google Slides Service and Add-ons (Sep 2017) ● Gmail Add-ons (Oct 2017) ● Command-line tool (CLASP), new API, developer dashboard (Jan 2018) ● Sheets service upgrades (Apr 2018) Apps Script (Sheets-bound) “Hello World!”

“Hello World!” results Apps Script intro goo.gl/1sXeuD What can you do with this?

function sendMap() { var sheet = SpreadsheetApp.getActiveSheet(); var address = sheet.getRange("A2").getValue(); var map = Maps.newStaticMap().addMarker(address); MailApp.sendEmail(EMAIL, "Map", map.getMapUrl()); } JS Accessing maps from spreadsheets?!? goo.gl/oAzBN9 More than you think... ...with help from Google Maps and Gmail

Apps Script powers add-ons & integrations - Extend functionality of G Suite editors - Embed your app within ours! - 2014: Google Docs, Sheets, Forms - 2017 Q3: Google Slides - 2017 Q4: Gmail - 2018 Q1: Hangouts Chat bots Generating Google Slides from images

Generating Google Slides from images var NAME = "My favorite images"; var deck = SlidesApp.getActivePresentation(); function addImageSlide(link, index) { var slide = deck.appendSlide(SlidesApp.PredefinedLayout.BLANK); var image = slide.insertImage(link); } function main() { var images = [ "http://www.google.com/services/images/phone-animation-results_2x.png", "http://www.google.com/services/images/section-work-card-img_2x.jpg", "http://gsuite.google.com/img/icons/product-lockup.png", "http://gsuite.google.com/img/home-hero_2x.jpg", ]; var [title, subtitle] = deck.getSlides()[0].getPageElements(); title.asShape().getText().setText(NAME); subtitle.asShape().getText().setText("Google Apps ScriptnSlides Service demo"); images.forEach(addImageSlide); } Introducing Slides Add-ons goo.gl/sYL5AM Gmail Add-ons ‐ Expense reports ‐ Can't we do them without leaving Gmail? ‐ On Web AND mobile?

Gmail Add-ons ‐ One place to do your expense report ExpenseIt! Gmail Add-on function createExpensesCard(opt_prefills, opt_status) { var card = CardService.newCardBuilder(); card.setHeader(CardService.newCardHeader().setTitle('Log Your Expense')); var clearForm = CardService.newAction().setFunctionName('clearForm') .setParameters({'Status': opt_status ? opt_status : ''}); var clearAction = CardService.newCardAction().setText('Clear form') .setOnClickAction(clearForm); card.addCardAction(clearAction); : var newSheetSection = CardService.newCardSection(); var sheetName = CardService.newTextInput().setFieldName('Sheet Name') .setTitle('Sheet Name'); var createExpensesSheet = CardService.newAction() .setFunctionName('createExpensesSheet'); var newSheetButton = CardService.newTextButton().setText('New Sheet') .setOnClickAction(createExpensesSheet); newSheetSection.addWidget(sheetName); newSheetSection.addWidget(CardService.newButtonSet().addButton(newSheetButton)); card.addSection(formSection); card.addSection(newSheetSection); return card; } Expediting expense reports goo.gl/KUVCDu

Simple vote bot function onMessage(e) { return createMessage(e.user.displayName, 0); } function onCardClick(e) { // Create a new vote card when 'NEW' button is clicked. if (e.action.actionMethodName === 'newvote') { return createMessage(e.user.displayName, 0); } // Updates the card in-place when '+1' or '-1' button is clicked. var voteCount = +e.action.parameters[0].value; e.action.actionMethodName === 'upvote' ? ++voteCount : --voteCount; return createMessage(e.user.displayName, voteCount, true); } Vote bot interactive card function createMessage(voter, voteCount, shouldUpdate) { var parameters = [{key: 'count', value: voteCount.toString()}]; return { actionResponse: { type: shouldUpdate ? 'UPDATE_MESSAGE' : 'NEW_MESSAGE' }, cards: [{ header: { title: 'Last vote by ' + voter + '!' }, sections: [{ widgets: [{ textParagraph: { text: voteCount + ' votes!' } }, { buttons: [{ textButton: { text: '+1', onClick: { action: { actionMethodName: 'upvote', parameters: parameters } } } }, { textButton: { text: '-1', onClick: { action: { actionMethodName: 'downvote', parameters: parameters } } } }, { textButton: { text: 'NEW', onClick: { action: { actionMethodName: 'newvote' } } } }] }] }] }] }; } Build Hangouts Chat bots goo.gl/jt3FqK

Part V: Miscellaneous & wrap-up ● Hangouts Chat launch (Feb 2018) ○ Available to G Suite customers ○ Features bot framework and API... build chat bots to: ■ Automate workflows ■ Query for information ■ Other heavy-lifting ○ Many development and hosting options Hangouts Chat

Traditional API workflow Traditional APIs vs. Bot architecture OAuth2 Bot architecture Hangouts Chat "echo bot" in Python/Flask from flask import Flask, request, json app = Flask(__name__) @app.route('/', methods=['POST']) def on_event(): event = request.get_json() msg = {} if event['type'] == 'MESSAGE': text = 'Hi {}. You sent: {}'.format( event['user']['displayName'], event['message']['text']) msg = {'text': text} return json.jsonify(msg) PY

Hangouts Chat "echo bot" in Node.js/CF4FB const functions = require('firebase-functions'); exports.echoBot = functions.https.onRequest((req, resp) => { var msg = {}; if (req.body.type === 'MESSAGE') { msg = { text: 'Hi ' + req.body.user.displayName + '. You sent: ' + req.body.message.text }; } return resp.send(msg).end(); }); JS Other Google APIs & platforms ● Firebase ○ firebase.google.com ● Actions on Google ○ developers.google.com/actions ● YouTube ○ Data, Analytics, and Livestreaming APIs ○ developers.google.com/youtube ● Google Maps ○ Maps, Routes, and Places APIs ○ developers.google.com/maps

References ● Keep up-to-date ○ Blogs: cloud.google.com/blog, developers.googleblog.com, gsuite-developers.googleblog.com, cloudplatform.googleblog.com ○ Twitters: @wescpy, @GoogleDevs, @GSuiteDevs, @GCPcloud, @GoogleCloud ● Get started today! (introductory hands-on, self-paced codelabs) ○ GCP codelabs: codelabs.developers.google.com/?cat=Cloud ○ G Suite (REST) APIs: g.co/codelabs/gsuite-apis-intro (Drive API) ○ Google Apps Script: g.co/codelabs/apps-script-intro ○ Other G Suite: codelabs.developers.google.com/?cat=Apps ● Read more ○ G Suite dev docs: developers.google.com/gsuite ○ GCP dev docs: cloud.google.com/docs ● NEXT '19 session: Build with All of Google Cloud: G Suite + GCP Interoperability ○ cloud.withgoogle.com/next/sf/sessions?session=DEV212 More references ● API developers console: console.developers.google.com ● GCP developers console: cloud.developers.google.com ● Google APIs Client Libraries: developers.google.com/api-client-library ○ Python: pip{,3} install -U google-api-python-client ○ Special versions for Django, Google App Engine ○ Node.js: npm install googleapis ● Developer video series ○ G Suite Dev Show: goo.gl/JpBQ40 ○ Launchpad Online: goo.gl/kFMUa6 ● Relevant videos ○ goo.gl/RbyTFD (new Google APIs project setup) ○ goo.gl/KMfbeK (common Python OAuth2 boilerplate code review) ○ goo.gl/ZIgf8k (APIs intro codelab [Drive API])

Thank you! Questions? Wesley Chun Developer Advocate, Google Cloud @wescpy & @GoogleDevs twitter.com/{wescpy,googledevs}

Exploring Google (Cloud) APIs with Python & JavaScript

More Related Content

What's hot

Similar to Exploring Google (Cloud) APIs with Python & JavaScript

More from wesley chun

Recently uploaded

Exploring Google (Cloud) APIs with Python & JavaScript