Introducing Program Management

Introducing Optimizely Program Management Claire Vo, VP of Product, Optimizely Heather Miller, Senior Experimentation Analyst, BBC SCALE YOUR PROGRAM AND 5X YOUR EXPERIMENT VELOCITY

3 Heather Miller Senior Experimentation Analyst BBC Claire Vo Vice President, Product Optimizely

Agenda ● Introduction ● Optimizely Program Management ● Demo ● BBC’s Experimentation Program ● Q&A

“Our success is a function of how many experiments we do per year, per month, per week, per day.” “Instead of saying ‘I have an idea,’ what if you said ‘I have a new hypothesis, let’s go test it.’ “Our company culture encourages experimentation and free flow of ideas.” 3 “One of the things I’m most proud of, and I think what is the key to our success, is this testing framework we’ve built.” Experimentation is the Next Great Business Transformation Jeff Bezos Larry Page Mark Zuckerberg Satya Nadella The Surprising Power of Online Experiments

Digital Experience Optimization: Digital Products, Commerce & Campaigns Up to 5X Increase in Yield: Revenue, Share of wallet, Funnel conversion, Risk mitigation, Ops efficiency Partner of Choice: Work with leading global enterprises & “digital disruptors” including 26 of F100 OUR COMPANY Digital Experimentation Platform: Next gen “Test and Learn” System Replaces Digital Guesswork: with Evidence-based optimization Speeds Innovation & Optimization: Single platform for marketing & product teams Best in class stats & machine learning Consumer grade usability Enterprise program management & prof. services 2 OUR SOLUTION

13 Global Companies Experimenting at Scale with Optimizely

Building an experimentation program can be challenging

Prioritization Ideation can be siloed leading to inefficient prioritization and execution Oversight & knowledge sharing Lack of executive oversight can lead to serious problems in a global organization Coordination As you scale, coordinating across additional teams and stakeholders becomes more and more difficult Resources Time, people, and effort is wasted duplicating and reinventing different processes Challenges of Building a Program

IDEATION & PRIORITIZATION TESTING & IMPLEMENTATION REQUIREMENTS GATHERING & DEVELOPMENT REPORTING & ANALYSIS DOCUMENTATION & IP PRESERVATION Does this reporting match internal data? What are we going to test next? Can someone code this test for me?

Program Management Scale your experimentation program across your entire enterprise

Manage your program across the entire experiment lifecycle Teams Web & Full Stack Experimentation Program Reporting OPTIMIZELY PROGRAM MANAGEMENT IDEATION REQUIREMENTS CREATIVE DEVELOPMENT TESTING ANALYSIS SHARING

5x Improved testing velocity through improved collaboration UP TO

Democratize ideation and empower your entire team to participate Make the experimentation process more accessible and transparent Improve your program workflow with new tools for collaboration Teams Collaborate more effectively with an integrated hub for your team

Track the velocity of your program Gain more oversight across projects and teams Uncover new insights and iterate on your best ideas Program Reporting Gain powerful insights into your program with a new executive dashboard

Program Management at the BBC – with Optimizely Heather Miller Senior Experimentation and Optimisation Analyst 17 January 2018

•The BBC is the oldest and largest broadcaster in the world •Founded in 1922, we employ 35,402 staff in total across the world •Our primary audience is the UK - but some of our work reaches over 200 countries •We deliver a wide range of services, from news, radio, video, music and online content •The BBC is considered to be of ‘significant cultural importance’ to the UK

“To enrich people’s lives with programmes and services that inform, educate and entertain.” - John Reith, Founder of the BBC

Experimentation at the BBC: our team •The Experimentation and Optimisation team sits within Data Insights •Our team operates as a Centre of Excellence •There are five us in the Experimentation team, who leverage our expertise to advise and consult with teams across the BBC •Our team assists with the entire end-to-end process - from strategy, to test design, to analysis, to final reports •We want to upskill teams and make them self-sufficient in experimentation best practice

•Right now, 13 teams use Optimizely across the whole of the BBC •Our ultimate goal is to have 40 teams using the tool •On any given day dozens of people use Optimizely •We strive to deliver continuous experimentation across web, mobile and connected devices •Our team maintains a close working relationship with Optimizely to maximise productivity and deliver the best results

● The BBC reaches millions ● We strive to deliver digital excellence ● Experimentation plays a vital role ● Ultimately saves time, money and mitigates risk ● We have run around 200 experiments with Optimizely Why does testing matter to the BBC?

How we generate ideas and prioritise tests ● Test ideas are generated and prioritised at product level ● Qualitative & Quantitative: data insights, usability work, internal ideas and previous tests ● Tests are prioritised based on wider BBC Objectives, product roadmaps and the PIE Model

Why knowledge sharing is important to us ● Sharing insights plays a key role in upscaling our testing programme to new teams ● We deliver a monthly newsletter to 300 subscribers and counting ● Confluence knowledge hub with historical result reports helping teams make informed decisions ● Upskilling teams via our bespoke training supports our adoption of a consistent approach to experimenting ● Host regular internal showcases, engineering meet ups and workshops

The challenges we face As we grow our programme across more teams and touch points we face some challenges • Multiple tools are used to document ideas and information • Different teams, different preferences leading to multiple places with similar information • Ability to measure maturity, and track experiment end-to-end velocity as a BBC overview and product level • A need for splitting out BBC tests by type to identify expansion into other forms of testing • Demonstrating and visualising test scale across the BBC

How Program Management will help the BBC As we continue to increase our pace of innovation, we are excited about what Program Management has to offer. • Consolidate multiple tools into a single slick environment • One stop shop for past results which are easily found • Improves collaboration and encourages transparency, allowing anyone within a product to add ideas • Drives efficiency of our end-to-end workflow • Offers progress reporting at BBC level and product level • Supports our goal to scale Optimizely to more teams

Learn More Visit optimizely.com/program-management

Introducing Program Management

Introducing Program Management

More Related Content

What's hot

Similar to Introducing Program Management

More from Optimizely

Recently uploaded

In this document

Introducing Program Management