Download as PDF, PPTX
Uploaded byOptimizely
A/B Testing for WordPress & Drupal
This document discusses experimentation and A/B testing. It provides an overview of what A/B testing is and how to effectively conduct experiments. Key points include: - A/B testing involves comparing two variations of a webpage to see which performs better. - To set up an experiment, you need analytics tools, a hypothesis to test, ways to create test variations, and metrics to analyze results. - Case studies show how experiments can significantly impact goals like reducing returns or increasing signups. - Platforms like Optimizely make experimentation easy for both technical and non-technical users.
A/B Testing for WordPress & Drupal
- 2. Matt Stodolnic VP Marketing& Alliances Pantheon Brad Taylor Senior Frontend Engineer Optimizely Matthew Auerbach Developer Evangelist Optimizely
- 3.
- 5.
- 7. What is A/BTesting? Comparing two variations of a webpage to see which performs better
- 8. Hypothesis: Attracting moreattention to the SHOP NOW button will increase impressions to the item catalog A B
- 9. Business Objective: Reducecost of free shipping program and deliver better online shopping experience Goal: Encourage visitors to purchase only one instead of multiple sizes Experiment: Target a helpful modal to ‘size-conscious’ shoppers to assist them with finding the correct fit Impact: 80% decrease in return rate for ‘size-conscious’ shoppers Case Study Brooks
- 10.
- 11.
- 12. The Optimizely Snippet https://help.optimizely.com/Set_Up_Optimizely/Implement_the_Optimizely_snippet Eachproject has its own project code, called the snippet. Optimizely snippet contents: • Active Experiments / Campaigns* • Draft Experiments / Campaigns* • Project Metrics • Project Audiences • Extensions* • Optimizely execution and tracking logic * Optimizely can be configured to either remove these from the snippet or load them asynchronously.
- 13. Drupal Module Makes iteasy to define which pages to load the Optimizely JS, no native editor functionality. WordPress Install JS snippet directly in the head module Easy Access to A/B Testing
- 15. ● Secure thenecessary tools ● Observe user behavior ● Develop a hypothesis ● Create tests for your hypothesis ● Analyze results
- 16. ● Google Analyticsor other analytics tool (make sure tools are working). ● Optimizely or other easy to use, easy to setup (SaaS/JS include) testing tool. ● Crazy Egg or other user behavior / click tracking tool
- 17. Google Behavior Flow. Whereare people going on our website? Click-path analysis (by URL) Crazy Egg What are people clicking on our website? More visual click-path analysis (by clicks)
- 18.
- 19.
- 22. ● New Messagingwill connect with our visitors and more our our visitors will navigate to the tools pages of our website. ● Sign-Ups will increase because we are our value propositions are more aligned to our audience.
- 24. ● A| BTesting Almost anything on your website that affects visitor behavior can be A/B tested. ● Testing Complete Pages like Pantheon did for this test requires more planning for redirection of traffic and measuring multiple changes at once.
- 28. 50% - CONTROL50% - VARIATION
- 29. ● Engagement -% of visitors who clicked on any part of the experiment page ● Traffic to new user-specific landing pages - % of visitors who visited any of the three new landing pages ● Developer Registration - % of visitors who registered for a free developer account
- 30.
- 33. ● New Homepage- A new homepage is both an A/B test and a multivariate test. ● Just Start Testing - Agree with testing purists and move on. Improve the site, create new tests. Improve the site, create new tests.
- 36. • Improve theuser experience • Verify new product features • Drive specific KPIs • Test messaging • Measure algorithm performance • Build a culture of experimentation Why is A/B testing important?
- 37. Working with yourorganization
- 38. Enable Rapid Experimentationby Everyone Built for business users See results in real-time Act with confidence and developers Always-on experimentation
- 39. Experiment across yourorganization and business objectives Marketing Data Science Engineering Operations Product Grow revenue Reduce costs Build brand Maximize Profits Experiment to roll out new features Experiment to optimize your site’s functionality Experiment with user experience to drive engagement Experiment to better understand recommendation algorithms Experiment for lead generation and conversion rate
- 40.
- 41. 1920s Data is expensive Datais slow Practitioners are trained experts 2017 Data is cheap Data is real-time Everyone is a practitioner Back then, stats was hard!
- 42. Before Stats Engine:Lots of work to get valid results To get valid results for a test, you had to follow some rules: ● Set a sample size in advance ● No peeking ● Can’t test too many goals and variations
- 43. After Stats Engine:Always valid results to make decisions in real-time Sequential testing Look at results in real time False discovery rate control Test as many goals and variations as needed = Always valid results with Stats Engine
- 44. Optimizely platform froma dev perspective Experiment SDKs Developer Tools REST API Behavior API Event API Data Export Extensions Web Tools Customer Profiles
- 45. Optimizely X: OperationalizeExperimentation Across the Organization Enable Rapid Experimentation by Everyone Experiment Everywhere on Any Channel and Device Deliver High-Performing Personalized Experiences Optimize Anything that Matters to your Business
- 47. Extensions • Reusable templatesfor making changes in the visual editor. • A developer can build a extension with HTML, CSS, and JavaScript • Customizable fields allow any user to alert the Extension • Pop-up message, banner, countdown timer, etc… • Sample Library
- 48.
- 49. Get started withexperimentation today! Documentation and developer information: developers.optimizely.com Free trial: optimizely.com/free-trial