Evolving Agile Testing Fran O’Hara, Inspire Quality Services www.eurostarconferences.com @esconfs #esconfs
We provide Agile, Quality and Process Improvement Services such as  Consulting/Coaching: – Strategic advice and hands-on Coaching/mentoring in areas such as agile, testing, process improvement, etc.  Training public/inhouse: – Lean/Agile: Certified/Advanced ScrumMaster, Succeeding with Agile/Scrum, Agile project management, Agile Testing, Product Owner training, Lean/Kanban, etc. – Testing (ISTQB Foundation and Advanced Test Manager/Analyst, Risk-based testing, Test design techniques, Testing for developers, TMap®, Peer Reviews, UAT, etc.) – Requirements/Business analysis – Software project management  Assessments – Agile practices – Industry standards and models such as CMMI®, TPI®, TMMi®, etc.
Agenda  Setting the scene  Requirements and Testing  Agile Test Strategy  Tester role
The Major Agile/Lean Methods • Scrum (1995) – PM Oriented – Timeboxing – Prioritized backlog – Daily standup meetings – Demo after each iteration – Correct the process through lessons learned XP (1999) – Engineering Oriented • (A)TDD, refactoring, pair programming, continuous integration, simplicity, whole team, planning game, … Kanban(2010) – Continuous Improvement Visualize Reduce WIP Manage Flow Make process Policies Explicit Nurture effective feedback loops Improve Collaboratively (using scientific method) 4
Some Underlying Shifts…
6 How agile changes things • Whole Team Approach - collaboration • Coding and testing are integrated rather than distinct phases • Early and frequent feedback • TDD/ATDD practices • Test-infected developers, better automation strategies, better designed tests Always working software
Agenda  Setting the scene  Requirements and Testing  Agile Test Strategy  Tester role
User stories - CCC As a <role> I need <action> so that <result> Card As a customer I search for users so that I can view their details Value: Med Risk: Low Estimate: 3 pts INVEST Conversation Confirmation •I can find users •I can use any search criteria I need •Once found I can view details •….  Independent  Negotiable  Valuable  Estimable  Small  Testable BDD: In order to <benefit> As a <role> I want <functionality>
Purpose of confirmation/acceptance criteria define the boundaries for a user story/feature help the product owner answer what she needs in order for this feature to provide value (typically these are the minimum functional requirements) help the team gain a shared understanding of the story/feature help developers and testers to derive tests help developers know when to stop adding more functionality to a story 9
Level of detail in confirmation/acceptance criteria Try to keep relatively high level Detail goes into • Memory or internal team documentation (notes) – can include wireframes and screen mockups, validation rules, etc. • Automated acceptance tests (ideally defined to be understood by all) Note: examples make excellent acceptance criteria (see Specification by example by Gojko Adzic) 10
Agenda  Setting the scene  Requirements and Testing  Agile Test Strategy  Tester role
Agile Testing Quadrants
The Automation Pyramid Manual Tests e.g. exploratory GUI layer e.g. Selenium API/Service layer Acceptance Tests e.g. Fitnesse, Cucumber Unit/Component layer Developer Tests e.g. JUnit Automate at feature/workflow level Automate at story level Automate at design level Based on Mike Cohn
Maintaining Context PRIORITY GRANULARITY 14
Sprints and Testing Strategy Sprint 1 Dev + Test* Sprint 2 Dev + Test* Sprint 3 Dev + Test* Additional testing Additional Testing *Sprint test = Automated Unit & Acceptance, Manual Exploratory Within a Sprint may need to perform additional testing as part of a defined but adaptive testing strategy e.g.: – Feature/’epic’ or workflow level testing – Combination/feature interaction testing – Business cycle & end-to-end scenario testing – exercising multiple stories, end of month processing, etc. – Performance testing – Usability testing – Security testing – System integration testing  Note: Ideally any testing needed should be included within the Sprint rather than being deferred….otherwise are we defining one ‘Done’ for Sprints and another for Release…? Evolve to fully Working Software!! Additional testing …
Agenda  Setting the scene  Requirements and Testing  Agile Test Strategy  Tester role
Role of tester • Involved continuously from start…e.g. – Facilitate communication between the technical & business stakeholders – Support early validation of requirements – Help the customer/business stakeholders define acceptance criteria – Support creation of automated acceptance tests • Or define for developers to script – Expand scope of ‘acceptance’ tests • see agile testing strategy – Advise the team about overall risks and trends – Perform manual/exploratory tests • Needs ‘technical awareness’
Role of developer in relation to testing • Perform code reviews • Use static analysis tools • Perform effective unit testing – Automated ideally using TDD achieving structural coverage • Perform component/component integration testing – Again ideally automated, API/Service level • Support testers in Acceptance/System testing – Frameworks for automated API/Service level testing – GUI based automation 18
From: Lisa Crispin, 2011 19
Evolving from sequential to iterative/incremental Sprint 1 Sprint 2 Code Code Code & Bug Fix Test Sprint 1 Sprint 2 Code Code & Bug Fix Test Code Code & Bug Fix Test Sprint 1 Sprint 2 Code & Bug Fix Test Code & Bug Fix Test A B C
Key Opportunities to Innovate & Renovate 21 Backlog grooming
Fran O’Hara InspireQS www.inspireqs.ie fran.ohara@inspireqs.ie Q&A

Fran O'Hara - Evolving Agile Testing - EuroSTAR 2012

  • 1.
    Evolving Agile Testing Fran O’Hara, Inspire Quality Services www.eurostarconferences.com @esconfs #esconfs
  • 2.
    We provide Agile,Quality and Process Improvement Services such as  Consulting/Coaching: – Strategic advice and hands-on Coaching/mentoring in areas such as agile, testing, process improvement, etc.  Training public/inhouse: – Lean/Agile: Certified/Advanced ScrumMaster, Succeeding with Agile/Scrum, Agile project management, Agile Testing, Product Owner training, Lean/Kanban, etc. – Testing (ISTQB Foundation and Advanced Test Manager/Analyst, Risk-based testing, Test design techniques, Testing for developers, TMap®, Peer Reviews, UAT, etc.) – Requirements/Business analysis – Software project management  Assessments – Agile practices – Industry standards and models such as CMMI®, TPI®, TMMi®, etc.
  • 3.
    Agenda  Settingthe scene  Requirements and Testing  Agile Test Strategy  Tester role
  • 4.
    The Major Agile/LeanMethods • Scrum (1995) – PM Oriented – Timeboxing – Prioritized backlog – Daily standup meetings – Demo after each iteration – Correct the process through lessons learned XP (1999) – Engineering Oriented • (A)TDD, refactoring, pair programming, continuous integration, simplicity, whole team, planning game, … Kanban(2010) – Continuous Improvement Visualize Reduce WIP Manage Flow Make process Policies Explicit Nurture effective feedback loops Improve Collaboratively (using scientific method) 4
  • 5.
  • 6.
    6 How agilechanges things • Whole Team Approach - collaboration • Coding and testing are integrated rather than distinct phases • Early and frequent feedback • TDD/ATDD practices • Test-infected developers, better automation strategies, better designed tests Always working software
  • 7.
    Agenda  Settingthe scene  Requirements and Testing  Agile Test Strategy  Tester role
  • 8.
    User stories -CCC As a <role> I need <action> so that <result> Card As a customer I search for users so that I can view their details Value: Med Risk: Low Estimate: 3 pts INVEST Conversation Confirmation •I can find users •I can use any search criteria I need •Once found I can view details •….  Independent  Negotiable  Valuable  Estimable  Small  Testable BDD: In order to <benefit> As a <role> I want <functionality>
  • 9.
    Purpose of confirmation/acceptance criteria define the boundaries for a user story/feature help the product owner answer what she needs in order for this feature to provide value (typically these are the minimum functional requirements) help the team gain a shared understanding of the story/feature help developers and testers to derive tests help developers know when to stop adding more functionality to a story 9
  • 10.
    Level of detailin confirmation/acceptance criteria Try to keep relatively high level Detail goes into • Memory or internal team documentation (notes) – can include wireframes and screen mockups, validation rules, etc. • Automated acceptance tests (ideally defined to be understood by all) Note: examples make excellent acceptance criteria (see Specification by example by Gojko Adzic) 10
  • 11.
    Agenda  Settingthe scene  Requirements and Testing  Agile Test Strategy  Tester role
  • 12.
  • 13.
    The Automation Pyramid Manual Tests e.g. exploratory GUI layer e.g. Selenium API/Service layer Acceptance Tests e.g. Fitnesse, Cucumber Unit/Component layer Developer Tests e.g. JUnit Automate at feature/workflow level Automate at story level Automate at design level Based on Mike Cohn
  • 14.
  • 15.
    Sprints and TestingStrategy Sprint 1 Dev + Test* Sprint 2 Dev + Test* Sprint 3 Dev + Test* Additional testing Additional Testing *Sprint test = Automated Unit & Acceptance, Manual Exploratory Within a Sprint may need to perform additional testing as part of a defined but adaptive testing strategy e.g.: – Feature/’epic’ or workflow level testing – Combination/feature interaction testing – Business cycle & end-to-end scenario testing – exercising multiple stories, end of month processing, etc. – Performance testing – Usability testing – Security testing – System integration testing  Note: Ideally any testing needed should be included within the Sprint rather than being deferred….otherwise are we defining one ‘Done’ for Sprints and another for Release…? Evolve to fully Working Software!! Additional testing …
  • 16.
    Agenda  Settingthe scene  Requirements and Testing  Agile Test Strategy  Tester role
  • 17.
    Role of tester • Involved continuously from start…e.g. – Facilitate communication between the technical & business stakeholders – Support early validation of requirements – Help the customer/business stakeholders define acceptance criteria – Support creation of automated acceptance tests • Or define for developers to script – Expand scope of ‘acceptance’ tests • see agile testing strategy – Advise the team about overall risks and trends – Perform manual/exploratory tests • Needs ‘technical awareness’
  • 18.
    Role of developerin relation to testing • Perform code reviews • Use static analysis tools • Perform effective unit testing – Automated ideally using TDD achieving structural coverage • Perform component/component integration testing – Again ideally automated, API/Service level • Support testers in Acceptance/System testing – Frameworks for automated API/Service level testing – GUI based automation 18
  • 19.
  • 20.
    Evolving from sequentialto iterative/incremental Sprint 1 Sprint 2 Code Code Code & Bug Fix Test Sprint 1 Sprint 2 Code Code & Bug Fix Test Code Code & Bug Fix Test Sprint 1 Sprint 2 Code & Bug Fix Test Code & Bug Fix Test A B C
  • 21.
    Key Opportunities toInnovate & Renovate 21 Backlog grooming
  • 22.
    Fran O’Hara InspireQS www.inspireqs.ie fran.ohara@inspireqs.ie Q&A