Testing! Web Applications presented by Seth McLaughlin @sethmc
E N G I N E E R I N G www.shapesecurity.com
testing is a BIG topic.
how to effectively test JavaScript code in your web applications?
Workflow.
Start Commit Code Staging QA Sign-off In ProdCode Review DEV Right away! PM Build me feature X ASAP!
Start Commit Code Staging QA Sign-off In ProdCode Review DEV working for the man…
Start Commit Code Staging QA Sign-off In ProdCode Review every night and day! DEV
Start Commit Code Staging QA Sign-off In ProdCode Review DEV everyone’s a critic
Start Commit Code Staging QA Sign-off In ProdCode Review DEV hallelujah!
Start Commit Code Staging QA Sign-off In ProdCode Review DEV my work here is done.
Start Commit Code Staging QA Sign-off In ProdCode Review DEV my work here is done. QA we’ll see about that!
Start Commit Code Staging QA Sign-off In ProdCode Review QA ship it!
Start Commit Code Staging QA Sign-off In ProdCode Review CUSTOMER feature X is awesome!
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 0 days start coding!
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 7 days start code review
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 9 days commit code
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 10 days deploy code to staging
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 12 days QA finds a bug in staging
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 13 days fix bug
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 15 days code review fix
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 17 days commit fixed code
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 17 days deploy (again) to staging
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 19 days QA is happy
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 20 days shipped to production
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 6 days Dev finds the bug while coding
Start Commit Code Staging QA Sign-off In ProdCode Review elapsed time: 15 days could have saved over 5 days by catching bug sooner alternatively…
find bugs as early as possible.
Static Analysis Start Commit Code Staging QA Sign-off In ProdCode Review var config = {! name: this.name,! style: 'solo',! }; blows up in IE7
Static Analysis Start Commit Code Staging QA Sign-off In ProdCode Review function getObjs(paths) { return paths.map(function (p) { var obj = null; ! try { obj = require(p); } catch (e) { console.error('Cannot load path', p); } }).filter(function (obj) { return obj !== null; }); }
Static Analysis Start Commit Code Staging QA Sign-off In ProdCode Review ESLint • Find common errors and enforce coding style • Run every time you save a file! (automatically) • Fast! www.jshint.com www.eslint.org JS Hint DEV eslint ftw!
Unit Tests Start Commit Code Staging QA Sign-off In ProdCode Review • Test code at a granular level • Test as you develop (TDD) • Great traceability • Fast! Fun! DEV I ♥ unit tests visionmedia.github.io/mocha simple, flexible, fun jasmine.github.io www.qunitjs.com mocha
Continuous Integration Start Commit Code Staging QA Sign-off In ProdCode Review Pre-Commit Post-Commit example: run eslint and mocha tests example: run selenium tests hudson-ci.org Hudson jenkins-ci.org Jenkins travis-ci.org circleci.com atlassian.com Bamboo jetbrains.com TeamCity
End to End Tests Start Commit Code Staging QA Sign-off In ProdCode Review • Scenario driven • Tests client and server code • More “realistic” vs. Unit Tests • Less traceability vs. Unit Tests • Slower to execute vs. Unit Tests I ♥ selenium QAseleniumhq.org Selenium saucelabs.com browserstack.com
Techniques.
End to End tests with Selenium Selenium Client API Test Script Selenium WebDriver Web Browser JavaScript, Java, C#, Python, … Converts commands to HTTP requests Communicates with web browser ...
End to End tests with Selenium Selenium Client API Test Script Selenium Grid VM #1 VM #2 VM #5 IE 7 VM #4 IE 8 VM #6 IE 9
End to End tests with Selenium module.exports = { 'Get free VMs': function (browser) { browser .url('http://www.modern.ie') .assert.title('Modern.IE') .end(); } }; Nightwatch.js
Unit Testing with Mocha (node.js) test_file.js Node.js test_file.js code_to_test.js mocha test results
Unit Testing with Mocha (node.js) module.exports = { name: function (title, first, last) { return [title, first, last].join(); } }; formatter.js
Unit Testing with Mocha (node.js) var fmt = require('../../formatter'); var assert = require('assert'); ! describe('format', function () { it('should return full name', function () { var actual = fmt.name('Mr.', 'Bob', 'Rogers'); var expected = 'Mr. Bob Rogers'; ! assert.equal(actual, expected); }); }); formatter.spec.js
Unit Testing with Venus & Mocha (browser) test_file.js Venus Node.js Browser test_file.js code_to_test.js mocha test results
Unit Testing with Venus & Mocha (browser) function formatName(title, first, last) { return [title, first, last].join(); } ! formatter.js
Unit Testing with Venus & Mocha (browser) /** * @venus-library mocha * @venus-code formatter.js */ describe('format', function () { var actual = formatName('Mr.', 'Bob', 'Rogers'); var expected = 'Mr. Bob Rogers'; ! it('should return full name', function () { expect(actual).to.be(expected); }); }); ! formatter.spec.js
Unit Testing with Venus & Mocha (browser) <!DOCTYPE html> <html> <head> <title>Test for Formatter</title> <script type="text/javascript" src=“mocha.js”></script> <script type="text/javascript" src=“venus_client.js”></script> <script type="text/javascript" src="formatter.js"></script> <script type="text/javascript" src=“specs/formatter.spec.js”></script> <script type="text/javascript"> testLibrary.run(); </script> </head> <body> <div id="results"></div> </body> </html> test_harness.html
Example: Spy var callback = sinon.spy(); ! PubSub.subscribe('message', callback); PubSub.publishSync('message'); ! assert(callback.called === true); ! A function that records information about how it is called. sinonjs.org
Example: Mock XMLHttpRequestmyLib jQuery Web Server
Example: Mock Mock XMLHttpRequestmyLib jQuery
A fake implementation of a code dependency. Sinon.js XHR Mock var xhr = sinon.useFakeXMLHttpRequest(); var requests = []; var callback = sinon.spy(); ! xhr.onCreate = function (xhr) { requests.push(xhr); }; ! myLib.getCommentsFor("/some/article", callback); assertEquals(1, requests.length); ! requests[0].respond(200, { "Content-Type": "application/json" }, '[{ "id": 12, "comment": "Hey there" }]' ); ! assert(callback.calledWith([{ id: 12, comment: "Hey there" }]));
Code Samples End-to-End test with Selenium Unit Test node.js code Unit Test browser code 1 2 3 http://sethmcl.com/testing-web-apps/
Learn More. Introduction to writing testable JavaScript! LinkedIn Tech Talk Smashing Magazine ! Unit Testing with Venus.js ! LinkedIn Tech Talk Venusjs.org ! End to End testing with Selenium! The Selenium Project Selenium Architecture Nightwatchjs.org ! Free Windows VMs and other testing resources! modern.ie http://sethmcl.com/testing-web-apps/

Testing Web Applications

  • 1.
  • 2.
    E N GI N E E R I N G www.shapesecurity.com
  • 3.
    testing is aBIG topic.
  • 4.
    how to effectivelytest JavaScript code in your web applications?
  • 5.
  • 6.
    Start Commit CodeStaging QA Sign-off In ProdCode Review DEV Right away! PM Build me feature X ASAP!
  • 7.
    Start Commit CodeStaging QA Sign-off In ProdCode Review DEV working for the man…
  • 8.
    Start Commit CodeStaging QA Sign-off In ProdCode Review every night and day! DEV
  • 9.
    Start Commit CodeStaging QA Sign-off In ProdCode Review DEV everyone’s a critic
  • 10.
    Start Commit CodeStaging QA Sign-off In ProdCode Review DEV hallelujah!
  • 11.
    Start Commit CodeStaging QA Sign-off In ProdCode Review DEV my work here is done.
  • 12.
    Start Commit CodeStaging QA Sign-off In ProdCode Review DEV my work here is done. QA we’ll see about that!
  • 13.
    Start Commit CodeStaging QA Sign-off In ProdCode Review QA ship it!
  • 14.
    Start Commit CodeStaging QA Sign-off In ProdCode Review CUSTOMER feature X is awesome!
  • 15.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 0 days start coding!
  • 16.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 7 days start code review
  • 17.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 9 days commit code
  • 18.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 10 days deploy code to staging
  • 19.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 12 days QA finds a bug in staging
  • 20.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 13 days fix bug
  • 21.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 15 days code review fix
  • 22.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 17 days commit fixed code
  • 23.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 17 days deploy (again) to staging
  • 24.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 19 days QA is happy
  • 25.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 20 days shipped to production
  • 26.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 6 days Dev finds the bug while coding
  • 27.
    Start Commit CodeStaging QA Sign-off In ProdCode Review elapsed time: 15 days could have saved over 5 days by catching bug sooner alternatively…
  • 28.
    find bugs asearly as possible.
  • 29.
    Static Analysis Start CommitCode Staging QA Sign-off In ProdCode Review var config = {! name: this.name,! style: 'solo',! }; blows up in IE7
  • 30.
    Static Analysis Start CommitCode Staging QA Sign-off In ProdCode Review function getObjs(paths) { return paths.map(function (p) { var obj = null; ! try { obj = require(p); } catch (e) { console.error('Cannot load path', p); } }).filter(function (obj) { return obj !== null; }); }
  • 31.
    Static Analysis Start CommitCode Staging QA Sign-off In ProdCode Review ESLint • Find common errors and enforce coding style • Run every time you save a file! (automatically) • Fast! www.jshint.com www.eslint.org JS Hint DEV eslint ftw!
  • 32.
    Unit Tests Start CommitCode Staging QA Sign-off In ProdCode Review • Test code at a granular level • Test as you develop (TDD) • Great traceability • Fast! Fun! DEV I ♥ unit tests visionmedia.github.io/mocha simple, flexible, fun jasmine.github.io www.qunitjs.com mocha
  • 33.
    Continuous Integration Start CommitCode Staging QA Sign-off In ProdCode Review Pre-Commit Post-Commit example: run eslint and mocha tests example: run selenium tests hudson-ci.org Hudson jenkins-ci.org Jenkins travis-ci.org circleci.com atlassian.com Bamboo jetbrains.com TeamCity
  • 34.
    End to EndTests Start Commit Code Staging QA Sign-off In ProdCode Review • Scenario driven • Tests client and server code • More “realistic” vs. Unit Tests • Less traceability vs. Unit Tests • Slower to execute vs. Unit Tests I ♥ selenium QAseleniumhq.org Selenium saucelabs.com browserstack.com
  • 35.
  • 36.
    End to Endtests with Selenium Selenium Client API Test Script Selenium WebDriver Web Browser JavaScript, Java, C#, Python, … Converts commands to HTTP requests Communicates with web browser ...
  • 37.
    End to Endtests with Selenium Selenium Client API Test Script Selenium Grid VM #1 VM #2 VM #5 IE 7 VM #4 IE 8 VM #6 IE 9
  • 38.
    End to Endtests with Selenium module.exports = { 'Get free VMs': function (browser) { browser .url('http://www.modern.ie') .assert.title('Modern.IE') .end(); } }; Nightwatch.js
  • 39.
    Unit Testing withMocha (node.js) test_file.js Node.js test_file.js code_to_test.js mocha test results
  • 40.
    Unit Testing withMocha (node.js) module.exports = { name: function (title, first, last) { return [title, first, last].join(); } }; formatter.js
  • 41.
    Unit Testing withMocha (node.js) var fmt = require('../../formatter'); var assert = require('assert'); ! describe('format', function () { it('should return full name', function () { var actual = fmt.name('Mr.', 'Bob', 'Rogers'); var expected = 'Mr. Bob Rogers'; ! assert.equal(actual, expected); }); }); formatter.spec.js
  • 42.
    Unit Testing withVenus & Mocha (browser) test_file.js Venus Node.js Browser test_file.js code_to_test.js mocha test results
  • 43.
    Unit Testing withVenus & Mocha (browser) function formatName(title, first, last) { return [title, first, last].join(); } ! formatter.js
  • 44.
    Unit Testing withVenus & Mocha (browser) /** * @venus-library mocha * @venus-code formatter.js */ describe('format', function () { var actual = formatName('Mr.', 'Bob', 'Rogers'); var expected = 'Mr. Bob Rogers'; ! it('should return full name', function () { expect(actual).to.be(expected); }); }); ! formatter.spec.js
  • 45.
    Unit Testing withVenus & Mocha (browser) <!DOCTYPE html> <html> <head> <title>Test for Formatter</title> <script type="text/javascript" src=“mocha.js”></script> <script type="text/javascript" src=“venus_client.js”></script> <script type="text/javascript" src="formatter.js"></script> <script type="text/javascript" src=“specs/formatter.spec.js”></script> <script type="text/javascript"> testLibrary.run(); </script> </head> <body> <div id="results"></div> </body> </html> test_harness.html
  • 46.
    Example: Spy var callback= sinon.spy(); ! PubSub.subscribe('message', callback); PubSub.publishSync('message'); ! assert(callback.called === true); ! A function that records information about how it is called. sinonjs.org
  • 47.
  • 48.
  • 49.
    A fake implementationof a code dependency. Sinon.js XHR Mock var xhr = sinon.useFakeXMLHttpRequest(); var requests = []; var callback = sinon.spy(); ! xhr.onCreate = function (xhr) { requests.push(xhr); }; ! myLib.getCommentsFor("/some/article", callback); assertEquals(1, requests.length); ! requests[0].respond(200, { "Content-Type": "application/json" }, '[{ "id": 12, "comment": "Hey there" }]' ); ! assert(callback.calledWith([{ id: 12, comment: "Hey there" }]));
  • 50.
    Code Samples End-to-End testwith Selenium Unit Test node.js code Unit Test browser code 1 2 3 http://sethmcl.com/testing-web-apps/
  • 51.
    Learn More. Introduction towriting testable JavaScript! LinkedIn Tech Talk Smashing Magazine ! Unit Testing with Venus.js ! LinkedIn Tech Talk Venusjs.org ! End to End testing with Selenium! The Selenium Project Selenium Architecture Nightwatchjs.org ! Free Windows VMs and other testing resources! modern.ie http://sethmcl.com/testing-web-apps/