Chapter 5: Big Experiments

Photo by Girl with red hat on Unsplash

Now equipped with broader participant samples and more diverse stimuli, we can create Big Data experiments. This chapter reviews research methods involved in running Big Data surveys and experiments. The chapter discusses overt and covert measurements that we can collect via online experiments. The chapter then discusses practical logistics to keep in mind when running a Big Data experiment, including experimental design decisions, and a behind-the-scenes look at how data is saved online via server-side coding. Next, once you have the data from an experiment, how do you clean the data and how do you visualize it? The chapter ends with discussion on the ethical implications of collecting covert measures and the useful applications of the web-coding skills to create public-facing websites.

  • Learn server-side coding (PHP, SQL)

    • A fantastic resource to learn PHP and SQL is W3 Schools. They are what I used to learn everything about these two languages.

  • Example survey code (HTML)

    • Here is simple example HTML code for how to implement the different input tags for making a survey. To look at the code, either right-click and "Save As" on the link, or right-click on the page and "View Source".

  • Example key and mouse tracking code (HTML and Javascript)

    • Here is example HTML and Javascript code that shows how to record mouse movements, time, and key presses. To look at the code, either right-click and "Save As" on the link, or right-click on the page and "View Source".

    Also, if you want to analyze the covert mouse-tracking, refer to the instructions on the Chapter 1 page.

  • Resources for running online experiments

    • Here are resources to help you code an online experiment. These platforms are constantly changing so I will update this list when I come across any new resources.

    • jsPsych - an open-sourced Javascript-based platform for coding experiments. Does not host the experiment or save its data.
    • lab.js - a different open-sourced Javascript-based platform for coding experiments. Does not host the experiment or save its data.
    • Pavlovia - an online experiment platform that works with PsychoPy (Python-based experiment platform), jsPsych, and lab.js. It also includes a repository of experiments and can host the experiment and data. May require a paid license.
    • Gorilla - an online experiment coder with a graphical interface that can then host the experiment and data. It costs money to collect data.
    • Psytoolkit - a free online experiment platform using a proprietary, easy-to-understand, but somewhat inflexible scripting language. It manages the hosting of the experiment and data as well.
    • Labvanced - online experiment builder that allows for eye-tracking. It costs money, but manages the hosting of the experiment and data.
    • Cognition - online experiment builder. It requires you to code your experiment yourself in jsPsych. It costs money, but manages the hosting of the experiment and data.
    • Qualtrics - easy-to-use platform for creating surveys and experiments using a graphical menu (and can add in Javascript code). It costs money, but manages the hosting of the experiment and data.

    Once you have coded your experiment, look at our list of crowd-sourcing platforms to then recruit your participants.

  • Example online experiments

    • Check out the Psytoolkit Experiment Library and Survey Library for easy-to-use demonstrations and code for running some well-validated tasks in the field.