Robotic Skill Learning with Large Human Datasets

News

  • [August 2021] robomimic – We released robomimic, a framework for robot learning from demonstration. It offers a broad set of demonstration datasets collected on robot manipulation domains with RoboTurk, and learning algorithms to learn from these datasets.

  • [May 2021] Multi-Arm RoboTurk – MART was nominated as a best multi-robot systems paper finalist at ICRA 2021!

  • [December 2020] Human-in-the-Loop Imitation Learning – We recently extended RoboTurk to enable human-in-the-loop teleoperation and developed Intervention Weighted Regression, a simple and effective algorithm to learn from such interventions.

System Features

6-DoF Intuitive User Interface

We provide 6 degree of freedom intuitive motion control which maps phone movement to robot arm movement.

Simultaneous Users

RoboTurk can host multiple simultaenous users that each control a robot arm in its own workspace, as well as multiple users that control robot arms in a shared workspace, allowing for demonstrations on collaborative and adversarial tasks.

Worldwide Low-Latency Robot Teleoperation

Real-time robot control of simulated and physical robot arms from across the world. This has been stress-tested by controlling robot arms at Stanford from far locations such as China and India.

Human-in-the-Loop Intervention Mechanism

Users can watch an autonomous robot arm try to solve tasks and provide assistance when necessary, helping the robot learn from its mistakes.

Datasets
Roboturk Pilot Dataset
For our first paper (CoRL 2018), we collected 1000+ successful demonstrations on each of two challenging manipulation tasks. Click to read more and download the dataset.

137.5 hrs

robot demonstrations

22 hrs

data collection

1071

successful Picking demos

1147

successful Assembly demos

3224

total demonstrations
Roboturk Real-World Dataset
In our second paper (IROS 2019, Best Cognitive Robotics Paper Finalist), we collected over 100 hours of data, resulting in one of the largest robot datasets collected via human teleoperation, on three challenging long-horizon tasks. Click to read more and download the dataset

111 hrs

robot demonstrations

1 week

data collection

3

dexterous manipulation tasks

54

non-expert users

2144

demonstrations
robomimic v0.1
We used RoboTurk to collect a suite of demonstration datasets across several simulated and real world tasks and released the datasets along with a framework to learn from such demonstration datasetsClick to read more and download the dataset

5

simulation tasks

3

real-world tasks

6

humans of varying proficiency

2500+

human trajectories

Core System Projects

Projects here primarily focus on building and enhancing the RoboTurk system, or offering large-scale human datasets to the community.

CoRL 2018
ROBOTURK: A Crowdsourcing Platform for Robotic Skill Learning through Imitation

Initial RoboTurk system with a focus on simulation

[Arxiv] [Website]

IROS 2019
Scaling Robot Supervision to Hundreds of Hours with RoboTurk

RoboTurk on Real World Tasks

[Arxiv] [Website]

ICRA 2021
Learning Multi-Arm Manipulation Through Collaborative Teleoperation

RoboTurk on Simulated Multi-Arm Environments

[Arxiv] [Website]

Arxiv 2020
Human-in-the-Loop Imitation Learning using Remote Teleoperation

RoboTurk with Human-in-the-Loop Interventions

[Arxiv] [Website]

Arxiv 2021
What Matters in Learning from Offline Human Demonstrations for Robot Manipulation

Large-scale evaluation of learning from human datasets

[Arxiv] [Website]

Projects Using RoboTurk System

These projects use the RoboTurk system to collect datasets and then use these datasets for other purposes.

Our Team

Ajay Mandlekar

amandlek[at]stanford[dot]edu

Albert Tung

atung3[at]stanford[dot]edu

Josiah Wong

jdwong[at]stanford[dot]edu

Roberto Martín-Martín

roberto[dot]martinmartin[at]stanford[dot]edu

Yuke Zhu

yukez[at]stanford[dot]edu

Animesh Garg

garg[at]cs.stanford[dot]edu

Fei-Fei Li

feifeili[at]cs.stanford[dot]edu

Silvio Savarese

silvio[at]cs.stanford[dot]edu

Alumni

Jonathan Booher

Max Spero

Anchit Gupta

Andrew Kondrich

Matthew Ricks

Julian Gao

John Emmons

Emre Orbay

Peter Dun

Tieler Callazo

Amelia (Qingyun) Bian

Alex (Kaiyi) Fu

Roboturk in the News

<