Skip to content

TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

Notifications You must be signed in to change notification settings

UT-Austin-RobIn/telemoma

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

Shivin Dass1, Wensi Ai2, Yuqian Jiang1, Samik Singh1, Jiaheng Hu2, Ruohan Zhang1, Peter Stone1,3, Ben Abbatematteo1, Roberto Martín-Martín1

1The University of Texas at Austin, 2Stanford University, 3Sony AI

[Paper], [Project Website]

TeleMoMa is a teleoperation toolkit that enables intuitive teleoperation of high-DoF mobile manipulators. We provide a common pipeline that supports several teleoperation interfaces such as vision, VR, mobile phones and more, and enables mixing-and-matching between them. While we provide code for several mobile manipulators in sim and real, the versatility of TeleMoMa allows easy plug-and-play teleoperation of any mobile manipulator in general.

Setup

System Requirements

  • Ubuntu 20.04
  • Python 3.9+

Other system configurations may work, but we haven’t tested them extensively and may only be able to provide limited support.

Note: TeleMoMa can work with higher versions of python but if iGibson sim environment is required then please use python3.9.

Installation

git clone https://github.com/UT-Austin-RobIn/telemoma.git
cd telemoma
conda create --name telemoma python==3.9
conda activate telemoma
pip install -e .

Install iGibson (Optional)

We provide a demo script to test different teleoperation modes in the iGibson sim environment. iGibson can be installed by running,

(telemoma) > pip install igibson

In case of installation or runtime issues with iGibson, please checkout their exhaustive documentation and github.

Usage

Quickstart

To verify your installation the following code can be used to run TeleMoMa in a sandbox environment in iGibson.

(telemoma) > python telemoma/demo_igibson.py --robot tiago --teleop_config telemoma/configs/only_keyboard.py 

--teleop_config defines the input modality to be used for controlling each part of the robot. Some example configurations are provided in telemoma/configs folder.

Make sure the input hardware is correctly setup by following the instructions provided in Human Interface.

Human Interface

Hardware specific setup instructions for devices compatible with TeleMoMa are provided here.

Robot Interface

The details on the sim environment installations, real robot codebase and instructions on setting up TeleMoMa on your own robot are provided here.

Citation

@article{dass2024telemoma,
  title={TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation},
  author={Dass, Shivin and Ai, Wensi and Jiang, Yuqian and Singh, Samik and Hu, Jiaheng and Zhang, Ruohan and Stone, Peter and Abbatematteo, Ben and Martín-Martín, Roberto},
  journal={arXiv preprint arXiv:2403.07869},
  year={2024}
}

Acknowledgements: Mentioned here

About

TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation

Topics

Resources

Stars

Watchers

Forks

Languages