RSL-RL

RSL-RL is a GPU-accelerated, lightweight learning library for robotics research. Its compact design allows researchers to prototype and test new ideas without the overhead of modifying large, complex libraries. RSL-RL can also be used out-of-the-box by installing it via PyPI, supports multi-GPU training, and features common algorithms for robot learning.

Key Features

• Minimal, readable codebase with clear extension points for rapid prototyping.
• Robotics-first methods including PPO and Student-Teacher Distillation.
• High-throughput training with native Multi-GPU support.
• Proven performance in numerous research publications.

Learning Environments

RSL-RL is currently used by the following robot learning libraries:

• Isaac Lab (built on top of NVIDIA Isaac Sim)
• Legged Gym (built on top of NVIDIA Isaac Gym)
• mjlab (built on top of MuJoCo Warp)
• MuJoCo Playground (built on top of MuJoCo MJX and Warp)

UW-Lab integration

This integration targets the upgraded UWLab workflow on Isaac Sim 6.1 and the upstream RSL-RL 5.4.1 API with separate actor and critic models. RSL-RL itself remains simulator-independent. The old combined ActorCritic interface is not restored. Keep old callers on the pinned 3.x stack; use the upgraded UWLab checkpoint-layout conversion and 5.x model/export interfaces for this stack.

Distributed PPO retains tensor broadcasts including normalization buffers and optional RND state. An existing distributed process group is reused rather than initialized twice.

Installation

Before installing RSL-RL, ensure that Python 3.9+ is available. It is recommended to install the library in a virtual environment (e.g. using venv or conda), which is often already created by the used environment library (e.g. Isaac Lab). If so, make sure to activate it before installing RSL-RL.

Installing RSL-RL as a dependency

pip install rsl-rl-lib

Installing RSL-RL for development

git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl
pip install -e .

Citation

If you use RSL-RL in your research, please cite the paper:

@article{schwarke2025rslrl,
  title={RSL-RL: A Learning Library for Robotics Research},
  author={Schwarke, Clemens and Mittal, Mayank and Rudin, Nikita and Hoeller, David and Hutter, Marco},
  journal={arXiv preprint arXiv:2509.10771},
  year={2025}
}