Reinforcement Learning
Dropbear uses reinforcement learning to learn locomotion, manipulation, and whole-body behaviors in simulation before deploying to real hardware. Our RL pipeline is built on NVIDIA Isaac Lab and supports massively parallel GPU-accelerated training.
Training Pipeline
The full pipeline from training to deployment:
Train in Isaac Lab
Run thousands of parallel Dropbear simulations on GPU. Train policies using PPO, SAC, or custom algorithms with domain randomization for sim-to-real transfer.
Export to ONNX
Trained PyTorch policies are exported to ONNX format for portable, optimized inference on edge devices like Jetson Orin.
Publish to Marketplace
Package the trained skill with metadata, evaluation metrics, and deployment config. Share it on the Hyperspawn Skills Marketplace for others to use.
Deploy to Hardware
Load the ONNX policy on the robot\'s compute module. The control loop runs at 50Hz with real-time joint command output.
Training Environments
We provide pre-built Isaac Lab environments for common tasks:
| Environment | Task | Observations |
|---|---|---|
| DropbearFlatWalk | Stable bipedal walking on flat ground | Joint pos/vel, IMU, foot contacts |
| DropbearRoughTerrain | Walking over uneven surfaces, stairs, slopes | Joint pos/vel, IMU, heightmap, foot contacts |
| DropbearStand | Balance and stand from various initial poses | Joint pos/vel, IMU, CoM |
| DropbearReach | Reach and grasp objects with arms | Joint pos/vel, end-effector pose, object pose |
Getting Started
# Clone the RL repository
git clone https://github.com/Hyperspawn/dropbear_rl.git
cd dropbear_rl
# Install (requires Isaac Lab 1.0+)
pip install -e .
# Train a flat-ground walking policy
python train.py --task DropbearFlatWalk --num_envs 4096 --headless
# Evaluate a trained policy
python eval.py --task DropbearFlatWalk --checkpoint runs/latest/model.pt
# Export to ONNX for deployment
python export.py --checkpoint runs/latest/model.pt --output walking_policy.onnxReward Design
Our walking rewards combine several objectives:
- Forward velocity tracking — match a target walking speed
- Upright posture — penalize torso tilt away from vertical
- Energy efficiency — minimize joint torques and velocities
- Smooth motion — penalize jerk and sudden changes in joint commands
- Foot clearance — encourage proper swing-leg height during walking
- Symmetry — encourage symmetric gait patterns