Isaac Lab-Arena and LeRobot: evaluating VLA policies | Keryc
NVIDIA and Hugging Face integrate Isaac Lab-Arena into the LeRobot ecosystem to make the evaluation of vision, language and action (VLA) policies in simulation easier, more reproducible and scalable. The result? A technical pipeline that links open models, simulated environments and deployment to hardware like Reachy 2 running on NVIDIA Jetson Thor.
What this integration brings
The proposal addresses the three key phases of robot development: training, simulation and execution on the robot. Each stage needs different software and hardware, and the integration reduces friction between them so you can iterate policies faster.
Isaac Lab-Arena is an open framework designed for efficient, large-scale evaluation of policies in simulation. It was built in collaboration with Lightwheel and is optimized for reproducible evaluation metrics.
LeRobot EnvHub lets you share simulation environments, load them directly and reuse them for training, evaluation or teleoperation. That means less time recreating scenes and more time testing policies.
Hugging Face and NVIDIA provide pre-trained VLA models (for example GR00T N, Pi, SmolVLA), physical datasets and tools to go from simulation evaluation to inference on a real robot.
In short: more reusable environments, open policies and clear paths to take a policy from the notebook to physical robot.
Architecture and technical workflow
Think of three layers: 1) VLA models and checkpoints, 2) simulation for evaluation and dataset generation, 3) deployment on embedded hardware. The integration lets you move between those layers without rewriting interfaces.
The policy (VLA) is loaded from Hugging Face and evaluated inside Isaac Lab-Arena using LeRobot as orchestrator.
EnvHub centralizes environments: you can register a scenario, change objects or embodiments and share it with the community.
Lightwheel already provides 250+ public tasks that are now directly accessible from LeRobot.
This flow enables homogeneous benchmarks: same seeds, same virtual sensors and same metrics to compare policies like GR00T N and SmolVLA.
Which components should you know?
Isaac Lab-Arena: evaluation and task framework.
LeRobot EnvHub: environment registry and distribution.
VLA policies (GR00T N, Pi, SmolVLA) loadable from Hugging Face.
Hardware platforms (for example Reachy 2 + Jetson Thor) to bring the policy to a real robot.
How to get started: installation and an evaluation example
These are the main technical steps. Check driver and Ubuntu version compatibility before you start.
Isaac Sim 5.1 requires numpy==1.26.0. If you work with other packages that don’t support it, you’ll need to isolate environments or wait for the next release.
Install SmolVLA locally if you plan to test that policy
Notice rename_map: it’s handy when observation space keys between the policy and the environment don’t match.
--trust_remote_code allows loading code associated with a policy published on Hugging Face. Use it with caution and review linked repositories.
Best practices and guidelines for developers
Share environments and task suites in LeRobot EnvHub so others can reproduce your results.
Use instances with GPUs compatible with Isaac Sim and update drivers following NVIDIA documentation.
Run evaluations headless for CI or cluster benchmarks, and enable video for debugging and reproducibility.
Leverage Lightwheel Tasks if you want a broad bank of curated tasks.
For fast scaling, NVIDIA Brev helps provision GPU instances ready for Isaac Lab.
From simulation to the real robot
Once a policy is validated in Isaac Lab-Arena you can deploy it on robots like Reachy 2 running on Jetson Thor. The typical flow is: generate data and metrics in simulation, post-train if needed, validate safety in critical scenarios and finally run inference on hardware with real latency profiles.
The integration doesn’t remove real challenges like sim2real gaps, but it does lower the experimentation cost and makes pipelines more reproducible.
Resources and next steps
Official repositories to clone are in the commands above. Check each project’s documentation for versions and driver requirements.
If you want to benchmark alternative policies, look at GR00T N 1.5 and the policies published on Hugging Face.
Contribute by creating Isaac Lab-Arena environments and registering them in EnvHub so the whole community benefits.
The union of Isaac Lab-Arena and LeRobot speeds up open physical AI development: less friction to create, compare and deploy VLA policies. If you work in robotics, this integration saves you time and gives you a direct path from simulation to robot.
