The shortage of real-world data is what’s holding back progress in learning for robotic manipulation today. Do you need a whole factory of robots and hours of teleoperation to train advanced policies? Not always. Grabette offers another path: record demonstrations with your human hand, a gripper, and cameras to recover 6-DoF trajectories ready to train robots.
¿Qué es Grabette?
Grabette is an open, low-cost system for recording manipulation data. Think of a hand-held gripper with sensors: a Raspberry Pi, a narrow or fisheye Pi camera for context, an RGBD OAK-D for robust tracking, magnetic encoders and a common clock to synchronize everything.
The idea is simple and powerful. Grabette captures what your hand does while you manipulate objects, processes the signal, and delivers a dataset in LeRobot format ready for the Hugging Face Hub. You don’t need a robot arm or an expensive lab to generate useful demonstrations.
Grabette turns a human demonstration into a
camera-local 6-DoF + gripper statesequence, synchronized and ready for learning.
Arquitectura y flujo de datos
Grabette splits roles between two cameras by design. The fisheye camera gives a wrist-style context view, handy for visuomotor policies that need to understand the scene. The RGBD camera does the heavy lifting: depth, color and IMU for SLAM and robust trajectory recovery.
Registro y sincronización:
- A single shared clock synchronizes the observation camera, the tracking camera (color, depth, IMU) and the gripper encoder values.
- Press the button to start a take; press again to stop and the session is saved locally on the Raspberry Pi.
- Then open the dashboard in your browser and with one click start the post-processing that generates the
LeRobotpackage.
What the pipeline produces:
- 6-DoF trajectories in the camera frame (Cartesian pose per frame).
- Gripper states (open, closed, finger positions) synchronized to those poses.
- Episode metadata: timestamps, IMU, depth frames and context camera video.
Compatibility and robot-agnostic design:
- The data does not assume a particular arm. When exported to
LeRobotit can be consumed by different controllers and learning architectures. - For physical execution, Pollen Robotics offers
Gripette, the robot gripper that shares the same hardware DNA. But conceptually, the data can serve other arms if you do the necessary adaptation.
Detalles técnicos útiles para investigadores
Grabette integrates well-known techniques from the field:
- SLAM on the RGBD + IMU camera to reconstruct the 6-DoF trajectory.
- Sensor synchronization to train temporal policies and models that consume multiple modalities.
LeRobotformat designed for interoperability with training and deployment pipelines.
What kinds of models can use this data? Visual-lingual transformer policies (VLA), diffusion or flow-matching policies, and even world models that learn dynamics from videos and poses. The key is that the data is real, varied and precisely time-synchronized, which reduces the gap between learning and real execution.
Some quality considerations:
- Initial calibration of cameras and encoders improves pose fidelity.
- Recording both successes and failures improves learning robustness.
- Accurate timestamps make training recurrent nets or temporal attention much easier.
Ejemplo práctico y resultados
To show the full cycle, the team publishes an example trained with 200 demonstrations. It’s not a magic bullet: it’s proof that with a moderate number of real demos you can train policies that are then evaluated in robotic setups.
If you work with this, watch metrics like task success rate, generalization to new objects and sim2real transition distance. For research, comparing models trained from scratch versus fine-tuning pretrained models with these demos usually yields useful insights about dataset efficiency.
Cómo puedes contribuir y buenas prácticas
Grabette’s strength is the community. The goal is that anyone can build one on a workbench, record everyday tasks and share episodes. Some practical recommendations:
- Start with a single behavior and record between 200 and 500 varied demos to cover context.
- Include variations in lighting, object position and deliberate failures to boost robustness.
- Upload episodes from the dashboard; browser-based post-processing avoids complex installs.
- Annotate metadata: task description, conditions, success or failure and tags that let people filter data.
Upcoming developments already announced include Casquette, a head-mounted device for POV captures that complement Grabette. The invitation is simple: if more hands record, models learn better and faster.
Reflexión final
Grabette lowers the barrier to generate manipulation data: accessible hardware, an open format and a clear path from human demo to a trainable dataset. What would change if anyone could record tasks and upload them to a common repository? We’d likely speed up the creation of more general policies and rely less on closed installations.
If you care about applied research, practical robotics or simply want to help build an open dataset, this is one of those initiatives where your contribution really matters.