Running may seem like a completely independent activity, but for many blind or low-vision athletes, it still requires a human guide, a physical tether, or a painted line on the track. Google wants to change that reality with the Running Guide agent, an artificial intelligence system that interprets the surroundings in real time and provides audio instructions while the person runs.
The proposal is still in development, but it points to a powerful idea: allowing a runner with low vision to train with greater autonomy, without constantly depending on someone else to detect obstacles, curves, or changes in the terrain.
How the Running Guide agent works
The system initially uses a Pixel 10 Pro placed on the athlete’s chest. The phone observes the path and delivers guidance through directional sounds and voice alerts.
Rather than simply following a fixed route, the agent tries to understand what is happening in front of the runner. It can identify sudden changes in the surface, obstacles, and situations that require an immediate reaction.
Google designed an architecture with two processing paths to balance speed and understanding:
- On-device segmentation: works without an internet connection and analyzes the path with very low latency. It can issue a "STOP" alert or directional sounds to help the athlete correct their trajectory.
- Advanced understanding with Gemma 4: the Gemma 4 E4B model analyzes images and text to interpret more complex scenes. This allows it to distinguish, for example, between an upcoming curve, a change in terrain, or an unexpected obstacle.
What’s the key? Not processing absolutely everything. Using a technique called Smarter Frame Selection, the system selects the frames containing the most information, such as those where a new object appears or the surface changes. This reduces the model’s workload and allows it to respond more quickly.
Three agents to support training
The Running Guide agent is not a single, isolated feature. Google presents it as a system made up of several specialized agents, each with a specific responsibility during the session.
Planner agent
Before getting started, the planning agent talks with the runner to learn about their goals. It can also check weather data and Google Maps, as well as help set the starting point and configure the workout.
In practice, this could help prepare a session based on the distance, weather conditions, or location chosen for the run.
Coach agent
During the run, the coaching agent provides brief, direct instructions. It is not designed to maintain a long conversation while the athlete is moving, but to communicate only what is necessary at the right moment.
Its alerts are organized into three levels:
- DANGER: there is an immediate hazard and the runner must react.
- WARNING: there is an obstacle or person nearby that requires attention.
- NOTICE: a curve or another relevant change in the route is approaching.
This hierarchy is important. During a fast-paced activity, receiving too many instructions can be just as problematic as receiving none. The system must prioritize safety and avoid overwhelming the user.
Break agent
The break agent manages pauses. The athlete can stop the session and resume it later without having to configure everything again.
From a phone on the chest to smart glasses
Although the Pixel 10 Pro provides a functional foundation, Google is also testing the system with smart glasses. This format could provide a wider, more stable field of view than a phone placed on the chest.
The glasses would send images to the Pixel phone, combining wearable hardware with the artificial intelligence installed on the device. Local processing could reduce dependence on a cellular connection and keep responses fast.
Even so, speed is not the only challenge. When a technology provides instructions during a run, trust and safety are essential. A late alert, an incorrect detection, or an ambiguous instruction could have real consequences.
The community takes part in testing
Google is working with SG Enable, Singapore’s agency dedicated to inclusion and disability. This collaboration allows blind and low-vision runners to test the system in real-world situations and provide feedback throughout its development.
This point deserves attention. An accessibility tool should not be designed solely in a laboratory. Who can better say what kind of sound is understandable while running, or which alerts are truly useful, than the people who will use the system?
The Running Guide agent represents an interesting step in the evolution of AI assistants. It is no longer just about answering questions or generating text, but about interpreting a physical environment and supporting an activity in real time.
For now, the project remains in the development and prototyping phase. Its true impact will depend on its accuracy, safety, and ability to adapt to each athlete’s needs. But the direction is clear: use AI to help more people move, train, and compete with greater independence.
Original source
https://blog.google/innovation-and-ai/models-and-research/google-deepmind/running-guide-agent