People — the Main Challenge for Navigation Algorithms
Robots are increasingly finding themselves in spaces where they must share the road with people: stores, hospitals, city streets. The ability to correctly interpret commands while not violating the personal boundaries of passersby is becoming critically important. However, most existing approaches to Vision-and-Language Navigation (VLN) have been tested in conditions where pedestrians are either absent entirely or their behavior is simplified to the extreme. This leads to even advanced agents getting lost in real-world settings.
Researchers have drawn attention to this gap and introduced a new open benchmark — HA-VLN 2.0. It is designed to test a robot's ability to simultaneously execute navigation commands and behave in a socially acceptable manner: avoiding people, not colliding with them, and maintaining distance.
From Discrete Maps to Living Streets
Traditional VLN benchmarks were divided into discrete and continuous ones. In the former case, the agent moves between pre-defined points and does not worry about the smoothness of motion. In the latter, free space appears, but in most scenarios it remains static: pedestrians do not move or are barely accounted for. Meanwhile, in reality, people are constantly walking toward each other, changing trajectories, and interacting with one another.
The authors of the paper on ArXiv:2503.14229 (accepted to IROS 2026) propose a unified approach: HA-VLN 2.0 combines both discrete and continuous environments, adding explicit social constraints to them. That is, before evaluating route accuracy, it is checked whether the agent has violated a person's personal space. This formulation of the task is significantly closer to the conditions in which robots will actually have to operate.
The paper, spanning 35 pages and 20 figures, describes in detail both the benchmark architecture and the obtained results. Twelve authors from academic and industrial laboratories worked on the project, including researchers from Carnegie Mellon University (Alexander G. Hauptmann) and other institutions.
HA-VLN 2.0: A Unified Standard for Social Navigation
The new benchmark includes several key components. First, there are standardized metrics that simultaneously measure the accuracy of goal achievement and the degree of personal space compliance. A simple metric like "percentage of successful episodes" is here supplemented with penalties for invading pedestrians' comfort zones.
Second, the researchers created an updated dataset, HAPS 2.0, and simulators that model interaction with multiple people at once. Not only indoor environments are considered, but also open spaces, as well as more precise alignment of language with gestures and movements. This allows testing agents in conditions close to real urban scenarios.
To verify the viability of the approach, the authors ran benchmarks on 16,844 socially grounded instructions. This number of examples allows for reliable assessment of algorithm behavior in different situations. It is specifically emphasized that the benchmark is published along with datasets, simulators, baselines, and protocols — everything needed to reproduce the experiments.
What the Tests Showed: Social Awareness Matters
The benchmark revealed a serious problem: the performance of leading navigation agents drops sharply when human dynamics and partial observability are added. Agents that performed excellently in static environments begin to collide with pedestrians or get stuck when they appear in the field of view unexpectedly.
At the same time, experiments showed that explicit social modeling provides tangible advantages. If an agent is pre-trained to account for the presence of people and predict their movement, navigation robustness improves and the number of collisions decreases. This confirms the need for human-centric approaches instead of simply finding the shortest path.
Importantly, the findings are verified not only in simulation. The authors conducted tests on a real robot, which confirmed the transfer of results from the virtual environment to the physical one. This simulation-to-reality transition is a mandatory condition for practical application, and HA-VLN 2.0 offers the infrastructure for its validation.
An Open Ecosystem for Advancing Robotics
One of the important aspects of the project is openness. Along with the paper, not only descriptions are published, but also working tools: datasets, simulators, baseline algorithm codes, and training protocols. In addition, a leaderboard has been created where teams can compare their navigation agents under transparent conditions. This is especially valuable because it allows tracking progress in the field of social navigation and avoiding inflated results on closed test sets.
The development of such benchmarks is an important step toward robots ceasing to be laboratory toys and learning to safely coexist with people in real life. New metrics and simulators push the community toward research focused on greater social awareness, not just technical accuracy.
In the future, we can expect an expansion of scenarios: more complex weather conditions, different types of pedestrians, and additional modalities in instructions. But even now, HA-VLN 2.0 sets a new standard for evaluating navigation algorithms oriented toward interaction with society.
