The robotaxi division of Hello Inc. has introduced HelloWorld, a generative world model for driving. This model is designed to create on-demand driving data and facilitate interactive testing of vehicle control policies.
The team announced the model on October 12th and published the corresponding scientific paper on arXiv, as well as a project page. HelloWorld integrates scene control, continuous generation, multi-sensor observation, and efficient data output.
Engineers can directly edit vehicles, pedestrians, and their trajectories, after which the model generates corresponding videos from multiple cameras along with lidar data. Examples of such edits include adding a pedestrian crossing the road, changing the route of a leading vehicle, or altering weather conditions—for instance, transitioning from rain to snow, or from day to night.
The model also accepts simple language descriptions of situations as a second method for building scenes. The user describes the desired scenario, and a specialized application pipeline converts this description into the conditions required for the model to operate. Elements such as construction sites, heavy traffic, pedestrians crossing the road, fog, and various lighting conditions can be used to guide the generation process.
The primary goal of the development is to fill gaps in real-world collected data. Long-tail scenarios, such as rare hazards or unusual combinations of weather and movement, are difficult to collect in sufficient quantity solely by gathering road data. The ability to generate such scenarios on demand allows developers to create targeted training data and conduct counterfactual tests: reproducing the same moment while changing one element to assess the reaction of a driving policy.
According to the presented scientific paper, HelloWorld contains approximately 2 billion parameters, generates views for seven cameras with conditional lidar output, and utilizes multi-step causal generation. This architecture is intended to ensure sufficient output speed for repeated interactive simulation, not just for generating individual clips.
World models have become a common tool in autonomous driving and embodied artificial intelligence, where teams use them to reduce the costs of collecting and labeling edge cases. HelloWorld is the first public research publication of this kind from this company's robotaxi team. However, the company has not yet announced when or how the model will be integrated into its vehicle testing programs.
