Tencent releases Hy4 model with computer vision and control capabilities

Tencent Hunyuan has presented a technical report on Hy-Nomas, its open model for interactive world generation, and released Hy-Nomas-Bench—a test suite for evaluating interactive world video models. The report was posted on arXiv on October 27, following the initial release of the code and weights on October 21.

Hy-Nomas is a world video model based on an 8-billion parameter diffusion core. Users can control it over multiple iterations using free-form text prompts, continuous camera control, or a combination of both methods. The model generates the world segment by segment, ensuring scene consistency over time by conditioning each new segment on the current input and saved history.

According to the report, the key concept involves aligning structured video captions with causal attention that accounts for actions. Descriptive context remains available between segments, while action-related instructions are applied only to the corresponding time window. A vision-and-language-based prompt enhancer transforms user queries and interaction history into structured conditions for the next segment.

The training process follows a multi-stage curriculum. It begins with causal generation, then moves to text interaction and camera control, after which the model trains on histories it generated itself, concluding with a diffusion distillation step. The distilled model requires only four to eight denoising steps. In human evaluations, the team noted that Hy-Nomas surpassed existing world models.

Hy-Nomas-Bench includes 280 test scenarios covering four tasks: multi-turn free-form text interaction, camera navigation, alternating text and camera control, and spatial memory. This suite tests the model's ability to follow sequential instructions, generate physically plausible responses, accurately execute camera movements, and maintain scene consistency when the camera returns to a previously visited location. The kit includes prompts, structured captions, interaction sequences, and reference camera trajectories, and is released under the Apache 2.0 license.

The model itself is also distributed under the Apache 2.0 license. Tencent provides an INT8 configuration that runs on a single RTX 4090 using a sliding window history and CPU offloading, as well as a Diffusers pipeline, LoRA fine-tuning code, and an interactive web demo. Among the released adapters are a cinematic LoRA and a robotics LoRA trained on real data from a two-armed robot.

Developers outside the ecosystem have already integrated support into SGLang, ComfyUI, and WanGP, and one community demo project reported generating 720p in real-time on Google TPUs.

Tencent's Executive Vice President, Dongsong Tang, announced at the Tencent Global Digital Ecosystem Conference on October 29 that the company plans to soon release the Hy4 and Hy4 flash models, adding the function of visual understanding to its lineup of artificial intelligence models.

Compared to the preliminary version of Hy4 presented in August, the full Hy4 model will demonstrate improved performance in tasks requiring extended reasoning and execution, including real productivity workflows. Furthermore, Tencent aims to enhance reasoning efficiency.

The Hy4 flash model will be focused on cost-effectiveness while retaining visual understanding capabilities and AI agent functions. The preliminary Hy4 version has a total of 770 billion parameters, with 49 billion parameters activated during inference, and it supports a context window of one million tokens. Tang noted that Tencent is jointly developing its models and products to improve their efficiency in practical applications.

Tencent has introduced Hy-SimulMT, an open-source model designed for simultaneous translation from Chinese to English. This system is capable of continuously processing incoming speech and providing both translated text and speech output.

The model includes a real-time voice cloning feature, developed to preserve the vocal characteristics of the original speaker. Users can test this technology through Tencent's Hy Translate application.

The release includes the model weights, inference software, and training tools. When tested on the RealSI benchmark for long audio recordings, Tencent reported a streaming translation latency of approximately three seconds. Furthermore, the translation quality metrics surpassed those of other open-source models, as well as four out of five commercial APIs participating in the comparison.

A configured text-to-speech service is required for speech generation and voice cloning. The release also mentions [Tencent Hunyuan] in this context.