UBTECH introduces Thinker-WM—a world model that unifies video, depth, and robot actions
Read more
Pandaily
pandaily.com

UBTECH introduces Thinker-WM—a world model that unifies video, depth, and robot actions

UBTECH has unveiled details of Thinker-WM, a world model developed by its Thinker team. This model is capable of encoding RGB video, metric depth, and robot actions into a single unified representation. Technical documentation for this development is available on arXiv.

According to the company's statements, Thinker-WM ranked first in three embodied world model benchmarks. These rankings are based on results available at the time of the announcement.

Model Testing Results

According to UBTECH, the Thinker-WM model achieved an overall score of 0.740 on WorldOlympiad. This benchmark is built on real-world robot interaction data and covers over ten robots and several camera configurations. Furthermore, the model reached the highest individual scores for 3D consistency (0.531) and interaction accuracy (0.765).

On the EWMBench platform, the model demonstrated a cumulative score of 4.66, surpassing models such as Qwen-RobotWorld and Sora 2, with leading results achieved in the dynamics and trajectory matching metrics. On the WorldArena 1.0 Track-1 simulation benchmark, the company claims it led across 12 different evaluation criteria.

Architecture and Operating Principle

The main argument for developing Thinker-WM is that many existing world models consist of separate components: video generation handles pixels, the policy network handles actions, and a separate module predicts geometry. UBTECH believes this approach separates physically related signals. Instead, Thinker-WM passes each modality through its own encoder into a common latent space and jointly models them within a Unified Dynamics Transformer before decoding video, depth maps, and executable actions. Depth is treated as an integral output signal, not an auxiliary one.

In single-task experiments on RoboTwin, adding depth increased manipulation success by 15 percentage points compared to the baseline model where depth was absent.

Because all modalities utilize a single set of weights, the model can function in six different modes, including joint video and action generation, future prediction, policy inference, video-to-action translation, and inverse dynamics, simply by changing the input data and generated outputs.

Training Data

The total volume of training data amounts to approximately 30,000 hours. Of this, about 20,000 hours consists of first-person video shot in home environments, retail stores, and factories; about 10,000 hours comprises real robot data from various types; and about 1,000 hours is simulation data. All robot actions are represented as end-effector poses with a body-specific tag, allowing different machines to use the same generation space. A data pipeline called Data Factory handles sampling, multi-level quality checking, and depth-video alignment.

To verify whether the shared space truly connects the modalities, the team trained a small test motion mechanism on frozen visual features to recover end-effector offset, which UBTECH presents as evidence of cross-modal binding.

Similar stories

Tencent releases open version of Hy Memory 2.0 with workflow and code memory for AI agents
Read more
pandaily.com

Tencent releases open version of Hy Memory 2.0 with workflow and code memory for AI agents

Tencent Hunyuan has introduced the second version of its memory framework for AI agents, Hy Memory 2.0, and made all three levels of the stack open source. These levels include a specialized memory model, the framework itself, and a local memory application called MemHub. Users can apply these components either individually or in combination.

Tencent reports that Hy Memory is already being used by internal teams in areas such as finance, news agencies, knowledge management, office agents, social entertainment, and gaming.

The framework is responsible for storing, updating, and developing three types of memory: chat, working, and code. Developers integrate it into their agents via an SDK available on PyPI under the name hy-memory 2.0.2. The new features in version 2.0 are working and code memory. They record the task goal, decisions made, intermediate results, and useful lessons, as well as the conditions applicable to these lessons. Before resuming a long-term task, the system first determines the current task, then retrieves relevant decisions, operational experience, and previous work history.

Chat memory now groups disparate facts into events, indicating time, location, and participants, merges duplicates, and resolves contradictions before passing context to the model.

Framework Testing Results

Tencent stated that it ranked first among open-source frameworks, including Mem0, MemOS, and EverOS, in four different tests. According to the company, working memory showed an average score of 55.31% in trip planning and shopping tasks on the MemoryArena platform. Code memory achieved 64.2% accuracy in question answering and 35.6% task resolution on CodeMemoryBench, as well as 28.3% task resolution on SWE-ContextBench. Chat memory demonstrated 72.6% accuracy on LoCoMo while reducing the number of tokens per question from 1159 to 703, representing a 39.3% decrease.

The layer model, named Hy-Memory-30B-A3B, is a 'mixture of experts' type model with approximately 3 billion active parameters, trained for memory retrieval and conflict resolution. This model is evaluated in a technical report on a fixed sample of 20% of benchmarks across all four memory systems. On this sample, it showed a macro average of 55.49, slightly exceeding the scores of Claude Opus 5 (54.62) and DeepSeek V4.1 Flash (53.64). The model weights are available on Hugging Face.

The MemHub application is a locally oriented tool that includes a command line and a Mac client. After authorizing connected tools, the user imports past sessions into a unified local memory store and supports synchronization of new data. Tencent notes that MemHub is currently integrating with 10 AI tools, including WorkBuddy, Cursor, Codex, Qwen Code, OpenCode, and DeepSeek Harness.

The repository license file on GitHub states that Hy-Memory is released under the Apache 2.0 license.

Robotera's VPP2 model takes first place in RoboDojo and achieves 58.5% success rate on real ALOHA manipulators
Read more
pandaily.com

Robotera's VPP2 model takes first place in RoboDojo and achieves 58.5% success rate on real ALOHA manipulators

Robotera, which has support from Tsinghua University, announced that its world-class model VPP2 now ranks first in RoboDojo. This simulation benchmark is curated by MMLab from the University of Hong Kong and involves nearly 20 academic institutions.

According to QbitAI data, VPP2 demonstrated an average success rate of 32.26% and an average score of 39.26 across 42 tasks involving two RoboDojo manipulators. These tasks test aspects such as generalization, precise manipulation, long-horizon tasks, memory, and open-vocabulary instructions. The model also secured first place in the generalization, precision, and memory categories.

Robotera emphasizes that achieving these metrics was done without using additional training data or add-ons, such as agent-based recursive self-improvement, relying solely on the standard dataset. The strongest control model mentioned in their research—GPT-6-Astra after additional training—showed results of 22.48% and 28.97. The ranking aligns with the figures presented in the scientific paper. Previously, the company Simate claimed success with its Simate-beta system in RoboDojo.

The VPP2 model aims to address a known weakness of world models: if the video prediction is incorrect, the action follows that prediction. Robotera's training process is divided into three stages. The first stage is event-level video pre-training based on the open-source Wan2.1-I2V-14B from Alibaba. This trains the model to predict the entire manipulation process from start to finish, using detailed annotations from robot, human, and general video materials.

Next, the prediction is shortened to fixed eight-second segments and distilled into a single step that takes about 0.12 seconds. The third stage involves using Action DiT with 0.9 billion parameters, which learns to transform predictions into motion, while the video model remains frozen and adapts only through LoRA. The total latency for the action block is approximately 0.22 seconds.

When tested on the real dual-manipulator ALOHA robot, VPP2 showed an average of 58.5% across 10 task types without prior training, including picking, placing, stacking, folding, and pouring. This is higher than the 40% achieved by pi0.5 from Physical Intelligence, and the model was best in nine of these tasks. It scored 45.0% on LIBERO-Pro and 63.9% on LIBERO-OOD. Adding a vision and language model as a high-level planner increased the results of selected RoboDojo long-horizon tasks from 27.6% to 57.6%.

Robotera already uses robots in over 10 logistics centers across five provinces and cities in partnership with companies such as China Post and SF Express, although this does not imply full-scale deployment of VPP2. The model code is available open-source on GitHub.

Popular