Researchers from Huawei Noah's Ark Lab, University College London, and other teams have introduced Memento 3—an agent system that successfully completed all 25 games and 183 levels in the public ARC-AGI-3 set, an interactive benchmark of the ARC competition, according to QbitAI.
ARC-AGI-3 is an environment where an agent is placed into unfamiliar pixel-style games without being provided with rules, goals, or descriptions of its actions. Memento 3 finished the public set using 7,518 actions, which is significantly less than the human benchmark of 17,135 actions, accounting for about 44% of the total human actions, and achieved a maximum score of 100 on the RHAE (Relative Human Action Efficiency) metric.
The previous best result in the public set was achieved by independent developer Sergey Rodionov using his base model, who also solved all games but scored 99 points while spending 8,347 actions.
However, this result has two important limitations. Firstly, the base model underlying Memento 3 is not a Huawei model; the system runs on Claude Opus 5 in the highest reasoning mode, without weight updates or fine-tuning for specific games. Secondly, this only pertains to public games, and an ideal result on the public set does not reflect how the system will perform in hidden test games, which test generalization ability.
Nevertheless, the same base model in the official ARC Prize apparatus shows an average RHAE of about 40.7, so the increase in efficiency is achieved through the agent's architecture itself.
The agent's architecture is called 'Code as Model'. During gameplay, the agent maintains a set of rules in natural language and then converts it into Python code that simulates the game, after which it plans moves within its own simulator. If reality contradicts the simulation, the agent rewrites both elements. In one game, it initially assumed that a special floor reflected the carried pattern, but then discovered that the pattern was actually rotated 90 degrees clockwise. The system replays previous moves and compares each 64x64 frame to ensure the code reproduces what happened. Every change is recorded in Git, allowing failed edits to be rolled back.
In experiments involving element exclusion across three games, maintaining the rule set reduced the number of actions by approximately 9%. When using two competing world models in one game, the number of actions decreased from 899 to 597, although this test was conducted only on that specific game.
The team also tested the system on Atari Pong. After studying the physics of the bounce based on approximately 9,500 frames, the code-based policy played without invoking the language model and won three matches with a score of 21:0. The authors note that the research requires a large number of tokens, and more complex real-world scenarios remain untested.