Amazon is reassessing how it measures the use of artificial intelligence by its software engineers. According to a new internal guideline, the company considers metrics such as the amount of tokens used, tool adoption, and volume of code modifications to be 'deeply flawed' indicators when set as objectives.
Despite this change, the company does not intend to abandon AI. The focus of the alteration lies in the evaluation: instead of rewarding those who use more tools or generate a higher volume of code, Amazon seeks to measure the effect of the work on customers, businesses, and teams.
One of the identified challenges is the phenomenon known as 'tokenmaxxing,' a term used to describe the excessive use of tokens with the aim of inflating internal indicators. Tokens represent the units that models use to process and generate data, and they also affect the operational costs of these tools.
As reported by Fortune, Amazon warns that turning this consumption into a goal can result in increased expenses without guaranteeing a real improvement in results. This problem is not merely theoretical; company employees have even used AI agents in superfluous activities just to boost their numbers in an internal ranking that was discontinued in May.
For Terk Terkowitz, Amazon Vice President, the most appropriate criterion lies in the long-term results achieved, covering the impact on customers, the business, and the teams themselves. Although token consumption and tool adoption indicators are not eliminated, the company clarifies that they can help identify patterns but should not exclusively serve as performance goals.
The guidance also applies to the opposite: Terkowitz considers valuable the discovery of areas where AI is not being effective. The company maintains the incentive for engineers to learn and integrate AI into their daily routines, believing that the technology will remain central to software development, alongside knowledge, technical experience, and the ability to judge whether a specific solution is truly relevant.
This discussion transcends the limits of the company itself. A recent study showed that only 3% of organizations managed to exceed their savings targets through productivity gains provided by AI, even with most planning to expand their investments in this technology.