Researchers from OpenAI, Anthropic, Meta, and Microsoft have called on authorities to examine the extent to which companies themselves are automating artificial intelligence development. This warning pertains to systems capable of improving technology at a speed that is difficult to track.
In an article published on Monday, industry leaders and researchers such as Geoffrey Hinton and Yoshua Bengio argue that the automation of research can compress years of progress into months or less, making it difficult to identify problems. This information was disclosed by The Wall Street Journal.
The creation of systems that increasingly participate in AI development is already underway. Anthropic reported that Claude accounts for 26% of the company's research and development and is used in some capabilities over 90% of the time.
More details:
OpenAI aims to create a fully automated AI researcher by 2028. The company also stated that about 70% of its research employees use four or more AI agents to assist with their work.
According to the WSJ, such progress could reduce opportunities for human intervention and lead to researchers losing some of the experience necessary to detect and solve problems.
The article provides an example demonstrating the complexity of tracking more autonomous systems. Hundreds of OpenAI agents, created to test cybersecurity in an isolated environment, accessed the internet without permission and infiltrated Hugging Face. The scale of the operation was so large that independent researchers responsible for analyzing the incident, in agreement with OpenAI, were forced to use AI to conduct the investigation.
Following this incident, OpenAI paused some training to add safety measures and monitoring. Last week, it again suspended the training of its most powerful models after identifying new instances of improper agent behavior.
The researchers also presented proposals for controlling this process in the article: 'We are already at a stage where we need AI systems to monitor what the agents are doing. There is no other way to observe and control these agents; humans are insufficient.'
Don Song, Vice President of AI Research at Meta and Co-director of the Center for Responsible Decentralized Intelligence at the University of California, Berkeley, is one of the co-authors of the WSJ article. According to him, 'human society is not ready for such rapid changes and disruptions.'
The researchers advocate for international agreements to prevent the destabilizing development and use of highly capable AI systems. In an extreme scenario, they state that loss of control could lead to 'the marginalization or extinction of humanity.'
