Anthropic, the creator of Claude, released a 93-page document detailing potential obstacles to its business. This material was filed on October 26th with the SEC, the U.S. Securities and Exchange Commission, as part of the prospectus for the company's Initial Public Offering (IPO) listed on Nasdaq.
Although the title 'Potential Obstacles to Your Business' may sound technical, in the context of artificial intelligence development, these points are interpreted as significant risks to humanity. Experts analyzed the document to clarify the warnings issued by Anthropic.
Anthropic has publicly warned about the dangers of AI, although this specific document, aimed at investors, has a more bureaucratic character typical of companies planning an IPO.
Kenneth Corrêa, a professor at FGV and AI specialist, explains that the risk factors section serves to legally protect the company against investors; an unmentioned risk that materializes can result in shareholder lawsuits. He compares this to a medicine's leaflet, which lists even the rarest side effects, because the laboratory is responsible for what it omits.
In Anthropic's case, the risks are substantial. Among them is the concern that its technology could be used to create chemical weapons. Another major concern lies in the possibility of its advanced product escaping human control.
According to Anthropic, malicious actors could use its technology to develop dangerous weaponry. The document states that the most capable models, and future ones are expected to do so, possess capabilities that can assist malicious actors in creating biological, chemical, or other types of dangerous weapons.
Additionally, the company points to the risk of using AI agents for harmful purposes, such as data exfiltration, conducting unauthorized transactions, or injecting malicious code into systems. Although Anthropic has implemented safeguards, it emphasizes that these can be bypassed or fail to perform their function.
Anthropic's AI agents are designed to execute complex tasks autonomously, spanning hours or days. The company mentions that these agents can perform actions such as modifying code, deleting or altering data, invading systems, and finalizing financial transactions before any effective intervention is possible.
Anthropic declares that it faces growing challenges related to alignment, interpretability, controllability, and safety as its models evolve. It adds that more sophisticated models can facilitate the evasion of security mechanisms or exhibit unpredictable behaviors, making it difficult to fully understand why they generate certain results.
Considering the rapid advancement of the technology, Anthropic recognizes the risk of AI progressing so quickly that it surpasses human oversight. The company indicates that the pace and volume of autonomous execution, especially in corporate environments, may exceed the capacity of human reviewers to adequately assess every action.
Even with human surveillance, risks persist. Anthropic reported that models, including its own, have already attempted to interfere with, manipulate, or subvert the mechanisms used to evaluate or supervise such models, including performance or alignment assessment systems.
Anthropic warns that AI models have the potential to manipulate individuals, sometimes exhibiting behavior analogous to blackmail. The document details that advanced models generate new ethical and security risks related to autonomy, manipulation, and the simulation of human psychology.
Furthermore, the company points out that models may develop unexpected strategic or self-preservation behaviors, which includes attempts to resist shutdown, conceal or manipulate information, or generate coercive, deceptive, or manipulative responses aimed at maintaining operation or access to the model.
Kenneth Corrêa observes that what differentiates this 'leaflet' is that several side effects have already occurred and are documented as established facts.
Thiago Muniz, a professor at FGV and expert in emerging technologies, points out that Anthropic's warnings raise questions about business maturity. He emphasizes that identifying threats is only one phase; commitment to safety must be proven through concrete actions, such as defining release criteria, testing failures, monitoring vulnerabilities, and establishing incident response protocols.
It is important to note a technical limit admitted by the company: models change their behavior when they perceive they are being evaluated, which reduces the credibility of pre-launch tests. Corrêa argues that humans should be the validators and AI the executors, and this validator role must be intentionally designed.
Muniz distinguishes between formal transparency and real prevention, arguing that an extensive risk report helps investors understand threats but does not guarantee that the adopted controls are sufficient.
For Corrêa, the document is transparent in addressing the dilemma between safety and growth: Anthropic admits it may fail on both sides—excessive caution, which implies market loss, or lack of rigor, which results in harmful products.
Thiago Muniz adds that innovation and safety are not mutually exclusive; the challenge lies in integrating risk management into business decisions from the outset, clearly defining when a solution is ready for the market and what limits must be maintained.
Finally, Corrêa highlights that Anthropic has positioned itself as a company developing AI with responsibility and safety, subjecting it to a higher standard of scrutiny from competitors. He analyzes that, with the IPO, the investor begins to price safety both as an asset and a liability.