The upcoming cutting-edge model Astra, which is about to be released, has sparked security discussions due to a new internal architecture design. The focus of the controversy is not on the capabilities of the model itself, but rather on how such designs may make it more difficult for researchers to understand how AI arrives at its conclusions when performing tasks.

Adopt a more power-efficient cyclic design

As previously reported by The Information, OpenAI uses " recurrent depth " in some of the internal structures of Astra, which is also known as " looped Transformers ". The characteristic of this method is that it allows a portion of the computation modules to be called repeatedly, rather than having each token go through the entire network completely.

The direct benefit of doing so is to reduce the consumption of computing power. Reports cite relevant research stating that such architectures can achieve results close to those of traditional Transformer with fewer computing resources in certain scenarios, which is particularly important for corporate clients that are already bearing high AI costs.

Security researchers are concerned about the increasing difficulty of monitoring.

The controversy arises from the visibility of the model's reasoning process. Existing reasoning models typically generate a series of intermediate steps, commonly referred to as a “thought chain.” Researchers can use this to examine why the model made a particular decision or whether there was any overstepping of authority.

However, in the cyclic Transformer, some of the intermediate reasoning is not written in natural language but is repeatedly processed within the model in a form that is difficult for humans to understand directly. Researchers sometimes refer to this type of intermediate representation as “neuralese”. This means that what humans can see may be only the final answer, rather than the complete reasoning path.

Some security experts believe that this will weaken one of the few current means available to monitor the behavior of AI proxies. A policy director at the Washington-based think tank AI Policy Network told Fortune that if OpenAI is reducing its reliance on monitoring thought chains, this direction "may be concerning" and could also lead to higher risks.

OpenAI claims to still retain monitoring capabilities

In the face of external doubts, OpenAI's chief scientist, Jakub Pachoki, stated on X that the company has always placed emphasis on monitoring thought chains and is also working hard to retain the relevant capabilities. He mentioned that the scope of use for this type of architecture by Astra is limited, and that the model reasoning remains readable. More details about the architecture will be disclosed in the future.

Pachoki also stated that monitoring thought chains in the future may indeed become more difficult, but the reason may not necessarily lie in this architectural change itself. He said that OpenAI is still investing resources to advance real-time thought chain monitoring, which is also one of the current research priorities.

However, some researchers are concerned not only about Astra itself but also about the industry's demonstration effect. The former OpenAI researcher Daniel Kokotajlo believes that even though OpenAI is currently used in a more restrained manner, other companies may continue to follow this path, ultimately making the model inference process completely opaque to humans.

Cost advantages coexist with industry precedents

The reason why this type of architecture has attracted attention is also that it possesses attractiveness at both the cost and competition levels. In addition to saving computational power, hiding some of the intermediate reasoning processes may also make it more difficult for large models to be distilled, meaning it is harder for smaller models to replicate them by learning the output results.

Against the backdrop of the U.S. government and multiple AI companies continuously paying attention to the risk of cutting-edge models being copied, this design may be seen by more enterprises as a solution that balances efficiency and protection against copying. However, security researchers are concerned that without a unified standard in the industry, even more efficient models in the future may become more difficult to audit and regulate.