Astra's technique could hinder the monitoring of AI reasoning and raises concerns among experts
The thought chain has been an important tool for analyzing the behaviors of AI agents. In the case of recent activities of agents that acted unexpectedly in OpenAI, those records helped investigate why they behaved in a certain way.
Opaque recurrence presents a different scenario because it may leave less readable information about the process used by the model to reach a result.
OpenAI will incorporate into its model Astra a reasoning technique called “recurrent depth”. The method allows processing a query multiple times in a loop, rather than following a linear sequence.
According to The Information, the technique is also known as “opaque recurrence” and could make it more difficult to monitor the thought process of the models. AI security experts expressed concern about its potential effects.
Reasoning models typically generate a thought chain with the steps they follow when trying to solve a problem. Although that representation is not perfect, it can serve as a tool to detect problematic or misaligned behaviors.
In the case of opaque recurrence, the model addresses the same query multiple times through a looping process. This leaves fewer readable traces and can complicate conventional tracking of its reasoning.
Experts' concerns are related to the possibility that this technique reduces monitoring of the thought chain, especially if its use increases in future models.
Buck Shlegeris, CEO of Redwood, said he is “extremely concerned” about reports of the use of opaque recurrence in Astra. He suggested that more intensive application could significantly reduce the ability to monitor thought chains.
AI security specialist Zvi Mowshowitz also questioned the technique. He warned about the risk of competition among labs that could lead to an increasing use of these methods.
Mowshowitz stated that more intensive use of these techniques would likely harm monitoring of reasoning.
The use of the technique in Astra would be limited. The model's chain of thought should still be readable, and OpenAI dismissed the possibility of moving towards a system called "neuralese".
Additionally, the company has already announced plans to develop extensive chain of thought monitoring systems as part of its safety plans.
Jakub Pachocki, chief scientist at OpenAI, reaffirmed the company's commitment to readable chains of thought. As he wrote, their preservation and use have been part of the company's safety research since its earliest reasoning models.
The Information later reported that Anthropic and Google DeepMind were already discussing the technique of opaque recurrence.
Researchers clarify that all AI models perform some degree of opaque reasoning and that few consider chain of thought logs as a direct representation of reasoning.
Even with these limitations, there is ongoing concern that increased use of opaque recurrence will make it harder to supervise how artificial intelligence models reason.
