“Study Reveals Advanced AI Faces Total Accuracy Breakdown with Complex Challenges”

Researchers at Apple have identified “fundamental limitations” within advanced artificial intelligence models, casting doubt on the tech industry’s pursuit of increasingly powerful systems. In a paper released over the weekend, Apple indicated that large reasoning models (LRMs)—a sophisticated form of AI—experienced a “complete accuracy collapse” when faced with highly complex problems. It was noted that conventional AI models outperformed LRMs in simpler tasks, but both types of models failed entirely in high-complexity scenarios.

Large reasoning models aim to address intricate queries by generating detailed thought processes that decompose issues into manageable steps. The study, which evaluated the models’ abilities to solve puzzles, revealed that as LRMs approached performance collapse, they began to “reduce their reasoning effort,” a finding the Apple researchers deemed “particularly concerning.”

Gary Marcus, a prominent academic voice on AI capabilities, characterized the Apple paper as “pretty devastating.” Writing in his Substack newsletter, he pointed out that the findings raised significant questions about the race toward artificial general intelligence (AGI)—a hypothetical stage where an AI system can perform any intellectual task at a human level. Marcus remarked that those who believe that large language models (LLMs), like those functioning behind tools such as ChatGPT, are a straightforward path to beneficial AGI are misled.

The research also illustrated that reasoning models depleted computing resources by identifying solutions for simpler tasks early in their processing. However, as problems grew slightly more complex, the models first pursued incorrect solutions before eventually finding the correct ones. For problems of even higher complexity, the models encountered “collapse,” failing to produce any valid solutions; in one instance, they were unable to solve a problem even when provided with the necessary algorithm.

The study concluded: “As models near a critical threshold—which aligns closely with their accuracy collapse point—they counterintuitively begin to minimize their reasoning effort despite the increasing difficulty of problems.” This observation suggested a “fundamental scaling limitation in the thinking capabilities of current reasoning models.”

The LRMs tested included challenges like the Tower of Hanoi and River Crossing puzzles. The researchers acknowledged that their focus on puzzles might limit the scope of their findings. The paper argued that current methodologies in AI may have reached significant barriers, as it examined models from OpenAI, Google, Anthropic, and DeepSeek. Comments from Anthropic, Google, and DeepSeek are pending, while OpenAI has opted not to comment.

Discussing “generalizable reasoning”—or the ability of an AI model to apply specific conclusions more broadly—the paper stated: “These insights challenge existing assumptions about LRM capabilities and suggest that contemporary approaches may be hitting fundamental roadblocks in generalizable reasoning.” Andrew Rogoyski from the Institute for People-Centered AI at the University of Surrey remarked that the Apple paper highlights the industry’s ongoing exploration regarding AGI, suggesting that current methods may have reached a “cul-de-sac.” He emphasized that the finding that large reasoning models struggle with complex problems while excelling at simpler ones indicates a potential dead end in existing approaches.