tolga boroglu.

Why Do AI Models “Give Up” on Hard Questions?

•2 min read•English•Tolga Boroğlu
#claude#ai#llm#apple#chatgpt

For a while I had questions about how AI models perform on hard problems. The more detailed and difficult my questions became, the more the quality of the answers seemed to drop. While looking for a technical basis for this observation, I came across Apple's much-discussed paper "The Illusion of Thinking".

The study puts recent popular "reasoning" models such as DeepSeek R1 and Claude 3.7 under the microscope, examining their limits and, surprisingly, their tendency to "give up".

As you know, the new trend in AI is thinking models (Large Reasoning Models, LRMs). These tools don't just answer; they stop to think, plan step by step and then respond. This creates the impression that they are far better at solving complex problems.

Apple's research suggests the opposite may be true. According to the paper, as questions get harder, the models give up, almost like a human reflex. Normally you would expect a model to think more (spend more tokens) as difficulty rises. But the study shows that beyond a certain difficulty threshold, the number of tokens spent decreases instead of increasing.

Memorisation or Intelligence?

So far we have mostly evaluated models with maths and coding benchmarks. Apple's researchers point to the problem of data contamination: are the models really solving the questions, or remembering them from their training data? To test this, the researchers used custom puzzle environments (such as the Tower of Hanoi) whose complexity can be adjusted and which the models cannot have memorised.

The results point to three distinct regimes that could change how we see AI:

  • Low complexity (simple questions): "Thinking" models aren't even needed here. Standard models are faster and more accurate.
  • Medium complexity: This is where LRMs shine. Where standard models get stuck, the "thinking process" kicks in and the models find the right answer.
  • High complexity (collapse): This is the critical point. Beyond a certain level of difficulty, both standard and thinking models hit a wall and accuracy drops to zero.

Giving Up and Algorithmic Shortcomings

The most striking finding: when a problem becomes very complex, the model cuts its thinking short and gives a wrong answer, as if saying "this is too hard", even though it still has enough token budget (time). This points to a serious limit on how far reasoning ability scales.

What's more, the researchers didn't just ask the models to "solve this"; they also gave them the algorithm (the recipe) for solving the problem. The expectation was that with the algorithm in hand, the model would execute flawlessly. Yet the models still failed to carry out the algorithm step by step.

This shows that the models still have serious gaps, and that they move forward with probabilistic guesses (pattern matching) rather than by following a logical process.

In short, AI is undoubtedly improving, but what we call "thinking" may, for now, be a very advanced form of imitation.

Paper: https://ml-site.cdn-apple.com/papers/the-illusion-of-thinking.pdf