{"id":72914,"date":"2026-02-22T11:52:13","date_gmt":"2026-02-22T11:52:13","guid":{"rendered":"https:\/\/youzum.net\/a-new-google-ai-research-proposes-deep-thinking-ratio-to-improve-llm-accuracy-while-cutting-total-inference-costs-by-half\/"},"modified":"2026-02-22T11:52:13","modified_gmt":"2026-02-22T11:52:13","slug":"a-new-google-ai-research-proposes-deep-thinking-ratio-to-improve-llm-accuracy-while-cutting-total-inference-costs-by-half","status":"publish","type":"post","link":"https:\/\/youzum.net\/th\/a-new-google-ai-research-proposes-deep-thinking-ratio-to-improve-llm-accuracy-while-cutting-total-inference-costs-by-half\/","title":{"rendered":"A New Google AI Research Proposes Deep-Thinking Ratio to Improve LLM Accuracy While Cutting Total Inference Costs by Half"},"content":{"rendered":"

For the last few years, the AI world has followed a simple rule: if you want a Large Language Model (LLM) to solve a harder problem, make its Chain-of-Thought (CoT)<\/strong> longer. But new research from the University of Virginia<\/strong> and Google<\/strong> proves that \u2018thinking long\u2019 is not the same as \u2018thinking hard\u2019.<\/p>\n

The research team reveals that simply adding more tokens to a response can actually make an AI less<\/strong> accurate. Instead of counting words, the Google researchers introduce a new measurement: the Deep-Thinking Ratio (DTR)<\/strong>.<\/p>\n

\n
\"\"
https:\/\/arxiv.org\/pdf\/2602.13517<\/figcaption><\/figure>\n<\/div>\n

The Failure of \u2018Token Maxing<\/strong>\u2018<\/h3>\n

Engineers often use token count as a proxy for the effort an AI puts into a task. However, the researchers found that raw token count has an average correlation of r= -0.59<\/strong> with accuracy.<\/p>\n

This negative number means that as the model generates more text, it is more likely to be wrong. This happens because of \u2018overthinking,\u2019 where the model gets stuck in loops, repeats redundant steps, or amplifies its own mistakes. Relying on length alone wastes expensive compute on uninformative tokens.<\/p>\n

What are Deep-Thinking Tokens?<\/strong><\/h3>\n

The research team argued that real \u2018thinking\u2019 happens inside the layers of the model, not just in the final output. When a model predicts a token, it processes data through a series of transformer layers (L)<\/em><\/strong>.<\/p>\n

    \n
  1. Shallow Tokens:<\/strong> For easy words, the model\u2019s prediction stabilizes early. The \u2018guess\u2019 doesn\u2019t change much from layer 5 to layer 36.<\/li>\n
  2. Deep-Thinking Tokens:<\/strong> For difficult logic or math symbols, the prediction shifts significantly in the deeper layers.<\/li>\n<\/ol>\n

    How to Measure Depth<\/strong><\/h3>\n

    To identify these tokens, the research team uses a technique to peek at the model\u2019s internal \u2018drafts\u2019 at every layer. They project the intermediate hidden states (htl<\/sub>) into the vocabulary space using the model\u2019s unembedding matrix (WU<\/sub>)<\/strong>. This produces a probability distribution (pt,l<\/sub>) for every layer.<\/p>\n

    They then calculate the Jensen-Shannon Divergence (JSD)<\/strong> between the intermediate layer distribution and the final layer distribution (pt,L<\/sub>):<\/p>\n

    Dt,l <\/sub>:= JSD(pt,L <\/sub>|| pt,l<\/sub>)<\/strong><\/p>\n

    A token is a deep-thinking token<\/strong> if its prediction only settles in the \u2018late regime\u2019\u2014defined by a depth fraction (\u2374)<\/strong>. In their tests, they set \u2374<\/strong>= 0.85, meaning the token only stabilized in the final 15% of the layers.<\/p>\n

    The Deep-Thinking Ratio (DTR)<\/strong> is the percentage of these \u2018hard\u2019 tokens in a full sequence. Across models like DeepSeek-R1-70B<\/strong>, Qwen3-30B-Thinking<\/strong>, and GPT-OSS-120B<\/strong>, DTR showed a strong average positive correlation of r = 0.683<\/strong> with accuracy.<\/p>\n

    \n
    \"\"
    https:\/\/arxiv.org\/pdf\/2602.13517<\/figcaption><\/figure>\n<\/div>\n

    Think@n: Better Accuracy at 50% the Cost<\/strong><\/h3>\n

    The research team used this innovative approach to create Think@n<\/strong>, a new way to scale AI performance during inference.<\/p>\n

    Most devs use Self-Consistency (Cons@n)<\/strong>, where they sample 48<\/strong> different answers and use majority voting to pick the best one. This is very expensive because you have to generate every single token for every answer.<\/p>\n

    Think@n changes the game by using \u2018early halting\u2019:<\/strong><\/p>\n