{"id":98731,"date":"2026-06-20T18:11:02","date_gmt":"2026-06-20T18:11:02","guid":{"rendered":"https:\/\/youzum.net\/vibethinker-3b-a-3b-dense-reasoning-model-built-on-qwen2-5-coder-3b-with-the-spectrum-to-signal-post-training-pipeline\/"},"modified":"2026-06-20T18:11:02","modified_gmt":"2026-06-20T18:11:02","slug":"vibethinker-3b-a-3b-dense-reasoning-model-built-on-qwen2-5-coder-3b-with-the-spectrum-to-signal-post-training-pipeline","status":"publish","type":"post","link":"https:\/\/youzum.net\/es\/vibethinker-3b-a-3b-dense-reasoning-model-built-on-qwen2-5-coder-3b-with-the-spectrum-to-signal-post-training-pipeline\/","title":{"rendered":"VibeThinker-3B: A 3B Dense Reasoning Model Built on Qwen2.5-Coder-3B With the Spectrum-to-Signal Post-Training Pipeline"},"content":{"rendered":"

While recent breakthroughs in AI reasoning have largely been driven by massive scale, pouring in billions of parameters to cross complex cognitive thresholds\u2014VibeThinker-3B<\/strong> is charting a completely different path.<\/p>\n

Created by researchers from Sina Weibo Inc (China), this 3-billion-parameter model proves that efficiency can punch far above its weight class. Released under an open-source MIT license, VibeThinker-3B matches the performance of models hundreds of times its size on verifiable tasks like mathematics, coding, and STEM disciplines.<\/p>\n

What is VibeThinker-3B<\/strong><\/h2>\n

VibeThinker-3B is a compact dense model built on the Qwen2.5-Coder-3B base. It is post-trained, not pretrained from scratch. The research team applies supervised fine-tuning, reinforcement learning, and self-distillation on top.<\/p>\n

The training framework continues the Spectrum-to-Signal Principle (SSP) from the earlier VibeThinker-1.5B. SFT (Supervised Fine-Tuning) builds a broad space of valid reasoning paths, the \u2018Spectrum.\u2019 RL then amplifies the correct paths, the \u2018Signal.\u2019<\/p>\n

The model targets one job: reasoning where a verifier can confirm the answer. The research team recommends larger general models for open-domain knowledge tasks. VibeThinker-3B is a specialist by design.<\/p>\n

It runs on standard stacks. The model weights require transformers>=4.54.0<\/code>. For faster inference it recommends vLLM==0.10.1<\/code> or SGLang>=0.4.9.post6<\/code>. The BF16 weights are roughly 6 GB, small enough for a single GPU.<\/p>\n

\n
\"\"
https:\/\/arxiv.org\/pdf\/2606.16140v1<\/figcaption><\/figure>\n<\/div>\n

Benchmark<\/strong><\/h2>\n

On AIME26, VibeThinker-3B scores 94.3. According to the research paper, this is comparable to DeepSeek V3.2 (671B) and Kimi K2.5 (1T).<\/p>\n

On LiveCodeBench v6, it reaches 80.2 Pass@1. On OJBench, another code benchmark, it scores 38.6, below the largest models. On HMMT25 it scores 89.3, and on BruMO25 it reaches 93.8. On IMO-AnswerBench, a 400-problem IMO-level set, it scores 76.4.<\/p>\n

The table below compares it against much larger reasoning models. The \u2018+CLR\u2019 row uses test-time scaling. It stands for Claim-Level Reliability Assessment<\/p>\n

\n\n\n\n\n\n\n\n\n\n\n
Model<\/th>\nParams<\/th>\nAIME26<\/th>\nHMMT25<\/th>\nIMO-Ans<\/th>\nLCBv6<\/th>\nGPQA-D<\/th>\n<\/tr>\n<\/thead>\n
VibeThinker-3B<\/strong><\/td>\n3B<\/strong><\/td>\n94.3<\/strong><\/td>\n89.3<\/strong><\/td>\n76.4<\/strong><\/td>\n80.2<\/strong><\/td>\n70.2<\/strong><\/td>\n<\/tr>\n
VibeThinker-3B +CLR<\/strong><\/td>\n3B<\/strong><\/td>\n97.1<\/strong><\/td>\n95.4<\/strong><\/td>\n80.6<\/strong><\/td>\n\u2014<\/td>\n72.9<\/strong><\/td>\n<\/tr>\n
GPT-OSS (high)<\/td>\n120B<\/td>\n93.2<\/td>\n90.0<\/td>\n75.6<\/td>\n81.9<\/td>\n80.1<\/td>\n<\/tr>\n
DeepSeek V3.2<\/td>\n671B<\/td>\n94.2<\/td>\n90.2<\/td>\n78.3<\/td>\n80.8<\/td>\n82.4<\/td>\n<\/tr>\n
GLM-5<\/td>\n744B<\/td>\n95.8<\/td>\n97.9<\/td>\n82.5<\/td>\n85.5<\/td>\n86.0<\/td>\n<\/tr>\n
Kimi K2.5<\/td>\n1T<\/td>\n93.3<\/td>\n95.4<\/td>\n81.8<\/td>\n85.0<\/td>\n87.6<\/td>\n<\/tr>\n<\/tbody>\n<\/table>
Source: VibeThinker-3B Technical Report, Table 2. GPQA-D is GPQA-Diamond.<\/em><\/figcaption><\/figure>\n

The pattern is consistent. On verifiable math and code, the 3B model sits near the top cluster. On GPQA-Diamond, a knowledge-heavy benchmark, the gap to large models stays visible.<\/p>\n

The research team also ran an out-of-distribution coding test. It used recent LeetCode weekly and biweekly contests, from Apr 25 to May 31, 2026. The model passed 123 of 128 first-attempt Python submissions. That is a 96.1% acceptance rate on unseen problems.<\/p>\n

Inside the Spectrum-to-Signal Pipeline<\/strong><\/h2>\n

The post-training pipeline runs in four stages. Each one targets a different weakness of small reasoning models.<\/p>\n

First comes<\/strong> curriculum-based two-stage SFT. Stage 1 covers math, code, STEM, dialogue, and instruction following broadly. Stage 2 shifts to harder, longer-horizon samples filtered by reasoning length and difficulty. Diversity-Exploring Distillation preserves multiple valid solution paths through both stages.<\/p>\n

Second comes<\/strong> multi-domain Reasoning RL. The research team reuses MaxEnt-Guided Policy Optimization (MGPO). MGPO weights prompts near the model\u2019s current capability boundary, where correct and incorrect rollouts coexist. Training runs sequentially across Math, Code, and STEM.<\/p>\n

A notable detail: VibeThinker-3B drops progressive context expansion. The research team found high-truncation warm-up hurt long reasoning at this scale. So RL uses a single 64K long-context window throughout.<\/p>\n

Math RL adds a Long2Short stage. It redistributes reward among correct trajectories by length. Shorter correct answers get higher reward, longer ones lower, with the group mean unchanged. The goal is fewer redundant tokens without losing accuracy.<\/p>\n

Third,<\/strong> Offline Self-Distillation merges the RL checkpoints back into one student model. Fourth<\/strong>, Instruct RL improves instruction adherence. That stage explains the 93.4 IFEval and 74.5 IFBench scores. Both show reasoning tuning did not break controllability.<\/p>\n

CLR: Scaling at Test Time, Not Parameter Count<\/strong><\/h2>\n

Claim-Level Reliability Assessment (CLR) is the report\u2019s test-time scaling method. It runs on answer-verifiable tasks and adds no parameters.<\/p>\n

The procedure has two steps. The model first generates K = 32 trajectories per problem. From each, it extracts M = 5 decision-relevant claims plus a final answer.<\/p>\n

The model then acts as its own verifier. It validates or falsifies each claim, producing binary verdicts. CLR maps these into a nonlinear trajectory reliability score, where one weak claim sharply lowers the weight.<\/p>\n

Answers are clustered by equivalence, and the highest reliability-weighted answer wins. The full flow runs 8 times, and the averaged Pass@1 is reported. CLR lifts AIME26 to 97.1 and BruMO25 to 99.2.<\/p>\n

The interactive demo below lets you flip claims and watch the score collapse. It also lets you switch benchmarks and compare against larger models.<\/p>\n

<\/p>\n

\n

Use Cases With Examples<\/strong><\/h2>\n<\/p>

The research team frames VibeThinker-3B as a specialist, so use cases follow the verifiable-reasoning boundary.<\/p>\n