Committee

Perplexity Research published a new post-training study. It trains a model inside Perplexity Computer on real user sessions, including failed ones. The method pairs rejection sampling fine-tuning with hint-guided self-distillation. In a live A/B test, tool-call failures fell from 2.24% to 1.77% between 2 trained checkpoints. Perplexity team reports this as a statistically significant 21.2% relative reduction. Is it deployable? Not directly. Perplexity has not released the post-trained weights or training code. The model runs only as a model option inside Perplexity Computer. The base model, GLM 5.2, is openly available on Hugging Face. Why Outcome-Only Filtering Falls Short Standard rejection sampling fine-tuning (RFT) judges each session and imitates only the successful ones. A successful outcome does not mean every step was correct. An agent can recover from a bad tool call and still deliver the right answer. Imitating that full trajectory can reinforce the error. Discarding failed sessions also throws away clear evidence of avoidable mistakes. Imitate, Correct, or Keep as Context Perplexity team separates 2 decisions: which sessions hold behavior worth imitating, and which turns hold mistakes worth correcting. Each assistant turn gets 1 of 3 treatments: Imitate: non-error turns in successful sessions receive cross-entropy (CE) loss. Correct: error turns with a validated hint receive Kullback-Leibler (KL) divergence loss, in any session. Keep as context: remaining turns stay in the input but receive no loss. Successful sessions can supply both imitation and correction targets. Unsuccessful sessions supply only correction targets. How a Hint Becomes a Training Signal A hint is a short corrective instruction grounded in information the model already had. In one example, a search call set recency_filter to ‘year.’ The schema allowed only ‘day,’ ‘week,’ or ‘month.’ The hint names the failed call, includes the validation error, and suggests an allowed value or omitting the optional field. The corrective part uses On-Policy Self-Distillation (OPSD). The trainer runs the same GLM 5.2 checkpoint twice on the recorded turn. The teacher pass sees the hint; the student pass does not. Both use teacher forcing, so no replacement answer is generated. The teacher’s next-token probabilities are detached and act as a soft target through forward KL. The combined loss is (CE + λ × KL), divided by the number of imitated tokens. Setting λ to 0 recovers standard SFT. The CE term matters. Correction-only training can let teacher and student agree by ignoring context. Tracing Complaints to the Real Mistake The pipeline draws from training-eligible Computer sessions served by GLM 5.2. Sessions with personally identifiable information and users who opted out are excluded. An LLM judge keeps tasks rated 4 or 5 on a 5-point difficulty scale. Two LLM judges must both approve the final delivery for a session to count as successful. For user feedback, threes LLM judges locate the responsible turn, and at least 2 must agree. This is important because the last assistant turn before a complaint is the root cause only about half the time. Each hint is also checked against information available before the mistake. That check reduces hindsight bias. One example: a user asked for their ‘w3’ on Paychex. The model assumed a W-2 typo and searched for the wrong form. The hint targets that earlier interpretation, not just the final answer. Interactive Explainer What the Evaluations Show Hints work before training: On 985 held-out tool-error turns, the unchanged base model avoided the original failure in 93.7% of cases with hints, up from 75.1%. The share taking the corrected action rose from 60.6% to 82.3%. On user-feedback turns, fixed or on-track rates rose from 40.0% to 75.0% for explicit evidence. For inferred intent, they rose from 32.5% to 80.0%. Offline tool errors fell: Recorded tool-error rates were 2.79% for stock GLM 5.2 and 1.35% for RFT only. The RFT plus OPSD checkpoint reached 0.87%. Perplexity notes these checkpoints used different training data, so this is not a matched ablation. Task-level benchmark results on suites like BrowseComp and SpreadsheetBench were mixed. Live results are narrower: Each A/B test used about 100,000 users per condition. An early checkpoint versus stock GLM 5.2 showed 2.82% versus 2.94% failures, which was not significant. The later checkpoint comparison produced the significant 21.2% drop, without hints at inference. Strong dissatisfaction moved from 2.58% to 2.54%, which was also not significant. Perplexity did not compare the later checkpoint directly against stock GLM 5.2 online. Key Takeaways Perplexity learns from failed sessions, not just successful ones. Validated hints turn avoidable mistakes into KL correction targets. 1 model acts as teacher (with hint) and student (without). Live tool-call failures fell from 2.24% to 1.77%. User dissatisfaction showed no significant change. Check out the Technical Details. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Perplexity Trains Its Computer Agent on Real Mistakes With Hint-Guided Self-Distillation appeared first on MarkTechPost.

Aikido Security has released Altar-1, its first open-weight security model. It is a compressed version of Z.AI’s GLM-5.3, built to run inside infrastructure the customer controls. Altar-1 powers Aikido Machine, the company’s autonomous pentesting appliance for on-prem and air-gapped networks. Is it deployable? Yes, the weights are public on Hugging Face and run with vLLM on a single node of 4x NVIDIA H200 GPUs. The Problem: Security Context Cannot Leave the Network Closed frontier models run on someone else’s infrastructure. Using them sends source code, architecture docs, and unremediated findings outside the network. Aikido points to banks under data-residency mandates and OT operators with no internet route. Open-weight models solve the residency problem but create a deployment gap. Mixture-of-experts (MoE) models must store every expert, even when a workload uses only a few of them. Security agents also build long-running context. That KV cache competes with model weights for the same GPU memory. How Altar-1 Was Built GLM-5.3 is a 753B parameter MoE model. Each token routes to 8 of 256 experts per layer, which is about 40B active parameters. Aikido applied 2 compression steps: Step 1-Quantization: Altar-1 starts from the cyankiwi GLM-5.3-AWQ-INT4 checkpoint. AWQ stores routed expert weights in 4 bits, with 16-bit activations (W4A16). Attention, the shared expert, dense layers, and the head stay in BF16. Step 2- Expert pruning: Aikido used Cerebras REAP (Router-weighted Expert Activation Pruning). REAP scores each expert by router weight and output magnitude, not just by how often it is selected. Altar-1 keeps 168 of 256 routed experts per layer and removes 88 (34.4%). No retraining is involved. Calibration used traces from Aikido’s pentesting harness, plus coding, tool calling, reasoning, and multilingual Wikipedia text. Aikido states no customer data was used. Each expert is scored by its largest share of any single domain’s routed work. That protects the specialist experts for code, rare languages, and structured output. Routing is unchanged. The router still picks 8 experts per token, now from 168, with about 40B active parameters. Checkpoint Stored weights GLM-5.3, BF16 1,506.7 GB GLM-5.3, AWQ INT4 488.2 GB Altar-1, pruned W4A16 328.0 GB Altar-1 is 78.2% smaller than BF16 and 32.8% smaller than the AWQ parent. On fidelity, Altar-1 has a KL divergence of 0.506 nats against full BF16 on a sealed 25-prompt panel. An EXL3 build of the same cut scores 0.511. The details are in the public fidelity study. Benchmark Results Aikido team tested Altar-1 on its internal CVE benchmark. The benchmark covers 32 known vulnerabilities across 30 repositories, with 3 runs per case. Model Avg recall per run Found at least once GLM-5.3, BF16 65.6% 25 of 32 GLM-5.3, AWQ INT4 61.5% 23 of 32 Altar-1 60.4% 23 of 32 Compared with the AWQ checkpoint, pruning cost about 1 point of recall and no coverage. Compared with the parent, Altar-1 keeps 23 of 25 covered vulnerabilities (92%) at 5.2 points lower recall. The benchmark’s scope is narrow. It measures targeted CVE rediscovery inside a pipeline that uses other models for the surrounding stages. It does not measure blind discovery, exploit validation, or fix proposals. Aikido also reports that Altar-1 found a valid critical-severity vulnerability during a client’s production pentest. That is a single result reported by the vendor. Deployment and License The model card requires Hopper GPUs (H100 or H200). Aikido says 328 GB across 4x H200 leaves room for a 128k-context KV cache at production batch sizes. Copy CodeCopiedUse a different Browser vllm serve AikidoSec/altar-1 –tensor-parallel-size 4 –trust-remote-code –max-model-len 131072 vLLM selects the Marlin MoE kernel automatically. A 4x H100 80 GB node has only 320 GB of memory, which is less than the 328 GB of weights. Altar-1 inherits the GLM-5.3 License. The license permits commercial use, modification, and redistribution. Model-as-a-Service operators with more than $10B in revenue over 12 months must first pass a Z.AI security review. Altar-1 is open-weight, not OSI-approved open source. Altar-1 also powers Aikido Attack, AI Code Analysis, and Deep Review. Next, Aikido plans to try lower-bit formats like EXL3 so it can keep more experts. It also plans to fine-tune models for security workflows. Key Takeaways Altar-1 compresses GLM-5.3 from 1,506.7 GB to 328 GB, a 78.2% cut. REAP pruning keeps 168 of 256 experts per layer, with 8 active per token. Recall drops from 65.6% to 60.4%, keeping 92% of the parent’s CVE coverage. It runs on 1 node of 4x H200 with vLLM, including air-gapped setups. The GLM-5.3 License allows commercial use, with a review clause above $10B revenue. Check out the Model Weights and Technical Details. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Aikido Security Releases Altar-1: An Open-Weight Security Model Pruned From GLM-5.3 to 328 GB appeared first on MarkTechPost.

Liquid AI has announced LFM2.5-VL-3B-DSpark, an experimental speculative-decoding draft model for its LFM2.5-VL-3B vision-language model. The drafter adds about 280M parameters and speeds up decoding without changing the model’s output. Liquid AI team reports up to 3.13x faster decoding on Apple silicon and up to 2.66x on an NVIDIA H100. Is it deployable? Yes, Weights are live on Hugging Face in Safetensors and GGUF, with day-one support in SGLang, MLX-VLM, and llama.cpp. Liquid AI team labels the release experimental, and it ships under the LFM Open License v1.0, which allows free commercial use only for companies under $10M in annual revenue. What Speculative Decoding Changes for a VLM A standard model generates one token per forward pass. Speculative decoding adds a small drafter that proposes several tokens ahead. The large target model then checks the whole block in one pass and keeps the tokens it agrees with. DSpark follows the recipe from Liquid AI’s text-model DSpark drafters, described in the DSpark paper. The drafter reads the target model’s hidden states from several layers and predicts the next k tokens. The key design point: modality does not matter to the drafter. By the time tokens reach the hidden layers, text and image patches are both just tensors. So Liquid AI team reuses the exact same inference algorithm for its vision-language model. Drafter Architecture and Training The drafter is a simplified attention-only model. Ablations picked 4 layers and a block size of 9. Liquid AI recommends a block size of 8 or 9 at inference, depending on hardware. Apple silicon runs use 8. Component Parameters Decoder stack (4 layers) 193.0M Hidden-state projection 21.0M Markov head 65.5M Norms + confidence head 6.4k Total 279.5M The embedding and LM head are tied to the target, so the drafter does not carry them. Liquid AI says this raises the deployed parameter count by 8.9%. Training used supervised fine-tuning data covering common vision-language tasks for 10 epochs. All ablations and training ran exclusively on AMD hardware. Benchmark Results Evaluation follows the MMSpec benchmark across 6 task types: General VQA, Text VQA, Image Captioning, Chart VQA, Complex Reasoning, and Multi-turn Conversation. All runs used batch size 1, temperature 0, and 16-bit weights for the vision encoder and backbone. Data was collected on Pipette, Liquid AI’s public device-benchmarking infrastructure. Stack Decode speedup End-to-end speedup Accepted tokens per pass MLX-VLM, M5 Max MacBook Pro (block 8) 2.30x to 3.13x 1.56x to 2.62x 3.24 to 4.34 llama.cpp, M3 Ultra (block 8) 1.57x to 2.14x 1.30x to 1.77x 3.31 to 4.50 SGLang, 1x H100 80GB (block 9) 2.04x to 2.66x 1.64x to 2.27x 3.46 to 4.57 The ‘up to’ decode and end-to-end figures often come from different tasks. On the M5 Max, 3.13x decode is from COCO captioning, while 2.62x end-to-end is from MMMU-Pro. Acceptance landed in a similar range on both Apple stacks. Liquid AI reads this as acceptance depending on the drafter and workload, not the runtime. At higher concurrency, DSpark kept a throughput advantage at every measured level on a single H100 in SGLang. The gap narrows as concurrency rises. Output Quality and Temperature Under greedy decoding, the target verifies every proposed token, so output is identical to the base model. At non-zero temperatures with matched sampling, speculative decoding preserves the target’s output distribution, as proven by Leviathan et al. Temperature does affect speed. Higher temperatures spread probability across more candidate tokens, so drafter and target disagree more often. In Liquid AI’s tests, this lowered acceptance and throughput. Why End-to-End Gains Are Smaller on Edge Speculative decoding only accelerates decoding. Image encoding and prefill run at the same speed. A VLM must encode the image, then process hundreds of visual tokens alongside the prompt. On edge devices with less compute than data center GPUs, prefill takes a larger share of latency. Liquid AI frames this as Amdahl’s law: total speedup is bounded by the part left unaccelerated. This explains cases like TextVQA on the M5 Max, where 2.69x faster decoding yields 1.56x end to end. How to Run It SGLang requires v0.5.19 or newer. Launch LiquidAI/LFM2.5-VL-3B with –speculative-algorithm DSPARK and point –speculative-draft-model-path at the drafter. On Apple silicon, MLX-VLM v0.7.2 or newer accepts the drafter through –draft-model. DSpark in MLX-VLM currently supports greedy sampling only, so set temperature to 0. For llama.cpp, pair the GGUF drafter with the LFM2.5-VL-3B-GGUF target. Integration work is public in the llama.cpp, SGLang, and MLX-VLM pull requests. Acceleration of quantized models is outside the scope of this release. Key Takeaways A 279.5M drafter adds 8.9% parameters to LFM2.5-VL-3B. Decoding runs up to 3.13x faster on M5 Max, 2.66x on H100. Output is identical under greedy decoding; distribution preserved when sampling. Prefill and vision encoding cap end-to-end gains, especially on edge. Tested at 16-bit only; quantized acceleration is not covered yet. Check out the Technical Details. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Liquid AI Releases LFM2.5-VL-3B-DSpark: Speculative Decoding for Vision-Language Models With Up to 3.13x Faster Decoding appeared first on MarkTechPost.

Contrastive-LM has released CLM-8B, the first open model in a new class called Contrastive Language Models (CLMs). CLM does not generate text. It scores a set of candidate actions against the current state and returns probabilities. Their main baseline is Jev, the proprietary System One model from TypeSafe AI. Is it deployable? Yes. The Apache-2.0 head weighs 75 MB. It runs on 1 NVIDIA GPU under Linux, with vLLM serving the Qwen3-8B encoder. What a System One Model Does Jev entered limited early access on 15 September 2026. It returns typed values with probabilities instead of text. CLM targets the same interface. The CLM GitHub repo serves CLM-8B behind a TypeSafe-compatible API. It exposes 3 question types: Noul: returns the probability that a statement is true. Choice: picks one option from a declared set, with probabilities. Score: returns an expected level on an ordered rubric. A request written for TypeSafe’s API can be replayed through CLM’s Python client. How CLM Works CLM trains a state encoder and an action encoder with a bidirectional InfoNCE loss. Each encoder is a frozen Qwen3-8B backbone plus a 20M-parameter trainable projection head. Training pulls each state toward the action actually taken and pushes it away from the others. At inference, CLM scores each candidate by the dot product of the state and action embeddings. A softmax over those scores becomes the answer distribution. The same primitive ranks best-of-N solutions, routes tools and answers typed decisions. This design disaggregates states and actions. In an agent loop, the state changes every step while the action set stays mostly fixed. clm-serve reserves a slab of GPU memory, similar to vLLM’s KV cache, and reuses cached vectors. On 1 RTX 4090 with 3 actions, revisited states drop from 1.7 ms to 0.6 ms. The model card reports CLM running 13× faster than Jev with about 1,000 candidates. A 3-Stage Training Recipe Pre-training on ~60M Nemotron DQA question-answer pairs. Mid-training on ~30M synthetic hard negatives generated by Gemini 2.5 Flash-Lite. Post-training on ~1M agent trajectories from Agent Data Protocol, Endless-Terminals and LiteCoder-Terminal-SFT. On ~100K held-out questions, pre-training alone reaches 52.1% top-1 accuracy. Mid-training lifts it to 69.2%. Training on hard negatives from the start peaks at 62.4%, then overfits. Zero-Shot Results Against Jev Task CLM-8B latency Jev latency CLM-8B success Jev success T-Rex game 16.5 ms 149.8 ms 5/5 5/5 Tool calling (BFCL v4) 76.8 ms 125.5 ms 95.2% 99.2% WikiRacing 79.8 ms 225 ms 26/30 30/30 Super Mario 33.5 ms 132.6 ms 5/5 5/5 The 9× figure comes from the T-Rex game, where actions repeat across states. CLM matches Jev on T-Rex and Super Mario. It trails on tool calling and WikiRacing while running faster on every task. CLM as a Verifier for Coding Agents Here a generator samples several candidate solutions and the verifier picks one. Opus 5 produced DeepSWE candidates (best-of-4). Fable 5 produced Terminal-Bench 2.1 candidates (best-of-5). The team evaluated 38 held-out DeepSWE tasks and 30 held-out Terminal-Bench 2.1 tasks. Latency was measured on an H100. Benchmark Pass@1 CLM (fine-tuned) Jev CLM latency Jev latency DeepSWE 73.7% 81.6% 71.1% 79 ms 449 ms Terminal-Bench 2.1 84.0% 87.6% 83.1% 32 ms 131 ms The research team reports these as new SOTA verifier results. Jev scores below pass@1 on both benchmarks, so selecting with Jev is worse than taking 1 sample. CLM runs 4.1× to 5.7× faster. These numbers use lightweight fine-tuned heads, not the zero-shot checkpoint. They are held-out subset results, not full leaderboard submissions. Interactive Explainer Key Takeaways CLM-8B scores candidate actions instead of generating text. Up to 9× lower latency than Jev in zero-shot tests. Fine-tuned heads reach 81.6% on DeepSWE and 87.6% on Terminal-Bench 2.1 subsets. Cached state and action vectors cut agent-loop latency. Apache-2.0 head, self-hosted on 1 NVIDIA GPU. Check out the Blog, Code and Data & Models. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Contrastive-LM Releases CLM-8B: An Open System One Model That Scores Agent Actions Up to 9× Faster Than Jev appeared first on MarkTechPost.

This week, world leaders descended on Manhattan for the UN General Assembly. It’s also New York Climate Week—investors, policymakers, advocates, and journalists are colliding at panels, talks, and fancy dinners. With so many climate voices in one place, the discourse can feel a little louder than usual. This year, the unavoidable topic is artificial intelligence. There’s been a growing tension bubbling up about AI’s impacts on climate and climate tech. Depending on where you stand, you might point to AI’s potential for advancing research, or to the way funding and attention from Big Tech is trickling into energy startups. Or you might focus on the emissions-heavy natural-gas buildout that’s unfolding to meet the sector’s electricity demand. Everyone seems to agree that AI is important. The big question for those in the climate world, and the conversation I’m constantly having and hearing this week, involves how you see its influence unfolding. UN Secretary-General António Guterres highlighted the AI and climate crossover in a speech on the first day of the assembly. “The climate crisis fuels instability and displacement,” Guterres said. “Artificial intelligence could help solve all these challenges, or it could make them worse.” It feels relevant that this year is the first that the world has really had to grapple with the fact that climate goals are slipping out of reach. A recent report from the UN Environment Program said that the world has nearly passed the point where we could possibly keep warming to less than 1.5 °C above preindustrial levels. Since we’ve essentially missed this target, the report lays out the need not only to quickly and drastically reduce greenhouse-gas emissions, but also to employ carbon removal to help suck up emissions that have already been released into the atmosphere. The billion-dollar question is whether AI could help with any of this.  The energy-intensive technology is certainly shining a spotlight on the need to build out electricity supplies and shore up grid reliability. The result is more attention and money for energy technologies, some of which happen to be low- or zero-emissions. As I’ve covered before, startups across the energy and climate sectors are benefiting. Firms in nuclear, geothermal, wind, and solar power have signed deals with the likes of Google, Meta, and others looking to power their new or growing data centers. Global climate-tech investment from venture capital hit $26 billion in the first half of 2026, according to data from Currence, a finance tracker for the industry. That’s 55% higher than last year, and products and services for data centers are getting a massive slice of that pie. But as a Semafor piece about the report points out, some sectors are slipping through the cracks: Carbon management and low-carbon fuels saw VC investment plummet this year. These are important solutions for addressing climate change but may not be able to sell themselves to a data center. So far, the data center buildout has come with a hefty emissions toll. A few years ago, Microsoft, Google, and Meta all had ambitious goals to reduce greenhouse-gas emissions. Now they’ve all seen emissions rise, largely because of data centers that are needed to power AI. Some people are optimistic. AI could help speed up progress in areas like the search for new catalysts, as Evelyn Wang, MIT’s VP of energy and climate, pointed out during a panel. And data centers won’t add to climate and water problems forever, Wang told the Associated Press. (She puts the timeline at about a decade until data centers no longer add to planet-warming emissions.) But a whole lot of natural gas is coming online to meet the immediate demand created by new data centers. And once those power plants are built, they have a decades-long lifetime.  That’s partly why public pushback to AI is growing—people are seeing more pollution and noise near these data centers and the power plants that provide them with electricity. Overall, what I’m hearing this week is that many in the climate sector are skeptical of AI, at best.  “AI leaders are now on thin ice when it comes to license to operate and sinking deep underwater when it comes to public support,” said UN climate chief Simon Stiell in a speech this week. “Tech titans need to start showing why the benefits of AI outweigh its skyrocketing costs—for the many, not just the tiny few.”  This article is from The Spark, MIT Technology Review’s weekly climate newsletter. To receive it in your inbox every Wednesday, sign up here. 

BottleCap AI has released ThinkingCap-Qwen3.8-27B, the second model in its ThinkingCap series. It is a fine-tune of the Qwen team’s Qwen3.8-27B with one narrow goal: shorter reasoning traces. Across 12 benchmarks, it spends 37.2% fewer thinking tokens on average. Macro-average accuracy moves from 86.65% to 85.79%, a 0.86pp drop. Deployable? Yes. It drops in for Qwen3.8-27B on vLLM or SGLang, with FP8, NVFP4, GGUF and MLX builds. The repo is gated, and commercial use beyond the small-business license needs a BottleCap agreement. What Problem Does ThinkingCap Target? Reasoning models often spend more thinking tokens than a question needs. BottleCap’s position is that many of those extra tokens do not change the final answer. The first release in the series applied this idea to Qwen3.6-27B. The objective this time was deliberately conservative. BottleCap did not try to add knowledge or change answer style. Reasoning ability, instruction following and safety behaviour were meant to pass through untouched. The research team also focused harder on math, reasoning, long-context and agentic benchmarks. Benchmark Results at xhigh Effort All main numbers use reasoning_effort=xhigh, the chat template default. Every benchmark gets shorter, with cuts ranging from 10.7% to 65.5%. Knowledge and multilingual tasks shrink the most. MMMLU drops 65.5% (1,656 to 571 tokens) and MMLU-Pro drops 57.3%. GPQA-Diamond falls from 12,772 to 7,267 tokens, a 43.1% cut. IFBench thinks 46.4% less with accuracy nearly flat (79.75% to 79.71%). Long-context retrieval improves. AA-LCR accuracy rises 2.25pp, from 81.75% to 84.00%, with 38.6% fewer thinking tokens. LiveCodeBench v6 edges up 0.07pp while thinking 20.3% less. Agentic results hold close to the base. τ²-bench gives up 1.01pp for a 30.9% cut. Terminal-Bench 2.1 loses 0.56pp, well inside its ±4.26 interval, for a 10.7% cut. The most expensive trade is AIME 2026. Accuracy falls 3.85pp, from 98.13% to 94.27%, for 30.2% less thinking. Please note that the 37.2% figure is the mean of the 12 per-benchmark reductions. Pooled mean thinking tokens fall from 15,735 to 12,144. BottleCap also reports a budget curve. Under a 16K-token cap per response, ThinkingCap scores higher than the base model. Truncated traces fall from 0.51% to 0.34%, and looping from 0.06% to 0.05%. How It Interacts With the Effort Dial Qwen3.8-27B exposes a reasoning-effort setting, and the compression stacks with it. All deltas below compare against the base model at xhigh, averaged over 11 benchmarks. At medium, the base model cuts 52.1% of thinking for -9.16pp. ThinkingCap cuts 60.2% for -9.90pp. At low, the figures are -55.4% and -9.71pp for the base, versus -62.3% and -10.79pp for ThinkingCap. With thinking off, ThinkingCap trails the base by 5.7pp. BottleCap team recommends xhigh for the best accuracy-to-token balance. It says individual thinking modes will get attention in a future release. How the Evaluation was Run Both models ran through the same harness on one NVIDIA H200 with vLLM 0.29.0. Sampling was identical: temperature 1.0, top_p 0.95, top_k 20, min_p 0.0. Multi-seed accuracy is the mean with a 95% interval. Seeds range from 32 on AIME 2026 to 1 on MMLU-Pro and MMMLU. MMMLU uses a fixed 10,000-question sample; the other 11 benchmarks run complete sets. MTP speculative decoding (3 draft tokens) was measured as accuracy-neutral on AIME 2026. It accepted 53% of drafted tokens, about 2.6 tokens per step, matching the base model. Deployment: Builds, Serving and License The bf16 checkpoint has 28B parameters and accepts image and text input. BottleCap publishes 5 quantized builds: FP8: 31 GB, vLLM, Hopper and Blackwell. NVFP4 weight-only: 21 GB, vLLM, Hopper (Marlin kernel) and Blackwell. NVFP4 W4A4 (AWQ): 23 GB, Blackwell only. GGUF: 16 to 55 GB, for llama.cpp, LM Studio and Ollama. MLX 4-bit DWQ: 21 GB, for Apple Silicon Macs with 32 GB. Serving uses the base model’s recipe: –reasoning-parser qwen3 with the qwen3_xml tool-call parser on vLLM. Thinking returns in a separate reasoning field. The license is PolyForm Small Business 1.0.0 plus a BottleCap personal-use grant. Upstream Qwen materials stay under Apache-2.0. Hugging Face lists no inference provider hosting the model yet. Key Takeaways 37.2% fewer thinking tokens on average across 12 benchmarks. Macro accuracy drops 0.86pp, from 86.65% to 85.79%. AA-LCR long-context accuracy rises 2.25pp; AIME 2026 falls 3.85pp. Drop-in for Qwen3.8-27B: same sampling, same vLLM or SGLang flags. Gated weights under PolyForm Small Business; commercial use needs a license. Check out the technical blog and model weights. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post BottleCap AI Releases ThinkingCap-Qwen3.8-27B: 37.2% Fewer Thinking Tokens at a 0.86pp Accuracy Cost appeared first on MarkTechPost.