Committee

As AI systems enter production, reliability and governance can’t depend on wishful thinking. Here’s how observability turns large language models (LLMs) into auditable, trustworthy enterprise systems. Why observability secures the future of enterprise AI The enterprise race to deploy LLM systems mirrors the early days of cloud adoption. Executives love the promise; compliance demands accountability; engineers just want a paved road. Yet, beneath the excitement, most leaders admit they can’t trace how AI decisions are made, whether they helped the business, or if they broke any rule. Take one Fortune 100 bank that deployed an LLM to classify loan applications. Benchmark accuracy looked stellar. Yet, 6 months later, auditors found that 18% of critical cases were misrouted, without a single alert or trace. The root cause wasn’t bias or bad data. It was invisible. No observability, no accountability. If you can’t observe it, you can’t trust it. And unobserved AI will fail in silence. Visibility isn’t a luxury; it’s the foundation of trust. Without it, AI becomes ungovernable. Start with outcomes, not models Most corporate AI projects begin with tech leaders choosing a model and, later, defining success metrics. That’s backward. Flip the order: Define the outcome first. What’s the measurable business goal? Deflect 15 % of billing calls Reduce document review time by 60 % Cut case-handling time by two minutes Design telemetry around that outcome, not around “accuracy” or “BLEU score.” Select prompts, retrieval methods and models that demonstrably move those KPIs. At one global insurer, for instance, reframing success as “minutes saved per claim” instead of “model precision” turned an isolated pilot into a company-wide roadmap. A 3-layer telemetry model for LLM observability Just like microservices rely on logs, metrics and traces, AI systems need a structured observability stack: a) Prompts and context: What went in Log every prompt template, variable and retrieved document. Record model ID, version, latency and token counts (your leading cost indicators). Maintain an auditable redaction log showing what data was masked, when and by which rule. b) Policies and controls: The guardrails Capture safety-filter outcomes (toxicity, PII), citation presence and rule triggers. Store policy reasons and risk tier for each deployment. Link outputs back to the governing model card for transparency. c) Outcomes and feedback: Did it work? Gather human ratings and edit distances from accepted answers. Track downstream business events, case closed, document approved, issue resolved. Measure the KPI deltas, call time, backlog, reopen rate. All three layers connect through a common trace ID, enabling any decision to be replayed, audited or improved. Diagram © SaiKrishna Koorapati (2025). Created specifically for this article; licensed to VentureBeat for publication. Apply SRE discipline: SLOs and error budgets for AI Service reliability engineering (SRE) transformed software operations; now it’s AI’s turn. Define three “golden signals” for every critical workflow: Signal Target SLO When breached Factuality ≥ 95 % verified against source of record Fallback to verified template Safety ≥ 99.9 % pass toxicity/PII filters Quarantine and human review Usefulness ≥ 80 % accepted on first pass Retrain or rollback prompt/model If hallucinations or refusals exceed budget, the system auto-routes to safer prompts or human review just like rerouting traffic during a service outage. This isn’t bureaucracy; it’s reliability applied to reasoning. Build the thin observability layer in two agile sprints You don’t need a six-month roadmap, just focus and two short sprints. Sprint 1 (weeks 1-3): Foundations Version-controlled prompt registry Redaction middleware tied to policy Request/response logging with trace IDs Basic evaluations (PII checks, citation presence) Simple human-in-the-loop (HITL) UI Sprint 2 (weeks 4-6): Guardrails and KPIs Offline test sets (100–300 real examples) Policy gates for factuality and safety Lightweight dashboard tracking SLOs and cost Automated token and latency tracker In 6 weeks, you’ll have the thin layer that answers 90% of governance and product questions. Make evaluations continuous (and boring) Evaluations shouldn’t be heroic one-offs; they should be routine. Curate test sets from real cases; refresh 10–20 % monthly. Define clear acceptance criteria shared by product and risk teams. Run the suite on every prompt/model/policy change and weekly for drift checks. Publish one unified scorecard each week covering factuality, safety, usefulness and cost. When evals are part of CI/CD, they stop being compliance theater and become operational pulse checks. Apply human oversight where it matters Full automation is neither realistic nor responsible. High-risk or ambiguous cases should escalate to human review. Route low-confidence or policy-flagged responses to experts. Capture every edit and reason as training data and audit evidence. Feed reviewer feedback back into prompts and policies for continuous improvement. At one health-tech firm, this approach cut false positives by 22 % and produced a retrainable, compliance-ready dataset in weeks. Cost control through design, not hope LLM costs grow non-linearly. Budgets won’t save you architecture will. Structure prompts so deterministic sections run before generative ones. Compress and rerank context instead of dumping entire documents. Cache frequent queries and memoize tool outputs with TTL. Track latency, throughput and token use per feature. When observability covers tokens and latency, cost becomes a controlled variable, not a surprise. The 90-day playbook Within 3 months of adopting observable AI principles, enterprises should see: 1–2 production AI assists with HITL for edge cases Automated evaluation suite for pre-deploy and nightly runs Weekly scorecard shared across SRE, product and risk Audit-ready traces linking prompts, policies and outcomes At a Fortune 100 client, this structure reduced incident time by 40 % and aligned product and compliance roadmaps. Scaling trust through observability Observable AI is how you turn AI from experiment to infrastructure. With clear telemetry, SLOs and human feedback loops: Executives gain evidence-backed confidence. Compliance teams get replayable audit chains. Engineers iterate faster and ship safely. Customers experience reliable, explainable AI. Observability isn’t an add-on layer, it’s the foundation for trust at scale. SaiKrishna Koorapati is a software engineering leader. Read more from our guest writers. Or, consider submitting a post of your own! See our guidelines here.

Why do current audio AI models often perform worse when they generate longer reasoning instead of grounding their decisions in the actual sound. StepFun research team releases Step-Audio-R1, a new audio LLM designed for test time compute scaling, address this failure mode by showing that the accuracy drop with chain of thought is not an audio limitation but a training and modality grounding problem? https://arxiv.org/pdf/2511.15848 The Core Problem, Audio Models Reason over Text Surrogates Most current audio models inherit their reasoning behavior from text training. They learn to reason as if they read transcripts, not as if they listen. The StepFun team calls this Textual Surrogate Reasoning. The model uses imagined words and descriptions instead of acoustic cues such as pitch contour, rhythm, timbre or background noise patterns. This mismatch explains why longer chain of thought often hurts performance in audio. The model spends more tokens elaborating wrong or modality irrelevant assumptions. Step-Audio-R1 attacks this by forcing the model to justify answers using acoustic evidence. The training pipeline is organized around Modality Grounded Reasoning Distillation, MGRD, which selects and distills reasoning traces that explicitly reference audio features. Architecture The architecture stays close to the previous Step Audio systems: A Qwen2 based audio encoder processes raw waveforms at 25 Hz. An audio adaptor downsamples the encoder output by a factor of 2, to 12.5 Hz, and aligns frames to the language token stream. A Qwen2.5 32B decoder consumes the audio features and generates text. The decoder always produces an explicit reasoning block inside <think> and </think> tags, followed by the final answer. This separation lets training objectives shape the structure and content of reasoning without losing focus on task accuracy. The model is released as a 33B parameter audio text to text model on Hugging Face under Apache 2.0. https://arxiv.org/pdf/2511.15848 Training Pipeline, from Cold Start to Audio Grounded RL The pipeline has a supervised cold start stage and a reinforcement learning stage that both mix text and audio tasks. Cold start uses about 5 million examples, covering 1 billion tokens of text only data and 4 billion tokens from audio paired data. Audio tasks include automatic speech recognition, paralinguistic understanding and audio question text answer style dialogs. A fraction of the audio data carries audio chain of thought traces generated by an earlier model. Text data covers multi turn dialog, knowledge question answering, math and code reasoning. All samples share a format where reasoning is wrapped in <think> tags, even when the reasoning block is initially empty. Supervised learning trains Step-Audio-R1 to follow this format and to generate useful reasoning for both audio and text. This gives a baseline chain of thought behavior, but it is still biased toward text based reasoning. Modality Grounded Reasoning Distillation MGRD MGRD is applied in several iterations. For each round, the research team samples audio questions where the label depends on real acoustic properties. For example, questions about speaker emotion, background events in sound scenes or musical structure. The current model produces multiple reasoning and answer candidates per question. A filter keeps only chains that meet three constraints: They reference acoustic cues, not just textual descriptions or imagined transcripts. They are logically coherent as short step by step explanations. Their final answers are correct according to labels or programmatic checks. These accepted traces form a distilled audio chain of thought dataset. The model is fine tuned on this dataset together with the original text reasoning data. This is followed by Reinforcement Learning with Verified Rewards, RLVR. For text questions, rewards are based on answer correctness. For audio questions, the reward mixes answer correctness and reasoning format, with a typical weighting of 0.8 for accuracy and 0.2 for reasoning. Training uses PPO with about 16 responses sampled per prompt and supports sequences up to around 10 240 tokens to allow long deliberation. https://arxiv.org/pdf/2511.15848 Benchmarks, closing the gap to Gemini 3 Pro On a combined speech to text benchmark suite that includes Big Bench Audio, Spoken MQA, MMSU, MMAU and Wild Speech, Step-Audio-R1 reaches an average score of about 83.6 percent. Gemini 2.5 Pro reports about 81.5 percent and Gemini 3 Pro reaches about 85.1 percent. On Big Bench Audio alone, Step-Audio-R1 reaches about 98.7 percent, which is higher than both Gemini versions. For speech to speech reasoning, the Step-Audio-R1 Realtime variant adopts listen while thinking and think while speaking style streaming. On Big Bench Audio speech to speech, it reaches about 96.1 percent reasoning accuracy with first packet latency around 0.92 seconds. This score surpasses GPT based realtime baselines and Gemini 2.5 Flash style native audio dialogs while keeping sub second interaction. https://arxiv.org/pdf/2511.15848 Ablations, what matters for audio reasoning The ablation section provides several design signals for engineers: A reasoning format reward is necessary. Without it, reinforcement learning tends to shorten or remove chain of thought, which lowers audio benchmark scores. RL data should target medium difficulty problems. Selecting questions where pass at 8 lies in a middle band gives more stable rewards and maintains long reasoning. Scaling RL audio data without such selection does not help. Quality of prompts and labels matters more than raw size. The researchers also describe a self cognition correction pipeline that reduces the frequency of answers such as ‘I can only read text and cannot hear audio’ in a model that is trained to process sound. This uses Direct Preference Optimization on curated preference pairs where correct behavior is to acknowledge and use audio input. Key Takeaways Step-Audio-R1 is one of the first audio language model that turns longer chain of thought into a consistent accuracy gain for audio tasks, solving the inverted scaling failure seen in previous audio LLMs. The model explicitly targets Textual Surrogate Reasoning by using Modality Grounded Reasoning Distillation, which filters and distills only those reasoning traces that rely on acoustic cues such as pitch, timbre and rhythm instead of imagined transcripts. Architecturally, Step-Audio-R1 combines a Qwen2 based audio encoder with an adaptor and a Qwen2.5 32B decoder that always generates <think> reasoning segments before

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. What we still don’t know about weight-loss drugs Weight-loss drugs have been back in the news this week. First, we heard that Eli Lilly, the company behind Mounjaro and Zepbound, became the first healthcare company in the world to achieve a trillion-dollar valuation. But we also learned that, disappointingly, GLP-1 drugs don’t seem to help people with Alzheimer’s disease. And that people who stop taking the drugs when they become pregnant can experience potentially dangerous levels of weight gain. On top of that, some researchers worry that people are using the drugs postpartum to lose pregnancy weight without understanding potential risks. All of this news should serve as a reminder that there’s a lot we still don’t know about these drugs. So let’s look at the enduring questions surrounding GLP-1 agonist drugs. —Jessica Hamzelou This article first appeared in The Checkup, MIT Technology Review’s weekly biotech newsletter. To receive it in your inbox every Thursday, and read articles like this first, sign up here. If you’re interested in weight loss drugs and how they affect us, take a look at: + GLP-1 agonists like Wegovy, Ozempic, and Mounjaro might benefit heart and brain health—but research suggests they might also cause pregnancy complications and harm some users. Read the full story. + We’ve never understood how hunger works. That might be about to change. Read the full story. + Weight-loss injections have taken over the internet. But what does this mean for people IRL? + This vibrating weight-loss pill seems to work—in pigs. Read the full story. What we know about how AI is affecting the economy There’s a lot at stake when it comes to understanding how AI is changing the economy right now. Should we be pessimistic? Optimistic? Or is the situation too nuanced for that? Hopefully, we can point you towards some answers. Mat Honan, our editor in chief, will hold a special subscriber-only Roundtables conversation with our editor at large David Rotman, and Richard Waters, Financial Times columnist, exploring what’s happening across different markets. Register here to join us at 1pm ET on Tuesday December 9. The event is part of the Financial Times and MIT Technology Review “The State of AI” partnership, exploring the global impact of artificial intelligence. Over the past month, we’ve been running discussions between our journalists—sign up here to receive future editions every Monday. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 Tech billionaires are gearing up to fight AI regulation By amassing multi-million dollar war chests ahead of the 2026 US midterm elections. (WSJ $)+ Donald Trump’s “Manhattan Project” for AI is certainly ambitious. (The Information $) 2 The EU wants to hold social media platforms liable for financial scamsNew rules will force tech firms to compensate banks if they fail to remove reported scams. (Politico) 3 China is worried about a humanoid robot bubbleBecause more than 150 companies there are building very similar machines. (Bloomberg $)+ It could learn some lessons from the current AI bubble. (CNN)+ Why the humanoid workforce is running late. (MIT Technology Review) 4 A Myanmar scam compound was blown upBut its residents will simply find new bases for their operations. (NYT $)+ Experts suspect the destruction may have been for show. (Wired $)+ Inside a romance scam compound—and how people get tricked into being there. (MIT Technology Review) 5 Navies across the world are investing in submarine drones They cost a fraction of what it takes to run a traditional manned sub. (The Guardian)+ How underwater drones could shape a potential Taiwan-China conflict. (MIT Technology Review) 6 What to expect from China’s seemingly unstoppable innovation driveIts extremely permissive regulators play a big role. (Economist $)+ Is China about to win the AI race? (MIT Technology Review) 7 The UK is waging a war on VPNsGood luck trying to persuade people to stop using them. (The Verge) 8 We’re learning more about Jeff Bezos’ mysterious clock projectHe’s backed the Clock of the Long Now for years—and construction is amping up. (FT $)+ How aging clocks can help us understand why we age—and if we can reverse it. (MIT Technology Review) 9 Have we finally seen the first hints of dark matter?These researchers seem to think so. (New Scientist $) 10 A helpful robot is helping archaeologists reconstruct PompeiiReassembling ancient frescos is fiddly and time-consuming, but less so if you’re a dextrous machine. (Reuters) Quote of the day “We do fail… a lot.” —Defense company Anduril explains its move-fast-and-break-things ethos to the Wall Street Journal in response to reports its systems have been marred by issues in Ukraine. One more thing How to build a better AI benchmark It’s not easy being one of Silicon Valley’s favorite benchmarks. SWE-Bench (pronounced “swee bench”) launched in November 2024 as a way to evaluate an AI model’s coding skill. It has since quickly become one of the most popular tests in AI. A SWE-Bench score has become a mainstay of major model releases from OpenAI, Anthropic, and Google—and outside of foundation models, the fine-tuners at AI firms are in constant competition to see who can rise above the pack. Despite all the fervor, this isn’t exactly a truthful assessment of which model is “better.” Entrants have begun to game the system—which is pushing many others to wonder whether there’s a better way to actually measure AI achievement. Read the full story. —Russell Brandom We can still have nice things A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line or skeet ’em at me.) + Aww, these sharks appear to be playing with pool toys.+ Strange things are happening over on Easter Island (even weirder than you can imagine) + Very cool—archaeologists have uncovered a Roman tomb that’s been sealed shut for 1,700 years.+ This Japanese mass media collage is making my

Agent memory remains a problem that enterprises want to fix, as agents forget some instructions or conversations the longer they run.  Anthropic believes it has solved this issue for its Claude Agent SDK, developing a two-fold solution that allows an agent to work across different context windows. “The core challenge of long-running agents is that they must work in discrete sessions, and each new session begins with no memory of what came before,” Anthropic wrote in a blog post. “Because context windows are limited, and because most complex projects cannot be completed within a single window, agents need a way to bridge the gap between coding sessions.” Anthropic engineers proposed a two-fold approach for its Agent SDK: An initializer agent to set up the environment, and a coding agent to make incremental progress in each session and leave artifacts for the next.   The agent memory problem Since agents are built on foundation models, they remain constrained by the limited, although continually growing, context windows. For long-running agents, this could create a larger problem, leading the agent to forget instructions and behave abnormally while performing a task. Enhancing agent memory becomes essential for consistent, business-safe performance.  Several methods emerged over the past year, all attempting to bridge the gap between context windows and agent memory. LangChain’s LangMem SDK, Memobase and OpenAI’s Swarm are examples of companies offering memory solutions. Research on agentic memory has also exploded recently, with proposed frameworks like Memp and the Nested Learning Paradigm from Google offering new alternatives to enhance memory.  Many of the current memory frameworks are open source and can ideally adapt to different large language models (LLMs) powering agents. Anthropic’s approach improves its Claude Agent SDK.  How it works Anthropic identified that even though the Claude Agent SDK had context management capabilities and “should be possible for an agent to continue to do useful work for an arbitrarily long time,” it was not sufficient. The company said in its blog post that a model like Opus 4.5 running the Claude Agent SDK can “fall short of building a production-quality web app if it’s only given a high-level prompt, such as ‘build a clone of claude.ai.’”  The failures manifested in two patterns, Anthropic said. First, the agent tried to do too much, causing the model to run out of context in the middle. The agent then has to guess what happened and cannot pass clear instructions to the next agent. The second failure occurs later on, after some features have already been built. The agent sees progress has been made and just declares the job done.  Anthropic researchers broke down the solution: Setting up an initial environment to lay the foundation for features and prompting each agent to make incremental progress towards a goal, while still leaving a clean slate at the end.  This is where the two-part solution of Anthropic’s agent comes in. The initializer agent sets up the environment, logging what agents have done and which files have been added. The coding agent will then ask models to make incremental progress and leave structured updates.  “Inspiration for these practices came from knowing what effective software engineers do every day,” Anthropic said.  The researchers said they added testing tools to the coding agent, improving its ability to identify and fix bugs that weren’t obvious from the code alone.  Future research Anthropic noted that its approach is “one possible set of solutions in a long-running agent harness.” However, this is just the beginning stage of what could become a wider research area for many in the AI space.  The company said its experiments to boost long-term memory for agents haven’t shown whether a single general-purpose coding agent works best across contexts or a multi-agent structure.  Its demo also focused on full-stack web app development, so other experiments should focus on generalizing the results across different tasks. “It’s likely that some or all of these lessons can be applied to the types of long-running agentic tasks required in, for example, scientific research or financial modeling,” Anthropic said. 

Hello, dear readers. Happy belated Thanksgiving and Black Friday! This year has felt like living inside a permanent DevDay. Every week, some lab drops a new model, a new agent framework, or a new “this changes everything” demo. It’s overwhelming. But it’s also the first year I’ve felt like AI is finally diversifying — not just one or two frontier models in the cloud, but a whole ecosystem: open and closed, giant and tiny, Western and Chinese, cloud and local. So for this Thanksgiving edition, here’s what I’m genuinely thankful for in AI in 2025 — the releases that feel like they’ll matter in 12–24 months, not just during this week’s hype cycle. 1. OpenAI kept shipping strong: GPT-5, GPT-5.1, Atlas, Sora 2 and open weights As the company that undeniably birthed the “generative AI” era with its viral hit product ChatGPT in late 2022, OpenAI arguably had among the hardest tasks of any AI company in 2025: continue its growth trajectory even as well-funded competitors like Google with its Gemini models and other startups like Anthropic fielded their own highly competitive offerings. Thankfully, OpenAI rose to the challenge and then some. Its headline act was GPT-5, unveiled in August as the next frontier reasoning model, followed in November by GPT-5.1 with new Instant and Thinking variants that dynamically adjust how much “thinking time” they spend per task. In practice, GPT-5’s launch was bumpy — VentureBeat documented early math and coding failures and a cooler-than-expected community reaction in “OpenAI’s GPT-5 rollout is not going smoothly,” but it quickly course corrected based on user feedback and, as a daily user of this model, I’m personally pleased with it and impressed with it. At the same time, enterprises actually using the models are reporting solid gains. ZenDesk Global, for example, says GPT-5-powered agents now resolve more than half of customer tickets, with some customers seeing 80–90% resolution rates. That’s the quiet story: these models may not always impress the chattering classes on X, but they’re starting to move real KPIs. On the tooling side, OpenAI finally gave developers a serious AI engineer with GPT-5.1-Codex-Max, a new coding model that can run long, agentic workflows and is already the default in OpenAI’s Codex environment. VentureBeat covered it in detail in “OpenAI debuts GPT-5.1-Codex-Max coding model and it already completed a 24-hour task internally.” Then there’s ChatGPT Atlas, a full browser with ChatGPT baked into the chrome itself — sidebar summaries, on-page analysis, and search tightly integrated into regular browsing. It’s the clearest sign yet that “assistant” and “browser” are on a collision course. On the media side, Sora 2 turned the original Sora video demo into a full video-and-audio model with better physics, synchronized sound and dialogue, and more control over style and shot structure, plus a dedicated Sora app with a full fledged social networking component, allowing any user to create their own TV network in their pocket. Finally — and maybe most symbolically — OpenAI released gpt-oss-120B and gpt-oss-20B, open-weight MoE reasoning models under an Apache 2.0–style license. Whatever you think of their quality (and early open-source users have been loud about their complaints), this is the first time since GPT-2 that OpenAI has put serious weights into the public commons. 2. China’s open-source wave goes mainstream If 2023–24 was about Llama and Mistral, 2025 belongs to China’s open-weight ecosystem. A study from MIT and Hugging Face found that China now slightly leads the U.S. in global open-model downloads, largely thanks to DeepSeek and Alibaba’s Qwen family. Highlights: DeepSeek-R1 dropped in January as an open-source reasoning model rivaling OpenAI’s o1, with MIT-licensed weights and a family of distilled smaller models. VentureBeat has followed the story from its release to its cybersecurity impact to performance-tuned R1 variants. Kimi K2 Thinking from Moonshot, a “thinking” open-source model that reasons step-by-step with tools, very much in the o1/R1 mold, and is positioned as the best open reasoning model so far in the world. Z.ai shipped GLM-4.5 and GLM-4.5-Air as “agentic” models, open-sourcing base and hybrid reasoning variants on GitHub. Baidu’s ERNIE 4.5 family arrived as a fully open-sourced, multimodal MoE suite under Apache 2.0, including a 0.3B dense model and visual “Thinking” variants focused on charts, STEM, and tool use. Alibaba’s Qwen3 line — including Qwen3-Coder, large reasoning models, and the Qwen3-VL series released over the summer and fall months of 2025 — continues to set a high bar for open weights in coding, translation, and multimodal reasoning, leading me to declare this past summer as “ Qwen’s summer.“ VentureBeat has been tracking these shifts, including Chinese math and reasoning models like Light-R1-32B and Weibo’s tiny VibeThinker-1.5B, which beat DeepSeek baselines on shoestring training budgets. If you care about open ecosystems or on-premise options, this is the year China’s open-weight scene stopped being a curiosity and became a serious alternative. 3. Small and local models grow up Another thing I’m thankful for: we’re finally getting good small models, not just toys. Liquid AI spent 2025 pushing its Liquid Foundation Models (LFM2) and LFM2-VL vision-language variants, designed from day one for low-latency, device-aware deployments — edge boxes, robots, and constrained servers, not just giant clusters. The newer LFM2-VL-3B targets embedded robotics and industrial autonomy, with demos planned at ROSCon. On the big-tech side, Google’s Gemma 3 line made a strong case that “tiny” can still be capable. Gemma 3 spans from 270M parameters up through 27B, all with open weights and multimodal support in the larger variants. The standout is Gemma 3 270M, a compact model purpose-built for fine-tuning and structured text tasks — think custom formatters, routers, and watchdogs — covered both in Google’s developer blog and community discussions in local-LLM circles. These models may never trend on X, but they’re exactly what you need for privacy-sensitive workloads, offline workflows, thin-client devices, and “agent swarms” where you don’t want every tool call hitting a giant frontier LLM. 4. Meta + Midjourney: aesthetics as a service One of the stranger twists this year: Meta

This article is divided into two parts; they are: • Fine-tuning a BERT Model for GLUE Tasks • Fine-tuning a BERT Model for SQuAD Tasks GLUE is a benchmark for evaluating natural language understanding (NLU) tasks.

How can an AI system learn to pick the right model or tool for each step of a task instead of always relying on one large model for everything? NVIDIA researchers release ToolOrchestra, a novel method for training a small language model to act as the orchestrator- the ‘brain’ of a heterogeneous tool-use agent https://arxiv.org/pdf/2511.21689 From Single Model Agents to an Orchestration Policy Most current agents follow a simple pattern. A single large model such as GPT-5 receives a prompt that describes available tools, then decides when to call web search or a code interpreter. All high level reasoning still stays inside the same model. ToolOrchestra changes this setup. It trains a dedicated controller model called as ‘Orchestrator-8B‘, that treats both classic tools and other LLMs as callable components. A pilot study in the same research shows why naive prompting is not enough. When Qwen3-8B is prompted to route between GPT-5, GPT-5 mini, Qwen3-32B and Qwen2.5-Coder-32B, it delegates 73 percent of cases to GPT-5. When GPT-5 acts as its own orchestrator, it calls GPT-5 or GPT-5 mini in 98 percent of cases. The research team call these self enhancement and other enhancement biases. The routing policy over uses strong models and ignores cost instructions. ToolOrchestra instead trains a small orchestrator explicitly for this routing problem, using reinforcement learning over full multi turn trajectories. What is Orchestrator 8B? Orchestrator-8B is an 8B parameter decoder only Transformer. It is built by fine tuning Qwen3-8B as an orchestration model and released on Hugging Face. At inference time, the system runs a multi turn loop that alternates reasoning and tool calls. The rollout has three main steps. First, Orchestrator 8B reads the user instruction and an optional natural language preference description, for example a request to prioritize low latency or to avoid web search. Second, it generates internal chain of thought style reasoning and plans an action. Third, it chooses a tool from the available set and emits a structured tool call in a unified JSON format. The environment executes that call, appends the result as an observation and feeds it back into the next step. The process stops when a termination signal is produced or a maximum of 50 turns is reached. Tools cover three main groups. Basic tools include Tavily web search, a Python sandbox code interpreter and a local Faiss index built with Qwen3-Embedding-8B. Specialized LLMs include Qwen2.5-Math-72B, Qwen2.5-Math-7B and Qwen2.5-Coder-32B. Generalist LLM tools include GPT-5, GPT-5 mini, Llama 3.3-70B-Instruct and Qwen3-32B. All tools share the same schema with names, natural language descriptions and typed parameter specs. End to End Reinforcement Learning with Multi Objective Rewards ToolOrchestra formulates the whole workflow as a Markov Decision Process. The state contains the conversation history, past tool calls and observations, and user preferences. Actions are the next text step, including both reasoning tokens and a tool call schema. After up to 50 steps, the environment computes a scalar reward for the full trajectory. The reward has three components. Outcome reward is binary and depends on whether the trajectory solves the task. For open-ended answers, GPT-5 is used as a judge to compare the model output with the reference. Efficiency rewards penalize both monetary cost and wall clock latency. Token usage for proprietary and open source tools is mapped to monetary cost using public API and Together AI pricing. Preference reward measures how well tool usage matches a user preference vector that can increase or decrease the weight on cost, latency or specific tools. These components are combined into a single scalar using the preference vector. The policy is optimized with Group Relative Policy Optimization GRPO, a variant of policy gradient reinforcement learning that normalizes rewards within groups of trajectories for the same task. The training process includes filters that drop trajectories with invalid tool call format or weak reward variance to stabilize optimization. https://arxiv.org/pdf/2511.21689 To make this training possible at scale, the research team plans to introduce ToolScale, a synthetic dataset of multi step tool calling tasks. For each domain, an LLM generates a database schema, database entries, domain specific APIs and then diverse user tasks with ground truth sequences of function calls and required intermediate information. Benchmark results and cost profile NVIDIA research team evaluates Orchestrator-8B on three challenging benchmarks, Humanity’s Last Exam, FRAMES and τ² Bench. These benchmarks target long horizon reasoning, factuality under retrieval and function calling in a dual control environment. On Humanity’s Last Exam text only questions, Orchestrator-8B reaches 37.1 percent accuracy. GPT-5 with basic tools reaches 35.1 percent in the same setting. On FRAMES, Orchestrator-8B achieves 76.3 percent versus 74.0 percent for GPT-5 with tools. On τ² Bench, Orchestrator-8B scores 80.2 percent versus 77.7 percent for GPT-5 with basic tools. https://arxiv.org/pdf/2511.21689 The efficiency gap is larger. In the configuration that uses basic tools plus specialized and generalist LLM tools, Orchestrator-8B has average cost 9.2 cents and latency 8.2 minutes per query, averaged over Humanity’s Last Exam and FRAMES. In the same configuration, GPT-5 costs 30.2 cents and takes 19.8 minutes on average. The model card summarizes this as about 30 percent of the monetary cost and 2.5 times faster for Orchestrator-8B compared to GPT-5. Tool use analysis supports this picture. Claude Opus 4.1 used as an orchestrator calls GPT-5 most of the time. GPT-5 used as an orchestrator prefers GPT-5 mini. Orchestrator-8B spreads calls more evenly across strong models, cheaper models, search, local retrieval and the code interpreter, and reaches higher accuracy at lower cost for the same turn budget. https://arxiv.org/pdf/2511.21689 Generalization experiments replace the training time tools with unseen models such as OpenMath Llama-2-70B, DeepSeek-Math-7B-Instruct, Codestral-22B-v0.1, Claude Sonnet-4.1 and Gemma-3-27B. Orchestrator-8B still achieves the best trade off between accuracy, cost and latency among all baselines in this setting. A separate preference aware test set shows that Orchestrator-8B also tracks user tool usage preferences more closely than GPT-5, Claude Opus-4.1 and Qwen3-235B-A22B under the same reward metric. Key Takeaways ToolOrchestra trains an 8B parameter orchestration model, Orchestrator-8B, that selects and sequences tools and LLMs to solve multi

Tencent Hunyuan has released HunyuanOCR, a 1B parameter vision language model that is specialized for OCR and document understanding. The model is built on Hunyuan’s native multimodal architecture and runs spotting, parsing, information extraction, visual question answering, and text image translation through a single end to end pipeline. HunyuanOCR is a lightweight alternative to general VLMs such as Gemini 2.5 and Qwen3 VL that still matches or surpasses them on OCR centric tasks. It targets production use cases like document parsing, card and receipt extraction, video subtitle extraction, and multilingual document translation. https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/HunyuanOCR_Technical_Report.pdf Architecture, Native Resolution ViT plus Lightweight LLM HunyuanOCR uses 3 main modules, a Native Resolution Visual Encoder called Hunyuan ViT, an Adaptive MLP Connector, and a Lightweight Language Model. The encoder is based on SigLIP-v2-400M and is extended to support arbitrary input resolutions through adaptive patching that preserves the original aspect ratio. Images are split into patches according to their native proportions and processed with global attention, which improves recognition on long text lines, long documents, and low quality scans. The Adaptive MLP Connector performs learnable pooling on the spatial dimension. It compresses the dense visual tokens into a shorter sequence, while keeping information from text dense regions. This reduces sequence length passed to the language model and lowers compute, while preserving OCR relevant details. The language model is based on the densely architected Hunyuan 0.5B model and uses XD RoPE. XD RoPE splits rotary position embeddings into 4 subspaces for text, height, width, and time. This gives the model a native way to align 1D token order with 2D layout and 3D spatiotemporal structure. As a result, the same stack can handle multi column pages, cross page flows, and sequences of video frames. Training and inference follow a fully end to end paradigm. There is no external layout analysis or post processing model in the loop. All tasks are expressed as natural language prompts and handled in a single forward pass. This design removes error propagation across pipeline stages and simplifies deployment. Data and Pre Training Recipe The data pipeline builds more than 200M image text pairs, across 9 real world scenarios, including street views, documents, advertisements, handwritten text, screenshots, cards and certificates and invoices, game interfaces, video frames, and artistic typography. The corpus covers more than 130 languages. Synthetic data comes from a multilingual generator that supports right to left scripts and paragraph level rendering. The pipeline controls font, language, rotation, and RGB values, and applies warping, blur, and local lighting changes to simulate mobile captures and other hard conditions. https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/HunyuanOCR_Technical_Report.pdf Pre training follows 4 stages. Stage-1 performs vision language alignment with pure text, synthetic parsing and recognition data, and general caption data, using 50B tokens and 8k context. Stage-2 runs multimodal pre training on 300B tokens that mix pure text with synthetic spotting, parsing, translation, and VQA samples. Stage-3 extends context length to 32k with 80B tokens focused on long documents and long text. Stage-4 is application oriented supervised fine tuning on 24B tokens of human annotated and hard negative data, keeping 32k context and unified instruction templates. Reinforcement Learning with Verifiable Rewards After supervised training, HunyuanOCR is further optimized with reinforcement learning. The research team use Group Relative Policy Optimization GRPO and a Reinforcement Learning with Verifiable Rewards setup for structured tasks. For text spotting, the reward is based on intersection over union matching of boxes combined with normalized edit distance over text. For document parsing, the reward uses normalized edit distance between the generated structure and the reference. For VQA and translation, the system uses an LLM as a judge. VQA uses a binary reward that checks semantic match. Translation uses a COMET style scoring LLM with scores in [0, 5], normalized to [0, 1]. The training framework enforces length limits and strict formats, and assigns zero reward when outputs overflow or break schema, which stabilizes optimization and encourages valid JSON or structured outputs. Benchmark Results, a 1B Model Competing with Larger VLMs On the internal text spotting benchmark of 900 images across 9 categories, HunyuanOCR reaches an overall score of 70.92. It outperforms traditional pipeline methods like PaddleOCR and BaiduOCR and also general VLMs such as Gemini 2.5 Pro, Qwen3 VL 2B, Qwen3 VL 235B, and Seed 1.6 Vision, despite using far fewer parameters. On OmniDocBench, HunyuanOCR achieves 94.10 overall, with 94.73 on formulas and 91.81 on tables. On the Wild OmniDocBench variant, which prints and recaptures documents under folds and lighting changes, it scores 85.21 overall. On DocML, a multilingual parsing benchmark across 14 non Chinese and non English languages, it reaches 91.03, and the paper reports state of the art results across all 14 languages. For information extraction and VQA, HunyuanOCR reaches 92.29 accuracy on cards, 92.53 on receipts, and 92.87 on video subtitles. On OCRBench, it scores 860, higher than DeepSeek OCR at similar scale and close to larger general VLMs like Qwen3 VL 2B Instruct and Gemini 2.5 Pro. In text image translation, HunyuanOCR uses the DoTA benchmark and a DocML based internal set. It achieves a strong COMET score on DoTA for English to Chinese document translation, and the model wins first place in Track 2.2 OCR free Small Model of the ICDAR 2025 DIMT competition. https://github.com/Tencent-Hunyuan/HunyuanOCR/blob/main/HunyuanOCR_Technical_Report.pdf Key Takeaways Compact end to end OCR VLM: HunyuanOCR is a 1B parameter OCR focused vision language model that connects a 0.4B native resolution ViT to a 0.5B Hunyuan language model through an MLP adapter, and runs spotting, parsing, information extraction, VQA and translation in one end to end instruction driven pipeline without external layout or detection modules. Unified support for diverse OCR scenarios: The model is trained on more than 200M image text pairs across 9 scenarios, including documents, street views, advertisements, handwritten content, screenshots, cards and invoices, game interfaces and video frames, with coverage of over 130 languages in training and support for more than 100 languages in deployment. Data pipeline plus reinforcement learning: Training uses a 4 stage recipe, vision language alignment, multimodal pre training,

Today’s IT leaders face competing mandates to do more (“make us an ‘AI-first’ enterprise—yesterday”) with less (“no new hires for at least the next six months”). VMware has become a focal point of these dueling directives. It remains central to enterprise IT, with 80% of organizations using VMware infrastructure products. But shifting licensing models are prompting teams to reconsider how they manage and scale these workloads, often on tighter budgets. DOWNLOAD THE ARTICLE For many organizations, the path forward involves adopting a LessOps model, an operational strategy that makes hybrid environments manageable without increasing headcount. This operational philosophy minimizes human intervention through extensive automation and selfservice capabilities while maintaining governance and compliance. In practice, VMware-to-cloud migrations create a “two birds, one stone” opportunity. They present a practical moment to codify the automation and governance practices LessOps depends on—laying the groundwork for a leaner, more resilient IT operating model. Download the full article. This content was produced by Insights, the custom content arm of MIT Technology Review. It was not written by MIT Technology Review’s editorial staff. This content was researched, designed, and written by human writers, editors, analysts, and illustrators. This includes the writing of surveys and collection of data for surveys. AI tools that may have been used were limited to secondary production processes that passed thorough human review.