{"id":92592,"date":"2026-05-24T16:58:45","date_gmt":"2026-05-24T16:58:45","guid":{"rendered":"https:\/\/youzum.net\/microsoft-research-releases-webwright-a-terminal-native-web-agent-framework-that-scores-60-1-on-odysseys-up-from-base-gpt-5-4s-33-5\/"},"modified":"2026-05-24T16:58:45","modified_gmt":"2026-05-24T16:58:45","slug":"microsoft-research-releases-webwright-a-terminal-native-web-agent-framework-that-scores-60-1-on-odysseys-up-from-base-gpt-5-4s-33-5","status":"publish","type":"post","link":"https:\/\/youzum.net\/zh\/microsoft-research-releases-webwright-a-terminal-native-web-agent-framework-that-scores-60-1-on-odysseys-up-from-base-gpt-5-4s-33-5\/","title":{"rendered":"Microsoft Research Releases Webwright: A Terminal-Native Web Agent Framework That Scores 60.1% on Odysseys, Up from Base GPT-5.4\u2019s 33.5%"},"content":{"rendered":"

Most web agents today drive a browser one action at a time. The model receives the current page state \u2014 as a screenshot or DOM text \u2014 and predicts the next click, keypress, or scroll. This action-at-a-time design made sense when language models had limited reasoning ability. As models have become more capable at writing and debugging code, that rigid loop has become a constraint rather than a structure that helps.<\/p>\n

Microsoft Research\u2019s AI Frontiers lab built a different approach. Their new open-source framework, Webwright<\/strong>, gives the agent a terminal instead of a stateful browser session. The agent writes Playwright code to control browsers, runs bash commands, inspects logs, and iteratively refines scripts. Playwright is an open-source browser automation library, also from Microsoft, that supports programmatic control of Chromium, Firefox, and WebKit browsers. <\/p>\n

What Webwright Does Differently<\/strong><\/h2>\n

Webwright<\/strong> separates the agent from the browser and treats the browser as something the agent can launch, inspect, and discard while developing a program. The persistent artifact is not the browser session but the code and logs in the local workspace. <\/p>\n

This is the same model a developer uses when writing an RPA (Robotic Process Automation) script. Instead of manually clicking through a site each time, they write a script once. That script can be rerun, adapted, and shared. Webwright applies this to LLM-powered agents.<\/p>\n

The system has three core components:<\/strong> a Runner, a Model Endpoint, and a terminal Environment. The runner is about 150 lines of code, the model interface about 550 lines, and the environment about 300 lines. There is no multi-agent orchestration or complex planning hierarchy \u2014 just a single agent loop.<\/p>\n

All intermediate code, logs, screenshots, and results are stored in the workspace, making each run easy to inspect. <\/p>\n

\n
\"\"
https:\/\/www.microsoft.com\/en-us\/research\/articles\/webwright-a-terminal-is-all-you-need-for-web-agents\/<\/figcaption><\/figure>\n<\/div>\n

The Agent Loop<\/strong><\/h2>\n

The Runner sends the current context to the model. The model returns a thinking block and a shell command. That command runs in the Environment, which returns terminal output, logs, screenshots, or error tracebacks. These observations go back into context, and the loop continues.<\/p>\n

Rather than issuing one primitive action at a time, a coding agent can naturally express multi-step interactions \u2014 such as selecting a date or filling out an entire form \u2014 as a compact program. Loops, functions, and abstractions allow the agent to generalize across similar tasks without repeatedly predicting similar sequences of low-level steps.<\/p>\n

Two Engineering Challenges<\/strong><\/h2>\n

Premature \u2018done\u2019 and context explosion are the two core issues. With open-ended bash actions, the model must self-report completion and often claims success without actually finishing. They added a gate: the agent must generate a self-reflection config, run a final script in a fresh folder with logs and screenshots, and pass its own self-reflection judgement that outputs success or failure before emitting done: true<\/code>. Otherwise, the flag is dropped and it retries. <\/p>\n

For context length, long coding trajectories quickly exceed context limits, so they compact history every 20 steps into a single summary. <\/p>\n

Benchmark Results<\/strong><\/h2>\n

Webwright was evaluated on two benchmarks: Online-Mind2Web and Odysseys.<\/strong><\/p>\n

Online-Mind2Web contains 300 tasks across 136 widely used sites and uses an automated LLM-as-a-Judge evaluation framework. GPT-5.4 achieves 86.67% overall accuracy, representing the highest among all open-sourced harness recipes in the AutoEval category of the Online-Mind2Web benchmark, with a 100-step budget. Claude Opus 4.7 reached 84.7% overall but performed better on hard tasks at N=100 steps \u2014 80.5% versus 76.6% for GPT-5.4. <\/p>\n

They also reproduced a GPT-5.4 baseline in a conventional screenshot-based agent setting, where the model predicts x,y coordinates for clicks and typing actions. Using the same underlying model, Webwright achieves substantial gains across all three difficulty categories, highlighting the benefit of the code-driven terminal-based approach over step-by-step coordinate prediction.<\/p>\n

Odysseys evaluates long-horizon browsing tasks spanning multiple websites. Tasks average 272.3 words of instructions. In the April 2026 leaderboard, the best-performing model was Opus 4.6, with a top score of 44.5. Webwright powered by GPT-5.4 reaches 60.1%, a 35.1% relative improvement over the previous state of the art. Compared to the base GPT-5.4 performance of 33.5%, this corresponds to a 79.4% relative improvement \u2014 or 26.6 absolute points.<\/p>\n

Cost Analysis<\/strong><\/h2>\n

Claude Opus 4.7 is more efficient in the number of steps to solve each task (mean 21.9 steps) compared to GPT-5.4 (mean 26.3 steps). However, Claude Opus 4.7 is priced significantly higher compared to GPT-5.4 ($5 vs. $2.50 per 1M input tokens, and $25 vs. $15.00 per 1M output tokens, April 2026), which makes the average per-task cost higher compared to GPT-5.4 ($2.37 vs. $6.09). The first 50 steps deliver 82% accuracy, and the next 50 steps deliver 3\u20134 additional points. <\/p>\n

Small Model Performance<\/strong><\/h2>\n

The research team also tested Qwen3.5-9B on the hard split of Online-Mind2Web. When tasks are augmented with pre-built reusable tool scripts, Qwen3.5-9B achieves 66.2% on Online-Mind2Web websites with more than five tools. This shows that smaller, lower-cost models can handle complex web tasks when paired with a pre-built tool library. <\/p>\n

Marktechpost\u2019s Visual Explainer<\/strong><\/h2>\n
\n
\n Webwright<\/span>
\n Quick Start Guide<\/span>\n<\/div>\n
\n
\n

<\/p>\n

\n 01 \/ 05 \u2014 Overview<\/span>
\n What Is Webwright?<\/span>
\n Webwright is an open-source, terminal-native web agent framework from Microsoft Research<\/b>. Instead of predicting one browser click at a time, the agent writes Playwright<\/b> code, runs bash commands, and stores reusable scripts in a local workspace.<\/span>\n
<\/div>\n