{"id":103948,"date":"2026-07-13T19:15:41","date_gmt":"2026-07-13T19:15:41","guid":{"rendered":"https:\/\/youzum.net\/prime-intellect-releases-verifiers-v1-composable-tasksets-harnesses-and-runtimes-for-agentic-rl-training-and-evaluations\/"},"modified":"2026-07-13T19:15:41","modified_gmt":"2026-07-13T19:15:41","slug":"prime-intellect-releases-verifiers-v1-composable-tasksets-harnesses-and-runtimes-for-agentic-rl-training-and-evaluations","status":"publish","type":"post","link":"https:\/\/youzum.net\/fr\/prime-intellect-releases-verifiers-v1-composable-tasksets-harnesses-and-runtimes-for-agentic-rl-training-and-evaluations\/","title":{"rendered":"Prime Intellect Releases Verifiers v1: Composable Tasksets, Harnesses, and Runtimes for Agentic RL Training and Evaluations"},"content":{"rendered":"

Prime Intellect launched verifiers 0.2.0<\/code><\/a><\/strong>. It previews a rewritten core, shipped under the new verifiers.v1<\/a><\/code> namespace. Modern evaluations now run coding agents with tools, compaction, and subagents. Accordingly, v1 rebuilds environments to run these agentic workloads at scale.<\/p>\n

What is verifiers v1?<\/strong><\/h2>\n

First, consider what verifiers is: Prime Intellect\u2019s environment stack for agentic reinforcement learning and evaluations. Previously, an environment bundled its data, agent logic, and infrastructure together. In contrast, v1 breaks that bundle into three composable pieces.<\/p>\n

A taskset<\/strong> defines the work: the data, tools, and scoring. A harness<\/strong> solves the task and produces a rollout. That harness can be a ReAct loop, a CLI agent, or your own. The rollout then runs inside a runtime<\/strong>, either local or in a sandbox. Because the pieces decouple, any taskset runs under any compatible harness.<\/p>\n

How the Architecture Works<\/strong>?<\/h2>\n

With those pieces defined, the next question is how they communicate. The central piece is the verifiers-managed interception server<\/strong>. It sits between the agent\u2019s runtime and the inference server. Specifically, it proxies requests to, and responses from, inference. Meanwhile, it records the trace, sets sampling parameters, and can rewrite tool responses. That rewriting helps mitigate reward hacks during training.<\/p>\n

For scale, each server multiplexes a constant number of rollouts, defaulting to 32. A pool then scales elastically with observed concurrency. The server also owns a client<\/strong> that relays those requests. During evaluation, an EvalClient<\/code> acts as a blind HTTP proxy. During training, a TrainClient<\/code> wraps renderers<\/code> for faithful token-in RL training.<\/p>\n

Because harnesses speak different dialects<\/strong>, verifiers supports three as of now. These are OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. A dialect adapter normalizes each wire format into canonical vf.types<\/code>. Consequently, your scoring logic stays independent of the agent tested.<\/p>\n


\n Run rollout<\/button>
\n <button id=”vf-reset” class=”vf-ghost”>Reset<\/button>
\n <span class="”vf-lab”">Harness dialect:<\/span>
\n <select id=”vf-dialect”>
\n <option value=”Chat”>OpenAI Chat Completions<\/option>
\n <option value=”Resp”>OpenAI Responses<\/option>
\n <option value=”Msg”>Anthropic Messages<\/option>
\n <\/select>
\n <\/div><\/p>\n

<div class="”vf-stage”">
\n <div class="”vf-row”" style="”margin-bottom:14px”">
\n <div class="”vf-node" vf-taskset” id="”n-taskset”">
\n <div class="”vf-nt”">Taskset<\/div>
\n <div class="”vf-nd”">what · data · tools · scoring<\/div>
\n <\/div>
\n <\/div><\/p>\n

<div class="”vf-runtime-wrap”">
\n <span class="”vf-runtime-tag”">RUNTIME · where (subprocess · Docker · sandbox)<\/span>
\n <div class="”vf-row”" id="”vf-flow”">
\n <div class="”vf-node" vf-harness” id="”n-harness”">
\n <div class="”vf-nt”">Harness<\/div>
\n <div class="”vf-nd”">how · Codex · Terminus 2 · ReAct<\/div>
\n <\/div>
\n <div class="”vf-arrow”">→<\/div>
\n <div class="”vf-node" vf-intercept” id="”n-intercept”">
\n <div class="”vf-nt”">Interception Server<\/div>
\n <div class="”vf-nd”">proxy · records trace<\/div>
\n <\/div>
\n <div class="”vf-arrow”">→<\/div>
\n <div class="”vf-node" vf-infer” id="”n-infer”">
\n <div class="”vf-nt”">Inference Server<\/div>
\n <div class="”vf-nd”">vLLM · model<\/div>
\n <\/div>
\n <div class="”vf-packet”" id="”vf-packet”">req<\/div>
\n <\/div>
\n <\/div><\/p>\n

<div class="”vf-status”" id="”vf-status”">Press “Run rollout” to send a request through the interception server.<\/div>
\n <\/div><\/p>\n

<div class="”vf-grid”">
\n <div class="”vf-panel”">
\n <h3>Trace · message graph (v1)<\/h3>
\n <div class="”vf-hint”">Each message is a unique node. Size grows linearly in turns.<\/div>
\n <div class="”vf-graph”" id="”vf-graph”">
\n <div class="”vf-empty”">No messages recorded yet.<\/div>
\n <\/div>
\n <\/div><\/p>\n

<div class="”vf-panel”">
\n <h3>Trace size: v0 vs v1<\/h3>
\n <div class="”vf-hint”">Drag to change turns. v0 repeats prompt-completion pairs; v1 stores unique nodes.<\/div>
\n <div class="”vf-chart”">
\n <svg viewBox=”0 0 260 150″ id=”vf-svg”>
\n <line x1=”30″ y1=”130″ x2=”255″ y2=”130″ stroke=”#dfe6ef” stroke-width=”1.5″\/>
\n <line x1=”30″ y1=”10″ x2=”30″ y2=”130″ stroke=”#dfe6ef” stroke-width=”1.5″\/>
\n <path id=”vf-v0″ fill=”none” stroke=”#d1477a” stroke-width=”2.5″\/>
\n <path id=”vf-v1″ fill=”none” stroke=”#0b8f8f” stroke-width=”2.5″\/>
\n <text x=”140″ y=”147″ font-size=”9″ fill=”#94a3b8″ text-anchor=”middle”>turns \u2192<\/text>
\n <\/svg>
\n <\/div>
\n <div class="”vf-legend”">
\n <span><i style="”background:#d1477a”"><\/i> v0 · quadratic<\/span>
\n <span><i style="”background:#0b8f8f”"><\/i> v1 · linear<\/span>
\n <\/div>
\n <div class="”vf-slider-row”">
\n <span>Turns<\/span>
\n <input type=”range” id=”vf-turns” min=”4″ max=”60″ value=”24″>
\n <span id="”vf-turns-val”" style="”width:26px;text-align:right”">24<\/span>
\n <\/div>
\n <\/div>
\n <\/div><\/p>\n

<div class="”vf-foot”">
\n Illustrative demo of the verifiers v1 architecture · Built by <b>Marktechpost<\/b>
\n <\/div>
\n<\/div><\/p>\n

<script>
\n(function(){
\n var root=document.getElementById(“vfv1-demo”);
\n var packet=document.getElementById(“vf-packet”);
\n var status=document.getElementById(“vf-status”);
\n var graph=document.getElementById(“vf-graph”);
\n var runBtn=document.getElementById(“vf-run”);
\n var resetBtn=document.getElementById(“vf-reset”);
\n var dialectSel=document.getElementById(“vf-dialect”);
\n var nHarness=document.getElementById(“n-harness”);
\n var nIntercept=document.getElementById(“n-intercept”);
\n var nInfer=document.getElementById(“n-infer”);
\n var flow=document.getElementById(“vf-flow”);<\/p>\n

var turn=0, running=false;
\n var msgs=[]; \/\/ recorded nodes
\n var dialectLabel={Chat:”Chat”,Resp:”Resp”,Msg:”Msg”};<\/p>\n

function pos(el){ \/\/ center x relative to flow
\n var f=flow.getBoundingClientRect();
\n var r=el.getBoundingClientRect();
\n return (r.left – f.left) + r.width\/2 – 32;
\n }
\n function clearActive(){ [nHarness,nIntercept,nInfer].forEach(function(n){n.classList.remove(“vf-active”);}); }<\/p>\n

function movePacket(fromEl,toEl,ms,label,isResp){
\n return new Promise(function(res){
\n packet.textContent=label;
\n packet.classList.toggle(“vf-resp”,!!isResp);
\n packet.style.transition=”none”;
\n packet.style.left=pos(fromEl)+”px”;
\n packet.style.opacity=”1″;
\n void packet.offsetWidth;
\n packet.style.transition=”left “+ms+”ms cubic-bezier(.45,.05,.35,1)”;
\n packet.style.left=pos(toEl)+”px”;
\n setTimeout(res,ms);
\n });
\n }<\/p>\n

function addNode(role,label,color){
\n if(msgs.length===0){ graph.innerHTML=””; }
\n var d=document.createElement(“div”);
\n d.className=”vf-msg”;
\n d.innerHTML='<span class="”vf-dot”" style="”background:’+color+'”"><\/span><code>’+label+'<\/code><span class="”vf-role”">’+role+'<\/span>’;
\n graph.appendChild(d);
\n graph.scrollTop=graph.scrollHeight;
\n msgs.push(label);
\n }<\/p>\n

function sleep(ms){return new Promise(function(r){setTimeout(r,ms);});}<\/p>\n

async function runTurn(){
\n if(running) return;
\n running=true; runBtn.disabled=true;
\n turn++;
\n var dl=dialectLabel[dialectSel.value];<\/p>\n

\/\/ seed system + user on first turn
\n if(turn===1){
\n addNode(“system”,”S1″,”#6366f1″); await sleep(160);
\n addNode(“user”,”U1″,”#6366f1″);
\n }<\/p>\n

clearActive();
\n nHarness.classList.add(“vf-active”);
\n status.textContent=”Harness builds a “+dl+” request\u2026”;
\n await sleep(350);<\/p>\n

\/\/ harness -> interception
\n nIntercept.classList.add(“vf-active”);
\n status.textContent=”Interception server proxies the request \u2192 inference.”;
\n await movePacket(nHarness,nInfer,850,dl+” req”);
\n clearActive(); nInfer.classList.add(“vf-active”);
\n status.textContent=”Inference server generates the reply (vLLM).”;
\n await sleep(350);<\/p>\n

\/\/ inference -> interception (records) -> harness
\n nIntercept.classList.add(“vf-active”);
\n status.textContent=”Interception server records the trace, relays the response.”;
\n await movePacket(nInfer,nHarness,850,”resp”,true);
\n packet.style.opacity=”0″;
\n clearActive();<\/p>\n

\/\/ record assistant node (+ occasional tool)
\n addNode(“assistant”,”A”+turn,”#0b8f8f”); await sleep(150);
\n if(turn%2===0){ addNode(“tool”,”T”+turn,”#e0a800″); }<\/p>\n

status.textContent=”Turn “+turn+” recorded as a unique node in the message graph.”;
\n running=false; runBtn.disabled=false;
\n }<\/p>\n

function reset(){
\n turn=0; msgs=[]; running=false; runBtn.disabled=false;
\n clearActive(); packet.style.opacity=”0″;
\n graph.innerHTML='<div class="”vf-empty”">No messages recorded yet.<\/div>’;
\n status.textContent=”Press \u201cRun rollout\u201d to send a request through the interception server.”;
\n }<\/p>\n

runBtn.addEventListener(“click”,runTurn);
\n resetBtn.addEventListener(“click”,reset);<\/p>\n

\/\/ —- v0 vs v1 growth chart —-
\n var v0=document.getElementById(“vf-v0”);
\n var v1=document.getElementById(“vf-v1”);
\n var turnsR=document.getElementById(“vf-turns”);
\n var turnsV=document.getElementById(“vf-turns-val”);<\/p>\n

function drawChart(N){
\n var x0=30,x1=255,y0=130,y1=12,W=x1-x0,H=y0-y1;
\n var maxV0=N*N; \/\/ quadratic reference
\n function ptV0(i){var x=x0+(i\/N)*W;var y=y0-((i*i)\/maxV0)*H;return x+”,”+y;}
\n function ptV1(i){var x=x0+(i\/N)*W;var y=y0-((i\/N)*H);return x+”,”+y;} \/\/ linear
\n var p0=”M”,p1=”M”;
\n for(var i=0;i<=N;i++){ p0+=(i?” L”:””)+ptV0(i); p1+=(i?” L”:””)+ptV1(i); }
\n v0.setAttribute(“d”,p0); v1.setAttribute(“d”,p1);
\n }
\n turnsR.addEventListener(“input”,function(){ turnsV.textContent=turnsR.value; drawChart(+turnsR.value); });
\n drawChart(+turnsR.value);<\/p>\n

\/\/ —- auto-resize for WordPress iframe embedding —-
\n function sendHeight(){
\n var h=document.getElementById(“vfv1-demo”).offsetHeight+40;
\n if(window.parent){ window.parent.postMessage({vfv1Height:h},”*”); }
\n }
\n window.addEventListener(“load”,sendHeight);
\n window.addEventListener(“resize”,sendHeight);
\n new MutationObserver(sendHeight).observe(document.getElementById(“vf-graph”),{childList:true});
\n setInterval(sendHeight,1200);
\n})();
\n<\/script>
\n<\/body>
\n<\/html>
\n“><\/p>\n

\n

v0 vs v1: A Quick Comparison<\/strong><\/h2>\n<\/p>

These changes separate v1 from v0.<\/p>\n

\n\n\n\n\n\n\n\n\n\n\n
Aspect<\/th>\nverifiers v0<\/th>\nverifiers v1<\/th>\n<\/tr>\n<\/thead>\n
Environment model<\/td>\nData, logic, and infra bundled together<\/td>\nSplit into taskset, harness, runtime<\/td>\n<\/tr>\n
Trace growth<\/td>\nQuadratic in turns (repeated pairs)<\/td>\nLinear in turns (unique nodes)<\/td>\n<\/tr>\n
Non-linear rollouts<\/td>\nAssumed linear<\/td>\nNative compaction and subagents via branches<\/td>\n<\/tr>\n
Runtime handling<\/td>\nBuilder manages lifecycle<\/td>\nFramework-managed run<\/code> \/ read<\/code> \/ write<\/code><\/td>\n<\/tr>\n
Harness coupling<\/td>\nTightly coupled to the environment<\/td>\nAny compatible harness (Codex, Terminus 2)<\/td>\n<\/tr>\n
Training data<\/td>\nRecomputed for prime-rl<\/td>\nConsumed directly from the trace<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n

Use Cases with Examples<\/strong><\/h2>\n

With the architecture clear, consider how teams use it. For example, you can run Nemotron 3 Ultra on Terminal-Bench 2 under Codex.<\/p>\n

Similarly, teams can reuse Harbor<\/strong> datasets without rewriting reward logic. Prime Intellect ported Terminal Bench 2 into v1 with only a small class. In its internal testing, verifiers matched Harbor\u2019s performance on the same tasks. Harbor is the first fully-supported third-party format; NeMo Gym and OpenEnv have alpha support.<\/p>\n

On the training side, the same environments plug into prime-rl directly. In a length-penalty ablation, GLM-4.5-Air trained on ScaleSWE across six H200 nodes. That run took two days and evaluated on SWE-Bench-Verified, showing stable agentic training.<\/p>\n

A Minimal Taskset and Launch<\/strong><\/h2>\n

Each run starts from a taskset that defines data and scoring, independent of any harness:<\/p>\n

\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
import verifiers.v1 as vf\n\nclass AdditionData(vf.TaskData):\n    answer: int\n\nclass AdditionTask(vf.Task[AdditionData]):\n    @vf.reward\n    async def exact_match(self, trace: vf.Trace) -> float:\n        return float(trace.last_reply == str(self.data.answer))\n\nclass AdditionTaskset(vf.Taskset[AdditionTask, vf.TasksetConfig]):\n    def load(self) -> list[AdditionTask]:\n        return [\n            AdditionTask(\n                AdditionData(idx=i, prompt=f\"What is {i} + {i}?\", answer=2 * i),\n                self.config.task,\n            )\n            for i in range(100)\n        ]\n\n__all__ = [\"AdditionTaskset\"]<\/code><\/pre>\n<\/div>\n<\/div>\n
Any taskset then runs under a chosen harness via TOML and the CLI:<\/p>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
model = \"nvidia\/NVIDIA-Nemotron-3-Ultra-550B-A55B\"\n\n[taskset]\nid = \"primeintellect\/terminal-bench-2\"\n\n[harness]\nid = \"codex\"\nversion = \"0.116.0\"<\/code><\/pre>\n<\/div>\n<\/div>\n
\n
\n
\n
\n
<\/div>\n
<\/div>\n
<\/div>\n<\/div>\n
Copy Code<\/span>Copied<\/span>Use a different Browser<\/span><\/a><\/div>\n<\/div>\n
uv run eval @ path\/to\/config.toml<\/code><\/pre>\n<\/div>\n<\/div>\n
Key Takeaways<\/strong><\/h2>\n