{"id":108384,"date":"2026-08-01T19:59:42","date_gmt":"2026-08-01T19:59:42","guid":{"rendered":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/"},"modified":"2026-08-01T19:59:42","modified_gmt":"2026-08-01T19:59:42","slug":"supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks","status":"publish","type":"post","link":"https:\/\/youzum.net\/fr\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/","title":{"rendered":"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks"},"content":{"rendered":"<p class=\"wp-block-paragraph\">Supabase has <a href=\"https:\/\/supabase.com\/blog\/introducing-supabase-evals\">open sourced Supabase Evals<\/a>, its benchmark and framework for testing how well AI agents build using Supabase. It runs coding agents including <a href=\"https:\/\/www.anthropic.com\/claude-code\">Claude Code<\/a>, Codex, and <a href=\"https:\/\/opencode.ai\/\">OpenCode<\/a> against real tasks, such as building a schema, debugging a failed <a href=\"https:\/\/supabase.com\/edge-functions\">Edge Function<\/a>, or fixing a broken RLS policy, then scores the result. It powers the public leaderboard at <a href=\"https:\/\/supabase.com\/evals\">supabase.com\/evals<\/a> and an internal regression suite monitored daily.<\/p>\n<h2 class=\"wp-block-heading\"><strong>Is it deployable?<\/strong><\/h2>\n<p class=\"wp-block-paragraph\">Yes, today. <a href=\"https:\/\/github.com\/supabase\/evals\"><code>supabase\/evals<\/code><\/a> is public under Apache-2.0 and runs locally via <code>pnpm<\/code>.<\/p>\n<ul class=\"wp-block-list\">\n<li><strong>Industries<\/strong>: Developer tooling, cloud infrastructure, data platforms, and regulated backends in fintech or healthcare, where an agent writing a wrong <a href=\"https:\/\/supabase.com\/docs\/guides\/database\/postgres\/row-level-security\">RLS<\/a> policy is a security incident.<\/li>\n<li><strong>Applications<\/strong>: Regression-testing docs and skill edits, gating SDK releases, and comparing agent harnesses head to head.<\/li>\n<li><strong>Constraints<\/strong>: Local-stack runs need a Docker daemon, provider API keys, and ports 54321\u201354329 free.<\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\"><strong>How the harness works<\/strong><\/h2>\n<p class=\"wp-block-paragraph\">Supabase defined three dimensions: <strong>products<\/strong> (database, auth, storage, edge-functions, realtime, cron, queues, vectors, data-api), <strong>topics<\/strong> (RLS, security, migrations, SQL, SDK, observability, self-hosting, tests, declarative-schema), and <strong>stages<\/strong> (build, deploy, investigate, resolve). It then picked the smallest scenario set touching each dimension once, grounded in support tickets, bug reports, and GitHub issues.<\/p>\n<p class=\"wp-block-paragraph\">Scenarios split into two suites. <strong>Benchmark<\/strong> scenarios cover breadth and are published. <strong>Regression<\/strong> scenarios cover known failure modes, refresh daily, and do not move published scores.<\/p>\n<p class=\"wp-block-paragraph\">Every scenario runs against a real environment. The framework boots a hosted-like stack and a local CLI project in containers, so agents call the actual <a href=\"https:\/\/supabase.com\/docs\/guides\/getting-started\/mcp\">MCP server<\/a> and CLI. A <code>platform-lite<\/code> runtime exposes a Management API-compatible surface backed by <a href=\"https:\/\/github.com\/supabase\/supabase-lite\"><code>@supabase\/lite<\/code><\/a>. Scoring combines deterministic checks with LLM-as-a-judge. Agents get one retry before grading.<\/p>\n<p class=\"wp-block-paragraph\">Each eval directory holds <code>PROMPT.md<\/code> (task plus frontmatter), <code>EVAL.ts<\/code> (the scorer), and optional <code>remote\/<\/code> and <code>local\/<\/code> starting states. Shipping a <code>local\/<\/code> workspace, or declaring <code>interface: cli<\/code>, boots a Docker sandbox with the real CLI installed.<\/p>\n<p><!-- Marktechpost interactive embed: Supabase Evals explainer. Paste into a WordPress Custom HTML block. --><br \/>\n Run the pipeline&lt;\/button&gt;<br \/>\n &lt;\/div&gt;<\/p>\n<p> &lt;!&#8211; ANATOMY &#8211;&gt;<br \/>\n &lt;div class=&quot;&rdquo;pane&rdquo;&quot; id=&quot;&rdquo;p2&Prime;&quot;&gt;<br \/>\n  &lt;div class=&quot;&rdquo;hint&rdquo;&quot;&gt;Every eval lives at &lt;b&gt;evals\/&amp;lt;id&amp;gt;\/&lt;\/b&gt;. Click a file to see what it holds.&lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;tree&rdquo;&quot; id=&quot;&rdquo;tree&rdquo;&quot;&gt;&lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;detail&rdquo;&quot; id=&quot;&rdquo;det2&Prime;&quot;&gt;&lt;\/div&gt;<br \/>\n &lt;\/div&gt;<\/p>\n<p> &lt;!&#8211; RUNTIMES &#8211;&gt;<br \/>\n &lt;div class=&quot;&rdquo;pane&rdquo;&quot; id=&quot;&rdquo;p3&Prime;&quot;&gt;<br \/>\n  &lt;div class=&quot;&rdquo;hint&rdquo;&quot;&gt;The harness picks a runtime &lt;b&gt;automatically&lt;\/b&gt;, per eval. Toggle to compare.&lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;tog&rdquo;&quot;&gt;<br \/>\n   &lt;button class=&#8221;tg on&#8221; data-r=&#8221;0&#8243;&gt;Tools evals&lt;\/button&gt;<br \/>\n   &lt;button class=&#8221;tg&#8221; data-r=&#8221;1&#8243;&gt;Local-stack evals&lt;\/button&gt;<br \/>\n  &lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;lanes&rdquo;&quot; id=&quot;&rdquo;lanes&rdquo;&quot;&gt;&lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;note&rdquo;&quot; id=&quot;&rdquo;rnote&rdquo;&quot;&gt;&lt;\/div&gt;<br \/>\n &lt;\/div&gt;<\/p>\n<p> &lt;!&#8211; FINDINGS &#8211;&gt;<br \/>\n &lt;div class=&quot;&rdquo;pane&rdquo;&quot; id=&quot;&rdquo;p4&Prime;&quot;&gt;<br \/>\n  &lt;div class=&quot;&rdquo;hint&rdquo;&quot;&gt;Published &lt;b&gt;Build stage&lt;\/b&gt; pass rates. Toggle the Supabase agent skill on and off.&lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;tog&rdquo;&quot;&gt;<br \/>\n   &lt;button class=&#8221;tg&#8221; data-s=&#8221;0&#8243;&gt;No skill loaded&lt;\/button&gt;<br \/>\n   &lt;button class=&#8221;tg on&#8221; data-s=&#8221;1&#8243;&gt;Skill loaded&lt;\/button&gt;<br \/>\n  &lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;rows&rdquo;&quot; id=&quot;&rdquo;rows&rdquo;&quot;&gt;&lt;\/div&gt;<br \/>\n  &lt;div class=&quot;&rdquo;note&rdquo;&quot;&gt;<br \/>\n   &lt;b&gt;Also measured:&lt;\/b&gt; Codex \/ GPT-5.6 reads about 8 docs pages per scenario, versus roughly 2 for Claude Code, which checks the docs in under 40% of scenarios even with skills loaded. Rewriting the Postgres best-practices skill description lifted its activation from about 1 in 10 sessions to 60%.&lt;br&gt;&lt;br&gt;<br \/>\n   Figures are a snapshot from Supabase&#8217;s launch post (31 Jul 2026). Results move as models change \u2014 check the live page.<br \/>\n  &lt;\/div&gt;<br \/>\n &lt;\/div&gt;<\/p>\n<p> &lt;div class=&quot;&rdquo;ft&rdquo;&quot;&gt;<br \/>\n  &lt;span&gt;Source: &lt;a href=&quot;\/fr\/&rdquo;https:\/\/supabase.com\/blog\/introducing-supabase-evals&rdquo;\/&quot; target=&quot;&rdquo;_blank&rdquo;&quot; rel=&quot;&rdquo;noopener&rdquo;&quot;&gt;Supabase blog&lt;\/a&gt; &middot; &lt;a href=&quot;\/fr\/&rdquo;https:\/\/github.com\/supabase\/evals&rdquo;\/&quot; target=&quot;&rdquo;_blank&rdquo;&quot; rel=&quot;&rdquo;noopener&rdquo;&quot;&gt;supabase\/evals&lt;\/a&gt; &middot; Apache-2.0&lt;\/span&gt;<br \/>\n  &lt;span&gt;&lt;b&gt;Marktechpost&lt;\/b&gt;&lt;\/span&gt;<br \/>\n &lt;\/div&gt;<br \/>\n&lt;\/div&gt;<\/p>\n<p>&lt;script&gt;<br \/>\nvar STAGES=[<br \/>\n {i:&#8221;<img decoding=\"async\" src=\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png\" alt=\"\ud83d\udcdd\" class=\"wp-smiley\" \/>&#8220;,l:&#8221;Scenario&#8221;,h:&#8221;1 \u00b7 A real scenario&#8221;,t:&#8221;Each eval is grounded in a real problem \u2014 a support ticket, bug report, or GitHub issue. &lt;code&gt;PROMPT.md&lt;\/code&gt; carries the task the agent sees plus frontmatter tagging its stage, product, and topic.&#8221;},<br \/>\n {i:&#8221;<img decoding=\"async\" src=\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f433.png\" alt=\"\ud83d\udc33\" class=\"wp-smiley\" \/>&#8220;,l:&#8221;Environments&#8221;,h:&#8221;2 \u00b7 Two real environments&#8221;,t:&#8221;The framework boots a hosted-like Supabase stack and a local CLI project in containers. &lt;code&gt;platform-lite&lt;\/code&gt; serves a Management API-compatible surface backed by &lt;code&gt;@supabase\/lite&lt;\/code&gt;.&#8221;},<br \/>\n {i:&#8221;<img decoding=\"async\" src=\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f916.png\" alt=\"\ud83e\udd16\" class=\"wp-smiley\" \/>&#8220;,l:&#8221;Agent runs&#8221;,h:&#8221;3 \u00b7 The agent works&#8221;,t:&#8221;Claude Code, Codex, OpenCode, or an AI SDK agent invokes the real Supabase MCP server and CLI \u2014 not mocks. Skills load lazily: only name and description sit in the system prompt.&#8221;},<br \/>\n {i:&#8221;<img decoding=\"async\" src=\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f501.png\" alt=\"\ud83d\udd01\" class=\"wp-smiley\" \/>&#8220;,l:&#8221;One retry&#8221;,h:&#8221;4 \u00b7 One retry allowed&#8221;,t:&#8221;To cut false negatives while keeping runs sustainable, agents may retry once after a failure before they are graded.&#8221;},<br \/>\n {i:&#8221;<img decoding=\"async\" src=\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/2696.png\" alt=\"\u2696\" class=\"wp-smiley\" \/>&#8220;,l:&#8221;Scoring&#8221;,h:&#8221;5 \u00b7 Deterministic + judge&#8221;,t:&#8221;&lt;code&gt;EVAL.ts&lt;\/code&gt; exports the scorer. Deterministic checks confirm things like whether a user can reach certain data or an Edge Function returns the expected result; an LLM judge handles semantic calls.&#8221;},<br \/>\n {i:&#8221;<img decoding=\"async\" src=\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4ca.png\" alt=\"\ud83d\udcca\" class=\"wp-smiley\" \/>&#8220;,l:&#8221;Results&#8221;,h:&#8221;6 \u00b7 Benchmark or regression&#8221;,t:&#8221;Benchmark scenarios go to the public site and run when assessing new changes or harnesses. Regression scenarios track known failure modes and refresh daily, without moving published scores.&#8221;}<br \/>\n];<br \/>\nvar FILES=[<br \/>\n {n:&#8221;PROMPT.md&#8221;,d:&#8221;Task + frontmatter&#8221;,h:&#8221;PROMPT.md&#8221;,t:&#8221;Frontmatter plus the task description the agent sees. Keys drive discovery and the site filters: &lt;code&gt;stage&lt;\/code&gt;, &lt;code&gt;suite&lt;\/code&gt;, &lt;code&gt;product&lt;\/code&gt;, &lt;code&gt;topic&lt;\/code&gt;, &lt;code&gt;motivation&lt;\/code&gt;. &lt;code&gt;suite&lt;\/code&gt; is required on every eval.&#8221;},<br \/>\n {n:&#8221;EVAL.ts&#8221;,d:&#8221;The scorer&#8221;,h:&#8221;EVAL.ts&#8221;,t:&#8221;A default-exported scorer. Scorers check what the agent produced, never what the harness provisioned \u2014 with &lt;code&gt;projectRunning: true&lt;\/code&gt;, only the agent&#8217;s deltas are scored.&#8221;},<br \/>\n {n:&#8221;remote\/&#8221;,d:&#8221;Hosted project state&#8221;,h:&#8221;remote\/ \u2014 optional&#8221;,t:&#8221;The hosted project&#8217;s starting state, seeded into platform-lite: &lt;code&gt;project.sql&lt;\/code&gt; for the database, &lt;code&gt;logs.jsonl&lt;\/code&gt; for observability logs, and &lt;code&gt;functions\/&lt;\/code&gt; for already-deployed Edge Functions.&#8221;},<br \/>\n {n:&#8221;local\/&#8221;,d:&#8221;Agent workspace&#8221;,h:&#8221;local\/ \u2014 optional&#8221;,t:&#8221;The developer&#8217;s working directory, copied into the sandbox before the agent starts. Its presence is also a runtime switch: ship a &lt;code&gt;local\/&lt;\/code&gt; and the eval boots a Docker sandbox.&#8221;}<br \/>\n];<br \/>\nvar RUN=[<br \/>\n {lanes:[[&#8220;PROMPT&#8221;,&#8221;Agent gets the task, with no local\/ directory and no interface: cli&#8221;],[&#8220;TOOLS&#8221;,&#8221;It works through the experiment&#8217;s MCP \/ tool surface only \u2014 there is no filesystem&#8221;],[&#8220;SKILLS&#8221;,&#8221;A load_skill tool returns a skill&#8217;s full instructions on demand&#8221;],[&#8220;SCORE&#8221;,&#8221;The resulting project state or report is graded&#8221;]],<br \/>\n  note:&rdquo;&lt;b&gt;Tools evals&lt;\/b&gt; exercise the MCP surface in isolation. Because the agent has no filesystem, skills are fetched through a tool call rather than read from disk.&rdquo;},<br \/>\n {lanes:[[&#8220;PROMPT&#8221;,&#8221;Eval ships a local\/ workspace or declares interface: cli&#8221;],[&#8220;SANDBOX&#8221;,&#8221;A fresh Docker container boots per attempt, with the real Supabase CLI installed&#8221;],[&#8220;STACK&#8221;,&#8221;The agent runs supabase init \/ start \/ db \/ test against a live local stack&#8221;],[&#8220;EXPORT&#8221;,&#8221;The workspace is copied back to the host so scorers run vite \/ vitest against it&#8221;]],<br \/>\n  note:&rdquo;&lt;b&gt;Local-stack evals&lt;\/b&gt; need a running Docker daemon, and default ports 54321&ndash;54329 free. A &lt;code&gt;services:&lt;\/code&gt; list keeps stack boots fast by starting only what the scenario needs.&rdquo;}<br \/>\n];<br \/>\nvar SCORES=[<br \/>\n {n:&#8221;Opus 5&#8243;,off:100,on:100},<br \/>\n {n:&#8221;Kimi K3&#8243;,off:100,on:100},<br \/>\n {n:&#8221;GPT-5.6 Sol&#8221;,off:89,on:100},<br \/>\n {n:&#8221;Sonnet 5&#8243;,off:78,on:100},<br \/>\n {n:&#8221;GPT-5.4 mini&#8221;,off:78,on:89}<br \/>\n];<\/p>\n<p>function $(s){return document.querySelector(s)}<br \/>\nfunction all(s){return [].slice.call(document.querySelectorAll(s))}<\/p>\n<p>\/* tabs *\/<br \/>\nall(&#8216;.tab&#8217;).forEach(function(b){b.onclick=function(){<br \/>\n all(&#8216;.tab&#8217;).forEach(function(x){x.classList.remove(&#8216;on&#8217;)});<br \/>\n all(&#8216;.pane&#8217;).forEach(function(x){x.classList.remove(&#8216;on&#8217;)});<br \/>\n b.classList.add(&#8216;on&#8217;);$(&#8216;#&#8217;+b.dataset.p).classList.add(&#8216;on&#8217;);<br \/>\n}});<\/p>\n<p>\/* pipeline *\/<br \/>\nvar flow=$(&#8216;#flow&#8217;);<br \/>\nSTAGES.forEach(function(s,i){<br \/>\n var d=document.createElement(&#8216;div&#8217;);d.className=&#8217;node&#8217;;d.dataset.i=i;<br \/>\n d.innerHTML=&#039;&lt;span class=&quot;&rdquo;dot&rdquo;&quot;&gt;&lt;\/span&gt;&lt;div class=&quot;&rdquo;nnum&rdquo;&quot;&gt;0&rsquo;+(i+1)+&#039;&lt;\/div&gt;&lt;div class=&quot;&rdquo;nico&rdquo;&quot;&gt;&rsquo;+s.i+&#039;&lt;\/div&gt;&lt;div class=&quot;&rdquo;nlab&rdquo;&quot;&gt;&rsquo;+s.l+&#039;&lt;\/div&gt;&rsquo;;<br \/>\n d.onclick=function(){pick(i)};flow.appendChild(d);<br \/>\n});<br \/>\nfunction pick(i){<br \/>\n all(&#8216;#flow .node&#8217;).forEach(function(n,j){n.classList.toggle(&#8216;on&#8217;,j===i)});<br \/>\n $(&lsquo;#det&rsquo;).innerHTML=&#039;&lt;h4&gt;&rsquo;+STAGES[i].h+&#039;&lt;\/h4&gt;&lt;p&gt;&rsquo;+STAGES[i].t+&#039;&lt;\/p&gt;&rsquo;;<br \/>\n}<br \/>\npick(0);<br \/>\nvar busy=false;<br \/>\n$(&#8216;#run&#8217;).onclick=function(){<br \/>\n if(busy)return;busy=true;this.disabled=true;<br \/>\n var nodes=all(&#8216;#flow .node&#8217;);nodes.forEach(function(n){n.classList.remove(&#8216;lit&#8217;)});<br \/>\n var i=0;var t=setInterval(function(){<br \/>\n  if(i&gt;0)nodes[i-1].classList.remove(&#8216;lit&#8217;);<br \/>\n  if(i&gt;=nodes.length){clearInterval(t);busy=false;$(&#8216;#run&#8217;).disabled=false;return}<br \/>\n  nodes[i].classList.add(&#8216;lit&#8217;);pick(i);i++;<br \/>\n },900);<br \/>\n};<\/p>\n<p>\/* anatomy *\/<br \/>\nvar tree=$(&#8216;#tree&#8217;);<br \/>\nFILES.forEach(function(f,i){<br \/>\n var d=document.createElement(&#8216;div&#8217;);d.className=&#8217;f&#8217;;<br \/>\n d.innerHTML=&#039;&lt;div class=&quot;&rdquo;fn&rdquo;&quot;&gt;&rsquo;+f.n+&#039;&lt;\/div&gt;&lt;div class=&quot;&rdquo;fd&rdquo;&quot;&gt;&rsquo;+f.d+&#039;&lt;\/div&gt;&rsquo;;<br \/>\n d.onclick=function(){<br \/>\n  all(&#8216;#tree .f&#8217;).forEach(function(x,j){x.classList.toggle(&#8216;on&#8217;,j===i)});<br \/>\n  $(&lsquo;#det2&prime;).innerHTML=&#039;&lt;h4&gt;&rsquo;+f.h+&#039;&lt;\/h4&gt;&lt;p&gt;&rsquo;+f.t+&#039;&lt;\/p&gt;&rsquo;;<br \/>\n };<br \/>\n tree.appendChild(d);<br \/>\n});<br \/>\nall(&#8216;#tree .f&#8217;)[0].click();<\/p>\n<p>\/* runtimes *\/<br \/>\nfunction drawRun(k){<br \/>\n var L=$(&#8216;#lanes&#8217;);L.innerHTML=&#8221;;<br \/>\n RUN[k].lanes.forEach(function(x,i){<br \/>\n  var d=document.createElement(&#8216;div&#8217;);d.className=&#8217;lane&#8217;;<br \/>\n  d.innerHTML=&#039;&lt;span class=&quot;&rdquo;lb&rdquo;&quot;&gt;&rsquo;+x[0]+&#039;&lt;\/span&gt;&lt;span class=&quot;&rdquo;lt&rdquo;&quot;&gt;&rsquo;+x[1]+&#039;&lt;\/span&gt;&rsquo;;<br \/>\n  L.appendChild(d);setTimeout(function(){d.classList.add(&#8216;act&#8217;)},120*i+60);<br \/>\n });<br \/>\n $(&#8216;#rnote&#8217;).innerHTML=RUN[k].note;<br \/>\n}<br \/>\nall(&#8216;[data-r]&#8217;).forEach(function(b){b.onclick=function(){<br \/>\n all(&#8216;[data-r]&#8217;).forEach(function(x){x.classList.remove(&#8216;on&#8217;)});<br \/>\n b.classList.add(&#8216;on&#8217;);drawRun(+b.dataset.r);<br \/>\n}});<br \/>\ndrawRun(0);<\/p>\n<p>\/* findings *\/<br \/>\nvar rows=$(&#8216;#rows&#8217;);<br \/>\nSCORES.forEach(function(s){<br \/>\n var d=document.createElement(&#8216;div&#8217;);d.className=&#8217;row&#8217;;<br \/>\n d.innerHTML=&#039;&lt;div class=&quot;&rdquo;rn&rdquo;&quot;&gt;&rsquo;+s.n+&#039;&lt;\/div&gt;&lt;div class=&quot;&rdquo;track&rdquo;&quot;&gt;&lt;div class=&quot;&rdquo;bar&rdquo;&quot;&gt;&lt;\/div&gt;&lt;\/div&gt;&lt;div class=&quot;&rdquo;rv&rdquo;&quot;&gt;&mdash;&lt;\/div&gt;&rsquo;;<br \/>\n rows.appendChild(d);<br \/>\n});<br \/>\nfunction drawScores(on){<br \/>\n all(&#8216;#rows .row&#8217;).forEach(function(r,i){<br \/>\n  var v=on?SCORES[i].on:SCORES[i].off;<br \/>\n  r.querySelector(&#8216;.bar&#8217;).style.width=v+&#8217;%&#8217;;<br \/>\n  r.querySelector(&#8216;.rv&#8217;).textContent=v+&#8217;%&#8217;;<br \/>\n });<br \/>\n}<br \/>\nall(&#8216;[data-s]&#8217;).forEach(function(b){b.onclick=function(){<br \/>\n all(&#8216;[data-s]&#8217;).forEach(function(x){x.classList.remove(&#8216;on&#8217;)});<br \/>\n b.classList.add(&#8216;on&#8217;);drawScores(b.dataset.s===&#8217;1&#8242;);<br \/>\n}});<br \/>\ndrawScores(true);<\/p>\n<p>\/* auto-resize *\/<br \/>\nfunction ping(){<br \/>\n try{parent.postMessage({mtpEmbed:&#8217;supabase-evals&#8217;,height:document.body.offsetHeight+40},&#8217;*&#8217;)}catch(e){}<br \/>\n}<br \/>\nwindow.addEventListener(&#8216;load&#8217;,ping);<br \/>\nsetInterval(ping,400);<br \/>\ndocument.addEventListener(&#8216;click&#8217;,function(){setTimeout(ping,60)});<br \/>\n&lt;\/script&gt;<br \/>\n&#8220;&gt;<\/p>\n<p class=\"wp-block-paragraph\">\n<h2 class=\"wp-block-heading\"><strong>Findings<\/strong><\/h2>\n<\/p><p class=\"wp-block-paragraph\">Agents pass most scenarios with no skill loaded. In the Build stage, Opus 5 and Kimi K3 both scored 100% unaided. Skills closed the rest of the gap: Sonnet 5 rose from 78% to 100%, GPT-5.6 Sol from 89% to 100%, and GPT-5.4 mini from 78% to 89%.<\/p>\n<p class=\"wp-block-paragraph\">Three weaknesses surfaced. Agents hand-write migrations instead of using <a href=\"https:\/\/supabase.com\/docs\/guides\/local-development\/declarative-database-schemas\">declarative schemas<\/a>, prompting a <a href=\"https:\/\/github.com\/supabase\/agent-skills\/pull\/120\">skill guidance update<\/a>. Agents verify auth by hand rather than reaching for <a href=\"https:\/\/supabase.com\/blog\/introducing-supabase-server\"><code>@supabase\/server<\/code><\/a>, prompting a <a href=\"https:\/\/supabase.com\/docs\/guides\/auth\/choosing-a-server-package\">package selection guide<\/a>. And docs usage varies sharply: Codex \/ GPT-5.6 reads roughly 8 docs pages per scenario versus about 2 for Claude Code, which checks docs in under 40% of scenarios even with skills loaded.<\/p>\n<h2 class=\"wp-block-heading\"><strong>Key Takeaways<\/strong><\/h2>\n<ul class=\"wp-block-list\">\n<li>Supabase open sourced <code>supabase\/evals<\/code> under Apache-2.0.<\/li>\n<li>Scenarios run against real containerized Supabase stacks, not mocks.<\/li>\n<li>Scoring mixes deterministic checks with LLM-as-a-judge; one retry allowed.<\/li>\n<li>Skills mattered least for top models, most for smaller ones.<\/li>\n<\/ul>\n<p class=\"wp-block-paragraph\">\n<hr class=\"wp-block-separator has-alpha-channel-opacity\" \/>\n<\/p><p class=\"wp-block-paragraph\">\n<\/p><p class=\"wp-block-paragraph\">Check out the<strong>\u00a0<a href=\"https:\/\/supabase.com\/blog\/introducing-supabase-evals\" target=\"_blank\" rel=\"noreferrer noopener\">Technical details<\/a> <\/strong>and <strong><a href=\"https:\/\/github.com\/supabase\/evals\" target=\"_blank\" rel=\"noreferrer noopener\">GitHub Repo<\/a>.\u00a0<\/strong>Also,\u00a0feel free to follow us on\u00a0<strong><a href=\"https:\/\/x.com\/intent\/follow?screen_name=marktechpost\" target=\"_blank\" rel=\"noreferrer noopener\"><mark>Twitter<\/mark><\/a><\/strong>\u00a0and don\u2019t forget to join our\u00a0<strong><a href=\"https:\/\/www.reddit.com\/r\/machinelearningnews\/\" target=\"_blank\" rel=\"noreferrer noopener\">150k+ML SubReddit<\/a><\/strong>\u00a0and Subscribe to\u00a0<strong><a href=\"https:\/\/www.aidevsignals.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">our Newsletter<\/a><\/strong>. Wait! are you on telegram?\u00a0<strong><a href=\"https:\/\/t.me\/machinelearningresearchnews\" target=\"_blank\" rel=\"noreferrer noopener\">now you can join us on telegram as well.<\/a><\/strong><\/p>\n<p class=\"wp-block-paragraph\">Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.?\u00a0<strong><a href=\"https:\/\/forms.gle\/wbash1wF6efRj8G58\" target=\"_blank\" rel=\"noreferrer noopener\"><mark>Connect with us<\/mark><\/a><\/strong><\/p>\n<p>The post <a href=\"https:\/\/www.marktechpost.com\/2026\/08\/01\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\">Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks<\/a> appeared first on <a href=\"https:\/\/www.marktechpost.com\/\">MarkTechPost<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"<p>Supabase has open sourced Supabase Evals, its benchmark and framework for testing how well AI agents build using Supabase. It runs coding agents including Claude Code, Codex, and OpenCode against real tasks, such as building a schema, debugging a failed Edge Function, or fixing a broken RLS policy, then scores the result. It powers the public leaderboard at supabase.com\/evals and an internal regression suite monitored daily. Is it deployable? Yes, today. supabase\/evals is public under Apache-2.0 and runs locally via pnpm. Industries: Developer tooling, cloud infrastructure, data platforms, and regulated backends in fintech or healthcare, where an agent writing a wrong RLS policy is a security incident. Applications: Regression-testing docs and skill edits, gating SDK releases, and comparing agent harnesses head to head. Constraints: Local-stack runs need a Docker daemon, provider API keys, and ports 54321&ndash;54329 free. How the harness works Supabase defined three dimensions: products (database, auth, storage, edge-functions, realtime, cron, queues, vectors, data-api), topics (RLS, security, migrations, SQL, SDK, observability, self-hosting, tests, declarative-schema), and stages (build, deploy, investigate, resolve). It then picked the smallest scenario set touching each dimension once, grounded in support tickets, bug reports, and GitHub issues. Scenarios split into two suites. Benchmark scenarios cover breadth and are published. Regression scenarios cover known failure modes, refresh daily, and do not move published scores. Every scenario runs against a real environment. The framework boots a hosted-like stack and a local CLI project in containers, so agents call the actual MCP server and CLI. A platform-lite runtime exposes a Management API-compatible surface backed by @supabase\/lite. Scoring combines deterministic checks with LLM-as-a-judge. Agents get one retry before grading. Each eval directory holds PROMPT.md (task plus frontmatter), EVAL.ts (the scorer), and optional remote\/ and local\/ starting states. Shipping a local\/ workspace, or declaring interface: cli, boots a Docker sandbox with the real CLI installed. Run the pipeline&lt;\/button&gt; &lt;\/div&gt; &lt;!&ndash; ANATOMY &ndash;&gt; &lt;div class=&quot;&rdquo;pane&rdquo;&quot; id=&quot;&rdquo;p2&Prime;&quot;&gt; &lt;div class=&quot;&rdquo;hint&rdquo;&quot;&gt;Every eval lives at &lt;b&gt;evals\/&amp;lt;id&amp;gt;\/&lt;\/b&gt;. Click a file to see what it holds.&lt;\/div&gt; &lt;div class=&quot;&rdquo;tree&rdquo;&quot; id=&quot;&rdquo;tree&rdquo;&quot;&gt;&lt;\/div&gt; &lt;div class=&quot;&rdquo;detail&rdquo;&quot; id=&quot;&rdquo;det2&Prime;&quot;&gt;&lt;\/div&gt; &lt;\/div&gt; &lt;!&ndash; RUNTIMES &ndash;&gt; &lt;div class=&quot;&rdquo;pane&rdquo;&quot; id=&quot;&rdquo;p3&Prime;&quot;&gt; &lt;div class=&quot;&rdquo;hint&rdquo;&quot;&gt;The harness picks a runtime &lt;b&gt;automatically&lt;\/b&gt;, per eval. Toggle to compare.&lt;\/div&gt; &lt;div class=&quot;&rdquo;tog&rdquo;&quot;&gt; &lt;button class=&quot;&rdquo;tg&quot; on&rdquo; data-r=&quot;&rdquo;0&Prime;&quot;&gt;Tools evals&lt;\/button&gt; &lt;button class=&quot;&rdquo;tg&rdquo;&quot; data-r=&quot;&rdquo;1&Prime;&quot;&gt;Local-stack evals&lt;\/button&gt; &lt;\/div&gt; &lt;div class=&quot;&rdquo;lanes&rdquo;&quot; id=&quot;&rdquo;lanes&rdquo;&quot;&gt;&lt;\/div&gt; &lt;div class=&quot;&rdquo;note&rdquo;&quot; id=&quot;&rdquo;rnote&rdquo;&quot;&gt;&lt;\/div&gt; &lt;\/div&gt; &lt;!&ndash; FINDINGS &ndash;&gt; &lt;div class=&quot;&rdquo;pane&rdquo;&quot; id=&quot;&rdquo;p4&Prime;&quot;&gt; &lt;div class=&quot;&rdquo;hint&rdquo;&quot;&gt;Published &lt;b&gt;Build stage&lt;\/b&gt; pass rates. Toggle the Supabase agent skill on and off.&lt;\/div&gt; &lt;div class=&quot;&rdquo;tog&rdquo;&quot;&gt; &lt;button class=&quot;&rdquo;tg&rdquo;&quot; data-s=&quot;&rdquo;0&Prime;&quot;&gt;No skill loaded&lt;\/button&gt; &lt;button class=&quot;&rdquo;tg&quot; on&rdquo; data-s=&quot;&rdquo;1&Prime;&quot;&gt;Skill loaded&lt;\/button&gt; &lt;\/div&gt; &lt;div class=&quot;&rdquo;rows&rdquo;&quot; id=&quot;&rdquo;rows&rdquo;&quot;&gt;&lt;\/div&gt; &lt;div class=&quot;&rdquo;note&rdquo;&quot;&gt; &lt;b&gt;Also measured:&lt;\/b&gt; Codex \/ GPT-5.6 reads about 8 docs pages per scenario, versus roughly 2 for Claude Code, which checks the docs in under 40% of scenarios even with skills loaded. Rewriting the Postgres best-practices skill description lifted its activation from about 1 in 10 sessions to 60%.&lt;br&gt;&lt;br&gt; Figures are a snapshot from Supabase&rsquo;s launch post (31 Jul 2026). Results move as models change &mdash; check the live page. &lt;\/div&gt; &lt;\/div&gt; &lt;div class=&quot;&rdquo;ft&rdquo;&quot;&gt; &lt;span&gt;Source: &lt;a href=&quot;\/fr\/&rdquo;https:\/\/supabase.com\/blog\/introducing-supabase-evals&rdquo;\/&quot; target=&quot;&rdquo;_blank&rdquo;&quot; rel=&quot;&rdquo;noopener&rdquo;&quot;&gt;Supabase blog&lt;\/a&gt; &middot; &lt;a href=&quot;\/fr\/&rdquo;https:\/\/github.com\/supabase\/evals&rdquo;\/&quot; target=&quot;&rdquo;_blank&rdquo;&quot; rel=&quot;&rdquo;noopener&rdquo;&quot;&gt;supabase\/evals&lt;\/a&gt; &middot; Apache-2.0&lt;\/span&gt; &lt;span&gt;&lt;b&gt;Marktechpost&lt;\/b&gt;&lt;\/span&gt; &lt;\/div&gt; &lt;\/div&gt; &lt;script&gt; var STAGES=[ {i:&rdquo;&ldquo;,l:&rdquo;Scenario&rdquo;,h:&rdquo;1 &middot; A real scenario&rdquo;,t:&rdquo;Each eval is grounded in a real problem &mdash; a support ticket, bug report, or GitHub issue. &lt;code&gt;PROMPT.md&lt;\/code&gt; carries the task the agent sees plus frontmatter tagging its stage, product, and topic.&rdquo;}, {i:&rdquo;&ldquo;,l:&rdquo;Environments&rdquo;,h:&rdquo;2 &middot; Two real environments&rdquo;,t:&rdquo;The framework boots a hosted-like Supabase stack and a local CLI project in containers. &lt;code&gt;platform-lite&lt;\/code&gt; serves a Management API-compatible surface backed by &lt;code&gt;@supabase\/lite&lt;\/code&gt;.&rdquo;}, {i:&rdquo;&ldquo;,l:&rdquo;Agent runs&rdquo;,h:&rdquo;3 &middot; The agent works&rdquo;,t:&rdquo;Claude Code, Codex, OpenCode, or an AI SDK agent invokes the real Supabase MCP server and CLI &mdash; not mocks. Skills load lazily: only name and description sit in the system prompt.&rdquo;}, {i:&rdquo;&ldquo;,l:&rdquo;One retry&rdquo;,h:&rdquo;4 &middot; One retry allowed&rdquo;,t:&rdquo;To cut false negatives while keeping runs sustainable, agents may retry once after a failure before they are graded.&rdquo;}, {i:&rdquo;&ldquo;,l:&rdquo;Scoring&rdquo;,h:&rdquo;5 &middot; Deterministic + judge&rdquo;,t:&rdquo;&lt;code&gt;EVAL.ts&lt;\/code&gt; exports the scorer. Deterministic checks confirm things like whether a user can reach certain data or an Edge Function returns the expected result; an LLM judge handles semantic calls.&rdquo;}, {i:&rdquo;&ldquo;,l:&rdquo;Results&rdquo;,h:&rdquo;6 &middot; Benchmark or regression&rdquo;,t:&rdquo;Benchmark scenarios go to the public site and run when assessing new changes or harnesses. Regression scenarios track known failure modes and refresh daily, without moving published scores.&rdquo;} ]; var FILES=[ {n:&rdquo;PROMPT.md&rdquo;,d:&rdquo;Task + frontmatter&rdquo;,h:&rdquo;PROMPT.md&rdquo;,t:&rdquo;Frontmatter plus the task description the agent sees. Keys drive discovery and the site filters: &lt;code&gt;stage&lt;\/code&gt;, &lt;code&gt;suite&lt;\/code&gt;, &lt;code&gt;product&lt;\/code&gt;, &lt;code&gt;topic&lt;\/code&gt;, &lt;code&gt;motivation&lt;\/code&gt;. &lt;code&gt;suite&lt;\/code&gt; is required on every eval.&rdquo;}, {n:&rdquo;EVAL.ts&rdquo;,d:&rdquo;The scorer&rdquo;,h:&rdquo;EVAL.ts&rdquo;,t:&rdquo;A default-exported scorer. Scorers check what the agent produced, never what the harness provisioned &mdash; with &lt;code&gt;projectRunning: true&lt;\/code&gt;, only the agent&rsquo;s deltas are scored.&rdquo;}, {n:&rdquo;remote\/&rdquo;,d:&rdquo;Hosted project state&rdquo;,h:&rdquo;remote\/ &mdash; optional&rdquo;,t:&rdquo;The hosted project&rsquo;s starting state, seeded into platform-lite: &lt;code&gt;project.sql&lt;\/code&gt; for the database, &lt;code&gt;logs.jsonl&lt;\/code&gt; for observability logs, and &lt;code&gt;functions\/&lt;\/code&gt; for already-deployed Edge Functions.&rdquo;}, {n:&rdquo;local\/&rdquo;,d:&rdquo;Agent workspace&rdquo;,h:&rdquo;local\/ &mdash; optional&rdquo;,t:&rdquo;The developer&rsquo;s working directory, copied into the sandbox before the agent starts. Its presence is also a runtime switch: ship a &lt;code&gt;local\/&lt;\/code&gt; and the eval boots a Docker sandbox.&rdquo;} ]; var RUN=[ {lanes:[[&ldquo;PROMPT&rdquo;,&rdquo;Agent gets the task, with no local\/ directory and no interface: cli&rdquo;],[&ldquo;TOOLS&rdquo;,&rdquo;It works through the experiment&rsquo;s MCP \/ tool surface only &mdash; there is no filesystem&rdquo;],[&ldquo;SKILLS&rdquo;,&rdquo;A load_skill tool returns a skill&rsquo;s full instructions on demand&rdquo;],[&ldquo;SCORE&rdquo;,&rdquo;The resulting project state or report is graded&rdquo;]], note:&rdquo;&lt;b&gt;Tools evals&lt;\/b&gt; exercise the MCP surface in isolation. Because the agent has no filesystem, skills are fetched through a tool call rather than read from disk.&rdquo;}, {lanes:[[&ldquo;PROMPT&rdquo;,&rdquo;Eval ships a local\/ workspace or declares interface: cli&rdquo;],[&ldquo;SANDBOX&rdquo;,&rdquo;A fresh Docker container boots per attempt, with the real Supabase CLI installed&rdquo;],[&ldquo;STACK&rdquo;,&rdquo;The agent runs supabase init \/ start \/ db \/ test against a live local stack&rdquo;],[&ldquo;EXPORT&rdquo;,&rdquo;The workspace is copied back to the host so scorers run vite \/ vitest against it&rdquo;]], note:&rdquo;&lt;b&gt;Local-stack evals&lt;\/b&gt; need a running Docker daemon, and default ports 54321&ndash;54329 free. A &lt;code&gt;services:&lt;\/code&gt; list keeps stack boots fast by starting only what the scenario needs.&rdquo;} ]; var SCORES=[ {n:&rdquo;Opus 5&Prime;,off:100,on:100}, {n:&rdquo;Kimi K3&Prime;,off:100,on:100}, {n:&rdquo;GPT-5.6 Sol&rdquo;,off:89,on:100}, {n:&rdquo;Sonnet 5&Prime;,off:78,on:100}, {n:&rdquo;GPT-5.4 mini&rdquo;,off:78,on:89} ]; function $(s){return document.querySelector(s)} function all(s){return [].slice.call(document.querySelectorAll(s))} \/* tabs *\/ all(&lsquo;.tab&rsquo;).forEach(function(b){b.onclick=function(){ all(&lsquo;.tab&rsquo;).forEach(function(x){x.classList.remove(&lsquo;on&rsquo;)}); all(&lsquo;.pane&rsquo;).forEach(function(x){x.classList.remove(&lsquo;on&rsquo;)}); b.classList.add(&lsquo;on&rsquo;);$(&lsquo;#&rsquo;+b.dataset.p).classList.add(&lsquo;on&rsquo;); }}); \/* pipeline *\/ var flow=$(&lsquo;#flow&rsquo;); STAGES.forEach(function(s,i){ var d=document.createElement(&lsquo;div&rsquo;);d.className=&rsquo;node&rsquo;;d.dataset.i=i; d.innerHTML=&#039;&lt;span class=&quot;&rdquo;dot&rdquo;&quot;&gt;&lt;\/span&gt;&lt;div class=&quot;&rdquo;nnum&rdquo;&quot;&gt;0&rsquo;+(i+1)+&#039;&lt;\/div&gt;&lt;div class=&quot;&rdquo;nico&rdquo;&quot;&gt;&rsquo;+s.i+&#039;&lt;\/div&gt;&lt;div class=&quot;&rdquo;nlab&rdquo;&quot;&gt;&rsquo;+s.l+&#039;&lt;\/div&gt;&rsquo;; d.onclick=function(){pick(i)};flow.appendChild(d); }); function pick(i){<\/p>","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"pmpro_default_level":"","site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"_pvb_checkbox_block_on_post":false,"footnotes":""},"categories":[52,5,7,1],"tags":[],"class_list":["post-108384","post","type-post","status-publish","format-standard","hentry","category-ai-club","category-committee","category-news","category-uncategorized","pmpro-has-access"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks - YouZum<\/title>\n<meta name=\"description\" content=\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/youzum.net\/fr\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\" \/>\n<meta property=\"og:locale\" content=\"fr_FR\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks - YouZum\" \/>\n<meta property=\"og:description\" content=\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\" \/>\n<meta property=\"og:url\" content=\"https:\/\/youzum.net\/fr\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\" \/>\n<meta property=\"og:site_name\" content=\"YouZum\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/DroneAssociationTH\/\" \/>\n<meta property=\"article:published_time\" content=\"2026-08-01T19:59:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png\" \/>\n<meta name=\"author\" content=\"admin NU\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u00c9crit par\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin NU\" \/>\n\t<meta name=\"twitter:label2\" content=\"Dur\u00e9e de lecture estim\u00e9e\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\"},\"author\":{\"name\":\"admin NU\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c\"},\"headline\":\"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks\",\"datePublished\":\"2026-08-01T19:59:42+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\"},\"wordCount\":2122,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\"},\"image\":{\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png\",\"articleSection\":[\"AI\",\"Committee\",\"News\",\"Uncategorized\"],\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\",\"url\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\",\"name\":\"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks - YouZum\",\"isPartOf\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png\",\"datePublished\":\"2026-08-01T19:59:42+00:00\",\"description\":\"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19\",\"breadcrumb\":{\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#breadcrumb\"},\"inLanguage\":\"fr-FR\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage\",\"url\":\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png\",\"contentUrl\":\"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/youzum.net\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/yousum.gpucore.co\/#website\",\"url\":\"https:\/\/yousum.gpucore.co\/\",\"name\":\"YouSum\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/yousum.gpucore.co\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"fr-FR\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/yousum.gpucore.co\/#organization\",\"name\":\"Drone Association Thailand\",\"url\":\"https:\/\/yousum.gpucore.co\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png\",\"contentUrl\":\"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png\",\"width\":300,\"height\":300,\"caption\":\"Drone Association Thailand\"},\"image\":{\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/DroneAssociationTH\/\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c\",\"name\":\"admin NU\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"fr-FR\",\"@id\":\"https:\/\/yousum.gpucore.co\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png\",\"contentUrl\":\"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png\",\"caption\":\"admin NU\"},\"url\":\"https:\/\/youzum.net\/fr\/members\/adminnu\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks - YouZum","description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/youzum.net\/fr\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/","og_locale":"fr_FR","og_type":"article","og_title":"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks - YouZum","og_description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","og_url":"https:\/\/youzum.net\/fr\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/","og_site_name":"YouZum","article_publisher":"https:\/\/www.facebook.com\/DroneAssociationTH\/","article_published_time":"2026-08-01T19:59:42+00:00","og_image":[{"url":"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png","type":"","width":"","height":""}],"author":"admin NU","twitter_card":"summary_large_image","twitter_misc":{"\u00c9crit par":"admin NU","Dur\u00e9e de lecture estim\u00e9e":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#article","isPartOf":{"@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/"},"author":{"name":"admin NU","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c"},"headline":"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks","datePublished":"2026-08-01T19:59:42+00:00","mainEntityOfPage":{"@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/"},"wordCount":2122,"commentCount":0,"publisher":{"@id":"https:\/\/yousum.gpucore.co\/#organization"},"image":{"@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage"},"thumbnailUrl":"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png","articleSection":["AI","Committee","News","Uncategorized"],"inLanguage":"fr-FR","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/","url":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/","name":"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks - YouZum","isPartOf":{"@id":"https:\/\/yousum.gpucore.co\/#website"},"primaryImageOfPage":{"@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage"},"image":{"@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage"},"thumbnailUrl":"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png","datePublished":"2026-08-01T19:59:42+00:00","description":"\u0e01\u0e34\u0e08\u0e01\u0e23\u0e23\u0e21\u0e40\u0e01\u0e35\u0e48\u0e22\u0e27\u0e01\u0e31\u0e1a\u0e42\u0e14\u0e23\u0e19","breadcrumb":{"@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#breadcrumb"},"inLanguage":"fr-FR","potentialAction":[{"@type":"ReadAction","target":["https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/"]}]},{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#primaryimage","url":"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png","contentUrl":"https:\/\/s.w.org\/images\/core\/emoji\/17.0.2\/72x72\/1f4dd.png"},{"@type":"BreadcrumbList","@id":"https:\/\/youzum.net\/supabase-releases-evals-an-open-source-benchmark-that-scores-claude-code-codex-and-opencode-on-real-supabase-tasks\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/youzum.net\/"},{"@type":"ListItem","position":2,"name":"Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks"}]},{"@type":"WebSite","@id":"https:\/\/yousum.gpucore.co\/#website","url":"https:\/\/yousum.gpucore.co\/","name":"YouSum","description":"","publisher":{"@id":"https:\/\/yousum.gpucore.co\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/yousum.gpucore.co\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"fr-FR"},{"@type":"Organization","@id":"https:\/\/yousum.gpucore.co\/#organization","name":"Drone Association Thailand","url":"https:\/\/yousum.gpucore.co\/","logo":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/","url":"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png","contentUrl":"https:\/\/youzum.net\/wp-content\/uploads\/2024\/11\/tranparent-logo.png","width":300,"height":300,"caption":"Drone Association Thailand"},"image":{"@id":"https:\/\/yousum.gpucore.co\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/DroneAssociationTH\/"]},{"@type":"Person","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/97fa48242daf3908e4d9a5f26f4a059c","name":"admin NU","image":{"@type":"ImageObject","inLanguage":"fr-FR","@id":"https:\/\/yousum.gpucore.co\/#\/schema\/person\/image\/","url":"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png","contentUrl":"https:\/\/youzum.net\/wp-content\/uploads\/avatars\/2\/1746849356-bpfull.png","caption":"admin NU"},"url":"https:\/\/youzum.net\/fr\/members\/adminnu\/"}]}},"rttpg_featured_image_url":null,"rttpg_author":{"display_name":"admin NU","author_link":"https:\/\/youzum.net\/fr\/members\/adminnu\/"},"rttpg_comment":0,"rttpg_category":"<a href=\"https:\/\/youzum.net\/fr\/category\/ai-club\/\" rel=\"category tag\">AI<\/a> <a href=\"https:\/\/youzum.net\/fr\/category\/committee\/\" rel=\"category tag\">Committee<\/a> <a href=\"https:\/\/youzum.net\/fr\/category\/news\/\" rel=\"category tag\">News<\/a> <a href=\"https:\/\/youzum.net\/fr\/category\/uncategorized\/\" rel=\"category tag\">Uncategorized<\/a>","rttpg_excerpt":"Supabase has open sourced Supabase Evals, its benchmark and framework for testing how well AI agents build using Supabase. It runs coding agents including Claude Code, Codex, and OpenCode against real tasks, such as building a schema, debugging a failed Edge Function, or fixing a broken RLS policy, then scores the result. It powers the\u2026","_links":{"self":[{"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/posts\/108384","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/comments?post=108384"}],"version-history":[{"count":0,"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/posts\/108384\/revisions"}],"wp:attachment":[{"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/media?parent=108384"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/categories?post=108384"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/youzum.net\/fr\/wp-json\/wp\/v2\/tags?post=108384"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}