{"id":113165,"date":"2026-08-23T00:42:02","date_gmt":"2026-08-23T00:42:02","guid":{"rendered":"https:\/\/youzum.net\/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the-provider-economics-behind-each\/"},"modified":"2026-08-23T00:42:02","modified_gmt":"2026-08-23T00:42:02","slug":"decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the-provider-economics-behind-each","status":"publish","type":"post","link":"https:\/\/youzum.net\/fr\/decoding-ais-open-source-course-maps-three-ways-to-run-an-agent-loop-and-the-provider-economics-behind-each\/","title":{"rendered":"Decoding AI\u2019s Open-Source Course Maps Three Ways to Run an Agent Loop and the Provider Economics Behind Each"},"content":{"rendered":"

Most teams treat \u2018which model\u2019 as the important decision. The harness engineering literature keeps pointing somewhere else. In LangChain\u2019s Terminal-Bench experiment<\/a>, changing only the harness\u2014same model throughout\u2014moved a coding agent from roughly 30th place into the top 5.<\/p>\n

That result reframes the question. If the harness decides quality, then how you run the loop<\/em> becomes an architecture decision, not a deployment detail. Paul Iusztin\u2019s open-source course Building a Coding Agent From Scratch<\/a> builds a Python agent called Decode<\/strong>. Published through Decoding AI<\/a>, it separates three run modes. Each mode has a different latency profile. Each one therefore wants a different inference provider.<\/p>\n

One headless core, three shapes<\/strong><\/h2>\n

The center of the system is a headless harness<\/strong> with no interface of its own. Inside it runs the agent loop every harness shares: the LLM picks an action, a tool executes, the observation feeds back. Everything reads from and writes to the context window.<\/p>\n

The agent itself is small. In Decode it is a ~20-line Pydantic AI<\/a> definition composing a model, tools, and an output type. In Claude Code\u2019s leaked source, the core loop is roughly 150 lines. Everything else\u2014memory, skills, sandbox, permissions, LSP feedback, compaction\u2014is harness.<\/p>\n

Interfaces then plug into that core. That is where the three modes appear:<\/strong><\/p>\n

Mode 1: Interactive, online<\/strong><\/h3>\n

A terminal UI is wired to one live session, in memory, in the same process. Events stream back through async generators as tokens arrive.<\/p>\n

The hard problem here is steering. If you type while a tool call is in flight, injecting the message immediately corrupts the turn. Decode\u2019s answer is a steering queue plus a priority gate<\/strong>. Input is buffered on arrival and injected only at a safe boundary. The loop exposes two: MODEL_REQUEST<\/code>, before the next model call, and WOULD_STOP<\/code>, when the turn would end.<\/p>\n

Three input modes map onto that. Plain Enter<\/code> steers within the turn. Alt+Enter<\/code> queues a follow-up until the turn stops. Esc<\/code> triggers a cooperative abort at the next boundary, clearing both queues so history stays intact.<\/p>\n

A human is reading every token. This mode is latency-bound, which is why it belongs on a low-latency hosted API.<\/p>\n

Mode 2: Remote, offline<\/strong><\/h3>\n

Remote mode keeps the harness headless and runs it on a server through an agent runtime. Decode uses Kitaru<\/a>, ZenML\u2019s agent runtime, deployed to GCP, with the agents themselves executing on Modal<\/a>.<\/p>\n

Nobody is watching. A backlog of tickets fans out to N harnesses in parallel, each producing its own PR. Because the runtime records progress step by step, a sandbox that dies mid-task resumes from its last recorded step instead of restarting. A run that pauses for human input freezes and consumes no compute while it waits.<\/p>\n

Tools execute inside Modal Sandboxes<\/a> remotely, Docker locally. The metric that matters is throughput per dollar, not time-to-first-token.<\/p>\n

Mode 3: Async, online<\/strong><\/h3>\n

The third shape sits between the two. A live session hands work to a job queue and returns immediately. Background workflows fan out LLM calls and post results back later.<\/p>\n

The user is online but not watching each step. The queue owns the work, so the run outlives the client that started it. This is the pattern behind Slack-triggered agents and background PR review, and it bills like batch, not like chat.<\/p>\n

The interactive explainer<\/strong><\/h2>\n
\n<\/div>\n

\n

Why the provider changes with the mode<\/strong><\/h2>\n<\/p>

The cost model follows the latency requirement, and the gap is large.<\/p>\n

Take 1,000 documents at 30,000 input tokens each, roughly 500 output tokens per document. At frontier API rates of $3 per million input and $15 per million output, the lesson’s arithmetic lands near $97<\/strong>. Prompt caching does not rescue it, because every document is a different prefix. Batched on a serverless GPU at around 3,000 tokens per second, the same work is under three hours of GPU time\u2014roughly $13<\/strong>.<\/p>\n

The reverse case is just as sharp. Decode’s default test model, Qwen3.6 35B<\/code>, runs on a single H200. Modal’s published pricing<\/a> lists H200 SXM at $0.001261 per second, or about $4.54 per hour<\/strong>. Leave an interactive agent idle overnight waiting on a y<\/code> confirmation, and ten idle hours add roughly $45<\/strong> to the bill.<\/p>\n

That is the whole argument. Interactive work pays per token because a human is waiting. Offline and async work pays per GPU-hour because throughput is the objective and idle time is the enemy.<\/p>\n

There is a second axis: serverless versus reserved capacity. Modal’s pricing analysis<\/a> reduces it to one comparison. Reservations charge the peak rate for the whole contract; serverless follows the demand curve. When the peak-to-average ratio exceeds the reservation discount<\/strong>, serverless is cheaper. Modal reports typical discounts of 2\u20135\u00d7 against peak-to-average ratios of 5\u201310\u00d7 for inference, training, and agentic development. Industry surveys it cites put reservation utilization below 30%, often under 10%.<\/p>\n

Key Takeaways<\/strong><\/h2>\n