Committee

DeepSeek published DeepSeek-V4-Flash-0731 on Hugging Face and moved the official V4-Flash API into public beta on July 31, 2026. The model card is explicit that this is the official release superseding the preview, and that the architecture and size are unchanged. The gains come from re-post-training, not a new design. The checkpoint ships with the DSpark speculative decoding module attached, matching the structure of DeepSeek-V4-Flash-DSpark. Hugging Face reports 304B parameters for the repo, which includes that draft module on top of the 284B base. On the API side, deepseek-v4-flash now natively supports the Responses API format and is adapted for Codex. The V4-Pro API and the app and web models were not updated. Is it deployable? Yes, in two very different ways. Via API, it is deployable by almost anyone: DeepSeek’s pricing page lists deepseek-v4-flash at $0.14 per 1M input tokens on a cache miss, $0.0028 on a cache hit, and $0.28 per 1M output tokens, with a 2,500 concurrency limit. That is roughly a third of deepseek-v4-pro output pricing ($0.87). Seed-stage startups, indie developers, and internal platform teams can run agent loops at this price without a GPU budget. Via self-hosting, the bar is much higher: The weights are MIT-licensed and ungated, but every expert stays resident in memory even though only 13B activate per token. DeepSeek’s vLLM example serves it on a single 4×GB300 node. Unsloth’s dynamic GGUFs put the lossless 8-bit build at 162 GB and a 3-bit build at 103 GB, needing roughly 110 GB of combined RAM plus VRAM. Self-hosting suits mid-size and large enterprises with a serving cluster, or one well-specced workstation at aggressive quantization. Architecture Per the DeepSeek-V4 technical report, V4-Flash is a 284B-parameter MoE with 13B activated per token and a 1M-token context window. Each MoE layer holds 1 shared expert and 256 routed experts with an intermediate dimension of 2048, and 6 routed experts fire per token. The first three MoE layers use hash routing. Multi-token prediction depth is 1. Attention is hybrid, combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA). Manifold-Constrained Hyper-Connections (mHC) replace conventional residual connections, with expansion factor 4 and 20 Sinkhorn-Knopp iterations. Pre-training used more than 32T tokens and the Muon optimizer. The paper’s headline efficiency figure — 27% of single-token inference FLOPs and 10% of KV cache versus DeepSeek-V3.2 at 1M context — is stated for V4-Pro, not Flash. <!– EMBED HERE: paste wordpress-embed.html into a Custom HTML block –> Benchmarks All figures below are DeepSeek-reported, from the 0731 model card. Benchmark V4-Flash-0731 V4-Flash (Preview) V4-Pro (Preview) GLM-5.2 Opus-4.8 Terminal Bench 2.1 82.7 61.8 72.1 81.0 85.0 NL2Repo 54.2 39.4 38.5 48.9 69.7 Cybergym 76.7 38.7 52.7 — 83.1 DeepSWE 54.4 7.3 12.8 46.2 58.0 Toolathlon-Verified 70.3 49.7 55.9 59.9 76.2 Agents’ Last Exam 25.2 15.8 16.5 23.8 25.7 AutomationBench Public 25.1 10.8 12.8 12.9 27.2 Two important things to note: Code Agent tasks were run with the minimal mode of DeepSeek Harness, which has not been released. DSBench-FullStack (68.7) and DSBench-Hard (59.6) are internal test sets. Agent scores are harness-sensitive, so independent runs may diverge. Serving it DSpark is enabled with one vLLM flag: –speculative-config ‘{“method”:”dspark”,”num_speculative_tokens”:7,”draft_sample_method”:”greedy”}’. The DSpark paper reports 60–85% faster per-user generation on V4-Flash versus the MTP-1 baseline at matched aggregate throughput. There is no Jinja chat template. DeepSeek ships an encoding/ folder with encode_messages and parse_message_from_completion_text instead. reasoning_effort takes low, high, or max. DeepSeek recommends temperature = 1.0, top_p = 0.95 for agentic use and 1.0 otherwise, with up to 384K output tokens at high and max. Key Takeaways Same 284B/13B architecture as the April preview: the jump is post-training only. Beats V4-Pro (Preview) on every agentic benchmark DeepSeek published, at a third of the output price. MIT-licensed and ungated, so on-premise commercial deployment is unblocked. Self-hosting needs ~110 GB memory at 3-bit, or a 4×GB300 node for full-precision serving. All benchmark numbers are vendor-reported on an unreleased harness — run your own evals first. Check out the Model Update on HF. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post DeepSeek Upgrades DeepSeek-V4-Flash-0731 with Major Agentic and Coding Gains appeared first on MarkTechPost.

Supabase has open sourced Supabase Evals, its benchmark and framework for testing how well AI agents build using Supabase. It runs coding agents including Claude Code, Codex, and OpenCode against real tasks, such as building a schema, debugging a failed Edge Function, or fixing a broken RLS policy, then scores the result. It powers the public leaderboard at supabase.com/evals and an internal regression suite monitored daily. Is it deployable? Yes, today. supabase/evals is public under Apache-2.0 and runs locally via pnpm. Industries: Developer tooling, cloud infrastructure, data platforms, and regulated backends in fintech or healthcare, where an agent writing a wrong RLS policy is a security incident. Applications: Regression-testing docs and skill edits, gating SDK releases, and comparing agent harnesses head to head. Constraints: Local-stack runs need a Docker daemon, provider API keys, and ports 54321–54329 free. How the harness works Supabase defined three dimensions: products (database, auth, storage, edge-functions, realtime, cron, queues, vectors, data-api), topics (RLS, security, migrations, SQL, SDK, observability, self-hosting, tests, declarative-schema), and stages (build, deploy, investigate, resolve). It then picked the smallest scenario set touching each dimension once, grounded in support tickets, bug reports, and GitHub issues. Scenarios split into two suites. Benchmark scenarios cover breadth and are published. Regression scenarios cover known failure modes, refresh daily, and do not move published scores. Every scenario runs against a real environment. The framework boots a hosted-like stack and a local CLI project in containers, so agents call the actual MCP server and CLI. A platform-lite runtime exposes a Management API-compatible surface backed by @supabase/lite. Scoring combines deterministic checks with LLM-as-a-judge. Agents get one retry before grading. Each eval directory holds PROMPT.md (task plus frontmatter), EVAL.ts (the scorer), and optional remote/ and local/ starting states. Shipping a local/ workspace, or declaring interface: cli, boots a Docker sandbox with the real CLI installed. Run the pipeline</button> </div> <!– ANATOMY –> <div class="”pane”" id="”p2″"> <div class="”hint”">Every eval lives at <b>evals/&lt;id&gt;/</b>. Click a file to see what it holds.</div> <div class="”tree”" id="”tree”"></div> <div class="”detail”" id="”det2″"></div> </div> <!– RUNTIMES –> <div class="”pane”" id="”p3″"> <div class="”hint”">The harness picks a runtime <b>automatically</b>, per eval. Toggle to compare.</div> <div class="”tog”"> <button class="”tg" on” data-r="”0″">Tools evals</button> <button class="”tg”" data-r="”1″">Local-stack evals</button> </div> <div class="”lanes”" id="”lanes”"></div> <div class="”note”" id="”rnote”"></div> </div> <!– FINDINGS –> <div class="”pane”" id="”p4″"> <div class="”hint”">Published <b>Build stage</b> pass rates. Toggle the Supabase agent skill on and off.</div> <div class="”tog”"> <button class="”tg”" data-s="”0″">No skill loaded</button> <button class="”tg" on” data-s="”1″">Skill loaded</button> </div> <div class="”rows”" id="”rows”"></div> <div class="”note”"> <b>Also measured:</b> Codex / GPT-5.6 reads about 8 docs pages per scenario, versus roughly 2 for Claude Code, which checks the docs in under 40% of scenarios even with skills loaded. Rewriting the Postgres best-practices skill description lifted its activation from about 1 in 10 sessions to 60%.<br><br> Figures are a snapshot from Supabase’s launch post (31 Jul 2026). Results move as models change — check the live page. </div> </div> <div class="”ft”"> <span>Source: <a href="/fr/”https://supabase.com/blog/introducing-supabase-evals”/" target="”_blank”" rel="”noopener”">Supabase blog</a> · <a href="/fr/”https://github.com/supabase/evals”/" target="”_blank”" rel="”noopener”">supabase/evals</a> · Apache-2.0</span> <span><b>Marktechpost</b></span> </div> </div> <script> var STAGES=[ {i:”“,l:”Scenario”,h:”1 · A real scenario”,t:”Each eval is grounded in a real problem — a support ticket, bug report, or GitHub issue. <code>PROMPT.md</code> carries the task the agent sees plus frontmatter tagging its stage, product, and topic.”}, {i:”“,l:”Environments”,h:”2 · Two real environments”,t:”The framework boots a hosted-like Supabase stack and a local CLI project in containers. <code>platform-lite</code> serves a Management API-compatible surface backed by <code>@supabase/lite</code>.”}, {i:”“,l:”Agent runs”,h:”3 · The agent works”,t:”Claude Code, Codex, OpenCode, or an AI SDK agent invokes the real Supabase MCP server and CLI — not mocks. Skills load lazily: only name and description sit in the system prompt.”}, {i:”“,l:”One retry”,h:”4 · One retry allowed”,t:”To cut false negatives while keeping runs sustainable, agents may retry once after a failure before they are graded.”}, {i:”“,l:”Scoring”,h:”5 · Deterministic + judge”,t:”<code>EVAL.ts</code> exports the scorer. Deterministic checks confirm things like whether a user can reach certain data or an Edge Function returns the expected result; an LLM judge handles semantic calls.”}, {i:”“,l:”Results”,h:”6 · Benchmark or regression”,t:”Benchmark scenarios go to the public site and run when assessing new changes or harnesses. Regression scenarios track known failure modes and refresh daily, without moving published scores.”} ]; var FILES=[ {n:”PROMPT.md”,d:”Task + frontmatter”,h:”PROMPT.md”,t:”Frontmatter plus the task description the agent sees. Keys drive discovery and the site filters: <code>stage</code>, <code>suite</code>, <code>product</code>, <code>topic</code>, <code>motivation</code>. <code>suite</code> is required on every eval.”}, {n:”EVAL.ts”,d:”The scorer”,h:”EVAL.ts”,t:”A default-exported scorer. Scorers check what the agent produced, never what the harness provisioned — with <code>projectRunning: true</code>, only the agent’s deltas are scored.”}, {n:”remote/”,d:”Hosted project state”,h:”remote/ — optional”,t:”The hosted project’s starting state, seeded into platform-lite: <code>project.sql</code> for the database, <code>logs.jsonl</code> for observability logs, and <code>functions/</code> for already-deployed Edge Functions.”}, {n:”local/”,d:”Agent workspace”,h:”local/ — optional”,t:”The developer’s working directory, copied into the sandbox before the agent starts. Its presence is also a runtime switch: ship a <code>local/</code> and the eval boots a Docker sandbox.”} ]; var RUN=[ {lanes:[[“PROMPT”,”Agent gets the task, with no local/ directory and no interface: cli”],[“TOOLS”,”It works through the experiment’s MCP / tool surface only — there is no filesystem”],[“SKILLS”,”A load_skill tool returns a skill’s full instructions on demand”],[“SCORE”,”The resulting project state or report is graded”]], note:”<b>Tools evals</b> exercise the MCP surface in isolation. Because the agent has no filesystem, skills are fetched through a tool call rather than read from disk.”}, {lanes:[[“PROMPT”,”Eval ships a local/ workspace or declares interface: cli”],[“SANDBOX”,”A fresh Docker container boots per attempt, with the real Supabase CLI installed”],[“STACK”,”The agent runs supabase init / start / db / test against a live local stack”],[“EXPORT”,”The workspace is copied back to the host so scorers run vite / vitest against it”]], note:”<b>Local-stack evals</b> need a running Docker daemon, and default ports 54321–54329 free. A <code>services:</code> list keeps stack boots fast by starting only what the scenario needs.”} ]; var SCORES=[ {n:”Opus 5″,off:100,on:100}, {n:”Kimi K3″,off:100,on:100}, {n:”GPT-5.6 Sol”,off:89,on:100}, {n:”Sonnet 5″,off:78,on:100}, {n:”GPT-5.4 mini”,off:78,on:89} ]; function $(s){return document.querySelector(s)} function all(s){return [].slice.call(document.querySelectorAll(s))} /* tabs */ all(‘.tab’).forEach(function(b){b.onclick=function(){ all(‘.tab’).forEach(function(x){x.classList.remove(‘on’)}); all(‘.pane’).forEach(function(x){x.classList.remove(‘on’)}); b.classList.add(‘on’);$(‘#’+b.dataset.p).classList.add(‘on’); }}); /* pipeline */ var flow=$(‘#flow’); STAGES.forEach(function(s,i){ var d=document.createElement(‘div’);d.className=’node’;d.dataset.i=i; d.innerHTML='<span class="”dot”"></span><div class="”nnum”">0’+(i+1)+'</div><div class="”nico”">’+s.i+'</div><div class="”nlab”">’+s.l+'</div>’; d.onclick=function(){pick(i)};flow.appendChild(d); }); function pick(i){

MiniMax releases MiniMax H3, a general-purpose multimodal generation model. MiniMax H3 is not a text-to-video model with add-ons. MiniMax describes it as a general-purpose multimodal generation model that reads text, images, video, and audio as one unified context and returns video with native stereo sound. The mains specs include: 2K output, 4–15 seconds, integer durations only. Previous video stacks split into text-to-video, image-to-video, first-and-last-frame, subject reference, motion reference, and video editing, each often a separate expert model. MiniMax H3 folds those into one pretraining paradigm where reference and editing relationships are expressed in natural language. MiniMax’s example prompt makes the point: reference the camera movement from Video 1, have the character in Image 2 sing, match the vocals to Audio 3. Is it deployable? Today: yes, through the API and no, on your own hardware. MiniMax launched H3 on July 31, 2026 with the model live in the platform API under the model ID MiniMax-H3 and in the consumer Hailuo AI app. Industries: MiniMax positions MiniMax H3 for advertising, branding, e-commerce, product design, UI/UX, and gaming along with film pre-visualization and retail catalog media. Applications: Ad variant generation, product and listing videos, animated posters, film title sequences, website hero loops, character-consistent game cinematics, and video-to-video motion transfer. The API surface The video generation guide documents three entry modes: text-to-video, first/last-frame image-to-video, and reference generation. Behind one endpoint and an asynchronous three-step flow: create a task, poll task_id, download content.url. Input limits worth designing around: Reference images: up to 9. Reference videos: up to 3 clips, 2–15 s each, ≤15 s total. Reference audio: up to 3 clips, and audio cannot be sent without an accompanying image or video. Mixed input caps at 12 files total. Prompt length ≤7,000 characters; request body ≤64 MB, with URL input recommended for large assets. File sizes: video ≤50 MB, image ≤30 MB, audio ≤15 MB, per asset. Formats: H.264/H.265 video, JPG/PNG/WEBP/HEIC/HEIF images, WAV/MP3 audio. Four technical pieces doing the work Contextual Omni Representation: MiniMax rebuilt captioning so it describes the relationship between context and target video, not just the target. Most source material requires roughly 100K tokens of inference, distilled to about 4K tokens on average. Language is the bridge that turns a fixed task set into an open, descriptive one. H3-VAE: A full tokenizer overhaul. Its high compression ratio delivers a stated 4× gain in effective sequence length, cutting training and inference cost and it is the enabling technology for native 2K. H3-Omni Transformer: MiniMax explicitly set aside the Hailuo-02 architecture here. Multimodal context tripled sequence-length variance, so the training architecture separates understanding and generation workloads and tunes hardware utilization for each. Reported result: end-to-end training throughput up nearly 30%. In-Context Regeneration: Instead of a bolt-on super-resolution module, the base model regenerates its own low-resolution output in-context, re-reading the original multimodal context. That is what recovers small text and fine detail that conventional upscalers guess at — directly relevant to brand and product rendering. Price and standing MiniMax’s own claim: at 2K, H3’s per-second price is less than a third of mainstream models; at 768p, less than half the price of mainstream 720p. The company amplified both the launch and the pricing framing on X (1, 2). Third-party trackers and launch coverage put the 2K pay-as-you-go rate at $0.13 per second, about $1.95 for a 15-second clip, but MiniMax’s pay-as-you-go page still listed only Hailuo 2.3 tiers at the time of writing, so treat that figure as reported, not primary. On placement: SCMP reports, citing Artificial Analysis, that H3 leads in video editing while trailing Google’s Gemini Omni Flash in text-to-video and sitting behind both Seedance 2.0 and Gemini Omni Flash in image-to-video. Key Takeaways H3 unifies text, image, video, and audio into one generation model — 2K, 4–15s, native stereo. Open weights are promised “in the coming days,” not shipped; the API is the only path today. H3-VAE’s 4× effective sequence-length gain is what makes native 2K economically viable. In-context regeneration replaces super-resolution, preserving small text and brand marks. Artificial Analysis ranks H3 first in video editing, behind rivals in text-to-video and image-to-video. Sentimental Analysis Check out the Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post MiniMax Releases MiniMax H3: An Omni-Modal Video Model That Generates 15-Second 2K Clips With Native Stereo Audio appeared first on MarkTechPost.

In this tutorial, we explore how NVIDIA Transformer Engine accelerates transformer workloads by combining fused GPU kernels, BF16 computation, and hardware-aware FP8 execution. We begin by installing Transformer Engine and detecting the active GPU architecture so that we can determine whether the runtime supports TE kernels, FP8 tensor cores, or only the pure-PyTorch fallback path. We then examine core fused components such as te.Linear, te.LayerNorm, te.LayerNormLinear, te.LayerNormMLP, and te.TransformerLayer, while also configuring a delayed-scaling FP8 recipe that manages tensor scaling, amax history, and hybrid E4M3/E5M2 formats. Using these components, we construct a compact GPT-style causal language model, train it on deterministic synthetic sequences, compare higher-precision and FP8 execution, measure runtime and peak GPU memory, inspect FP8 metadata, and validate the trained model through autoregressive generation. Copy CodeCopiedUse a different Browser import subprocess, sys, os def pip_install(*pkgs): subprocess.run([sys.executable, “-m”, “pip”, “install”, “-q”, “–no-build-isolation”, *pkgs], check=False) print(“>> Installing transformer_engine[pytorch] (this can take a few minutes)…”) pip_install(“transformer_engine[pytorch]”) import time, math, gc import torch import torch.nn as nn import torch.nn.functional as F assert torch.cuda.is_available(), “Enable a GPU runtime in Colab first!” DEVICE = “cuda” props = torch.cuda.get_device_properties(0) CC = (props.major, props.minor) GPU_NAME = props.name print(f”>> GPU: {GPU_NAME} | compute capability {CC[0]}.{CC[1]} | ” f”{props.total_memory/1e9:.1f} GB”) TE_CAPABLE = CC >= (8, 0) FP8_CAPABLE = CC >= (8, 9) te = None if TE_CAPABLE: try: import transformer_engine.pytorch as te from transformer_engine.common import recipe print(“>> Transformer Engine imported OK:”, getattr(te, “__version__”, “unknown version”)) except Exception as e: print(f”>> TE import failed ({e}); using pure-PyTorch fallback.”) TE_CAPABLE = FP8_CAPABLE = False else: print(“>> GPU is pre-Ampere (e.g. T4): TE kernels unsupported -> fallback mode.”) if TE_CAPABLE and FP8_CAPABLE and te is not None: try: ok, reason = te.fp8.check_fp8_support() FP8_CAPABLE = bool(ok) if not ok: print(“>> TE reports FP8 unsupported:”, reason) except Exception: pass print(f”>> Mode: TE={‘ON’ if TE_CAPABLE else ‘OFF’} | ” f”FP8={‘ON’ if FP8_CAPABLE else ‘OFF (will use BF16)’}”) torch.manual_seed(1234) if TE_CAPABLE: H = 768 x_demo = torch.randn(8, 32, H, device=DEVICE, dtype=torch.bfloat16) lin = te.Linear(H, H, bias=True, params_dtype=torch.bfloat16).to(DEVICE) ln = te.LayerNorm(H, params_dtype=torch.bfloat16).to(DEVICE) ln_lin = te.LayerNormLinear(H, 3 * H, params_dtype=torch.bfloat16).to(DEVICE) ln_mlp = te.LayerNormMLP(H, 4 * H, params_dtype=torch.bfloat16).to(DEVICE) with torch.no_grad(): print(“n>> Module tour (shapes):”) print(” te.Linear “, tuple(lin(x_demo).shape)) print(” te.LayerNorm “, tuple(ln(x_demo).shape)) print(” te.LayerNormLinear”, tuple(ln_lin(x_demo).shape)) print(” te.LayerNormMLP “, tuple(ln_mlp(x_demo).shape)) del lin, ln, ln_lin, ln_mlp, x_demo gc.collect(); torch.cuda.empty_cache() fp8_recipe = None if FP8_CAPABLE: fp8_recipe = recipe.DelayedScaling( fp8_format=recipe.Format.HYBRID, amax_history_len=16, amax_compute_algo=”max”, ) print(“n>> FP8 recipe:”, fp8_recipe) We install NVIDIA Transformer Engine and initialize the PyTorch environment required for GPU-accelerated execution. We inspect the active GPU, compute capability, and memory capacity to determine whether fused TE kernels and FP8 tensor cores are available. We also validate the core fused modules and configure a delayed-scaling FP8 recipe while preserving an automatic PyTorch fallback for unsupported hardware. Copy CodeCopiedUse a different Browser VOCAB, D_MODEL, N_HEADS, N_LAYERS, FFN, SEQ = 96, 768, 12, 4, 3072, 256 class MiniGPT_TE(nn.Module): “””Causal LM where every block is a single fused te.TransformerLayer.””” def __init__(self): super().__init__() self.emb = nn.Embedding(VOCAB, D_MODEL) self.pos = nn.Embedding(SEQ, D_MODEL) self.blocks = nn.ModuleList([ te.TransformerLayer( hidden_size=D_MODEL, ffn_hidden_size=FFN, num_attention_heads=N_HEADS, self_attn_mask_type=”causal”, layer_number=i + 1, params_dtype=torch.bfloat16, hidden_dropout=0.0, attention_dropout=0.0, ) for i in range(N_LAYERS) ]) self.ln_f = nn.LayerNorm(D_MODEL) self.head = nn.Linear(D_MODEL, VOCAB, bias=False) def forward(self, idx): B, T = idx.shape h = self.emb(idx) + self.pos(torch.arange(T, device=idx.device)) h = h.to(torch.bfloat16) for blk in self.blocks: h = blk(h) h = self.ln_f(h.float()) return self.head(h) class Block_PT(nn.Module): “””Plain-PyTorch transformer block, mirrors te.TransformerLayer.””” def __init__(self): super().__init__() self.ln1 = nn.LayerNorm(D_MODEL) self.attn = nn.MultiheadAttention(D_MODEL, N_HEADS, batch_first=True) self.ln2 = nn.LayerNorm(D_MODEL) self.mlp = nn.Sequential(nn.Linear(D_MODEL, FFN), nn.GELU(), nn.Linear(FFN, D_MODEL)) def forward(self, x, mask): a, _ = self.attn(self.ln1(x), self.ln1(x), self.ln1(x), attn_mask=mask, need_weights=False) x = x + a return x + self.mlp(self.ln2(x)) class MiniGPT_PT(nn.Module): def __init__(self): super().__init__() self.emb = nn.Embedding(VOCAB, D_MODEL) self.pos = nn.Embedding(SEQ, D_MODEL) self.blocks = nn.ModuleList([Block_PT() for _ in range(N_LAYERS)]) self.ln_f = nn.LayerNorm(D_MODEL) self.head = nn.Linear(D_MODEL, VOCAB, bias=False) def forward(self, idx): B, T = idx.shape mask = torch.triu(torch.full((T, T), float(“-inf”), device=idx.device), diagonal=1) h = self.emb(idx) + self.pos(torch.arange(T, device=idx.device)) for blk in self.blocks: h = blk(h, mask) return self.head(self.ln_f(h)) model = (MiniGPT_TE() if TE_CAPABLE else MiniGPT_PT()).to(DEVICE) n_params = sum(p.numel() for p in model.parameters()) print(f”n>> Model: {‘TE fused’ if TE_CAPABLE else ‘pure PyTorch’} | ” f”{n_params/1e6:.1f}M params | {N_LAYERS} layers x {D_MODEL}d”) We define a compact causal language model using fused te.TransformerLayer blocks for Transformer Engine execution. We also implement an equivalent pure-PyTorch transformer architecture with multi-head attention, layer normalization, residual connections, and feed-forward networks. We select the appropriate model dynamically according to GPU support and report the final parameter count and architectural dimensions. Copy CodeCopiedUse a different Browser def make_batch(bsz=16): phase = torch.randint(0, VOCAB, (bsz, 1)) stride = torch.randint(1, 7, (bsz, 1)) steps = torch.arange(SEQ + 1).unsqueeze(0) seq = (phase + stride * steps) % VOCAB return seq[:, :-1].to(DEVICE), seq[:, 1:].to(DEVICE) opt = torch.optim.AdamW(model.parameters(), lr=3e-4) def run_step(x, y, use_fp8): if TE_CAPABLE and use_fp8: with te.fp8_autocast(enabled=True, fp8_recipe=fp8_recipe): logits = model(x) else: logits = model(x) loss = F.cross_entropy(logits.float().reshape(-1, VOCAB), y.reshape(-1)) opt.zero_grad(set_to_none=True) loss.backward() opt.step() return loss.item() print(f”n>> Training 60 steps ({‘FP8’ if FP8_CAPABLE else ‘BF16/FP32’})…”) t0 = time.time() for step in range(1, 61): x, y = make_batch() loss = run_step(x, y, use_fp8=FP8_CAPABLE) if step % 10 == 0: print(f” step {step:3d} | loss {loss:.4f} | ” f”{(time.time()-t0)/step*1000:.0f} ms/step”) print(f”>> Final loss: {loss:.4f} (random guess would be ~{math.log(VOCAB):.2f})”) We create deterministic arithmetic-pattern sequences that allow the model to learn predictable token transitions across the vocabulary. We configure the AdamW optimizer and implement a training step that conditionally wraps the forward pass in te.fp8_autocast when FP8 execution is supported. We train the model for multiple iterations, monitor the loss and step latency, and compare the final loss against the random-guess baseline. Copy CodeCopiedUse a different Browser def bench(use_fp8, iters=30, warmup=10): x, y = make_batch(bsz=32) for _ in range(warmup): run_step(x, y, use_fp8) torch.cuda.synchronize() torch.cuda.reset_peak_memory_stats() t = time.time() for _ in range(iters): run_step(x, y, use_fp8) torch.cuda.synchronize() ms = (time.time() – t) / iters * 1000 mem = torch.cuda.max_memory_allocated() / 1e9 return ms, mem print(“n>> Benchmark (batch 32, seq 256, fwd+bwd+optim):”) ms_hi, mem_hi = bench(use_fp8=False) print(f” {‘BF16’ if TE_CAPABLE else ‘FP32’}: {ms_hi:7.1f} ms/step | ” f”peak mem {mem_hi:.2f} GB”) if

PolyAI has introduced Dialog-RSN-1, a dialog model that perceives the caller’s audio directly instead of reading a transcript. It fuses turn-taking, speech recognition, function calling and response generation into one audio-native model, and is already handling live production calls. Key Takeaways Dialog-RSN-1 is audio-aware on the input side only; TTS stays separate, so the output voice remains controllable. It runs as a request-based LLM probed on demand, not an always-on stream that pins a GPU. Turn-taking is the model’s first output token: EMPTY, ONGOING or COMPLETE. PolyAI reports sub-300ms responses, +11% relative containment at a restaurant group, and −37% latency at an insurer. English only at launch, delivered through PolyAI’s platform rather than open weights or a public API. Is it deployable, and by whom? Yes, but only through PolyAI: no open weights, no public API yet. Existing customers can enable it today; new customers can request early access. Company level: large, high-call-volume enterprises. PolyAI reports 100+ enterprise customers and 2,000+ live deployments at its $86M Series D in December 2025. Self-serve developers and SMBs are not the target. Industries: restaurants, insurance, financial services, healthcare, hotels, retail, telecom, travel and utilities. Applications: booking and reservations, billing and payments, authentication, call routing, order management and troubleshooting. The architecture Two architectures dominate, and each concedes something. A cascaded stack sends only the ASR’s best guess to the LLM, so tone, hesitation and recognition uncertainty are gone before the LLM sees anything. Tuning means hand-adjusting end-pointing parameters and ASR biasing that rarely generalize across use cases. Speech-to-speech models such as GPT Realtime and Gemini Live keep the audio but bake the voice into the model, limiting pronunciation control, and always-on full-duplex variants pin a GPU for the entire call. Dialog-RSN-1 is audio-aware on input only: one model reasons over raw audio and hands generation to a separate, promptable TTS system. It is probed on demand rather than streamed: a high-recall VAD plus a few timers decide when to run it, and the first token of the reply settles whether the agent should speak. Cheap acoustic cues only choose when to ask; the model, with full context, makes the actual turn-taking call. How it was built PolyAI post-trained open-weight multimodal models with supervised and reinforcement finetuning on in-house data. The pipeline is broadly base-model agnostic; PolyAI evaluated Gemma, GPT-OSS, Qwen and Mistral. Targeting sub-300ms on A100 GPUs puts candidates in the 8B dense to 30B sparse range. Latency work includes prefilling the attention cache while the user speaks, an append-only prompt template to minimize cache invalidation, routing each caller to the same GPU, a finetuned speculative drafter with mean acceptance of 3.9 tokens, and auto-reasoning learned during RFT. Transcription runs last, after the response or tool call, in parallel with speech generation. Results PolyAI evaluated on Dialog-Eval, an internal benchmark it plans to open-source. Each example is a call truncated at one decision point, scoring a single atomic next step rather than a full rollout. PolyAI reports Dialog-RSN-1 as the highest-scoring real-time capable model, puts the cascaded Audio-score ceiling near 77, and notes GPT Realtime 2.1 scoring on par with cascades on audio-aware examples. On transcription, gpt-4o-transcribe’s WER improved from 7.8% to 6.9% once given the same context, with Dialog-RSN-1 lower still. For this release PolyAI focused on English; Raven 3.5 remains its recommendation for non-English and rich web chat. A technical report and a Dialog-Eval paper are planned. Check out the Technical details here. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post PolyAI Releases Dialog-RSN-1: An Audio-Native Dialog Model That Fuses Turn-Taking, Speech Recognition, Function Calling, And Response appeared first on MarkTechPost.

In this tutorial, we build and execute a multi-agent workflow with Omnigent using a reliable, isolated Python environment created with uv. We configure a financial research lead agent that retrieves a live USD-to-EUR exchange rate from an external API, prepares a concise client-ready summary, and delegates its draft to a dedicated text-auditing sub-agent for clarity and length validation. We define reusable Python functions as callable agent tools, describe the complete agent structure in YAML, and use the Claude Agent SDK as the execution harness. We also manage the Anthropic API key securely through environment variables, apply non-interactive policies that limit tool calls and control session costs, and run the workflow directly from Colab without requiring Node.js, tmux, or an interactive terminal. Through this implementation, we explore how Omnigent combines agents, tools, delegation, live data access, and governance within a single configurable system. Copy CodeCopiedUse a different Browser import os, sys, subprocess, textwrap, pathlib, getpass def sh(cmd, **kw): “””Run a command, and on failure show the ACTUAL error, not just a code.””” print(“$”, ” “.join(map(str, cmd))) p = subprocess.run(cmd, text=True, capture_output=True, **kw) if p.returncode != 0: print(p.stdout or “”, p.stderr or “”, sep=”n”) raise RuntimeError(f”Command failed ({p.returncode}): {‘ ‘.join(map(str, cmd))}”) return p WORKDIR = pathlib.Path(“/content/omnigent_tutorial”) WORKDIR.mkdir(parents=True, exist_ok=True) VENV = WORKDIR / “.venv” subprocess.run([sys.executable, “-m”, “pip”, “install”, “-q”, “uv”], check=True) if not (VENV / “bin” / “python”).exists(): sh([“uv”, “venv”, “–python”, “3.12”, str(VENV)]) PY = str(VENV / “bin” / “python”) sh([“uv”, “pip”, “install”, “–python”, PY, “-q”, “omnigent”, “requests”]) OMNI = str(VENV / “bin” / “omnigent”) print(“n”, subprocess.run([OMNI, “–version”], capture_output=True, text=True).stdout.strip()) We import the required Python modules and define a helper function that executes shell commands while displaying detailed error information when a command fails. We create a dedicated working directory and use uv to build an isolated Python 3.12 virtual environment that avoids Colab’s ensurepip limitation. We then install Omnigent and Requests inside the environment, locate the Omnigent CLI executable, and verify the installation by printing its version. Copy CodeCopiedUse a different Browser if not os.environ.get(“ANTHROPIC_API_KEY”): os.environ[“ANTHROPIC_API_KEY”] = getpass.getpass(“Anthropic API key: “) env = os.environ.copy() env[“OMNIGENT_NO_UPDATE_CHECK”] = “1” We securely collect the Anthropic API key only when it is not already available in the notebook environment. We store the credential in the current process environment so that Omnigent can detect it without writing sensitive information to a file. We also create a separate environment configuration for the subprocess and turn off Omnigent’s automatic update check during execution. Copy CodeCopiedUse a different Browser (WORKDIR / “agent_tools.py”).write_text(textwrap.dedent(”’ “””Local tools exposed to the Omnigent agents in this tutorial.””” import requests def get_exchange_rate(base_currency: str, target_currency: str) -> dict: “””Look up the latest FX rate between two ISO-4217 currency codes.””” r = requests.get( “https://api.frankfurter.app/latest”, params={“from”: base_currency.upper(), “to”: target_currency.upper()}, timeout=10, ) r.raise_for_status() data = r.json() return { “base”: base_currency.upper(), “target”: target_currency.upper(), “rate”: data[“rates”][target_currency.upper()], “date”: data[“date”], } def word_count(text: str) -> int: “””Count the words in a piece of text.””” return len(text.split()) ”’)) We generate a Python module containing the local functions that Omnigent exposes as callable tools to the agents. We define a live exchange-rate tool that sends a request to the Frankfurter API and returns the latest rate, currency codes, and applicable date. We also implement a simple word-count tool that allows the auditing sub-agent to measure the length of the financial summary. Copy CodeCopiedUse a different Browser (WORKDIR / “fx_research_lead.yaml”).write_text(textwrap.dedent(”’ name: fx_research_lead prompt: | You are a financial research lead. For any question about currency movements: call get_exchange_rate to fetch the live rate, then hand your draft summary to the text_auditor sub-agent for a clarity and length check before giving your final answer to the user. executor: harness: claude-sdk tools: get_exchange_rate: type: function callable: agent_tools.get_exchange_rate text_auditor: type: agent prompt: | You audit short pieces of financial writing. Call word_count to report its length, flag any unexplained jargon, and suggest one concrete clarity improvement. tools: word_count: type: function callable: agent_tools.word_count policies: cap_calls: type: function handler: omnigent.policies.builtins.safety.max_tool_calls_per_session factory_params: limit: 20 budget: type: function handler: omnigent.policies.builtins.cost.cost_budget factory_params: max_cost_usd: 1.00 ”’)) We define the complete multi-agent architecture through a YAML configuration file. We configure the financial research lead, connect it to the exchange-rate tool, and add a text-auditing sub-agent that evaluates the draft using the word-count function. We also apply hard governance policies that restrict the number of tool calls and limit the maximum API cost for the session. Copy CodeCopiedUse a different Browser env[“PYTHONPATH”] = str(WORKDIR) question = ( “What is the current USD to EUR exchange rate? Give me a two-sentence ” “summary I could paste into a client note.” ) result = subprocess.run( [OMNI, “run”, str(WORKDIR / “fx_research_lead.yaml”), “-p”, question, “–no-session”], cwd=WORKDIR, env=env, stdin=subprocess.DEVNULL, capture_output=True, text=True, timeout=300, ) print(“n” + “=” * 70) print(result.stdout.strip() or “(no stdout)”) if result.returncode != 0 or “error” in result.stdout.lower(): print(“-” * 70) print(“stderr:”, result.stderr[-2000:]) print(f”nDebug: check ~/.omnigent/logs/runner/ , or rerun with:n” f” !{OMNI} –debug –log-to-stderr run {WORKDIR/’fx_research_lead.yaml’} -p “…” –no-session”) print(“=” * 70) print(f””” Next steps: • Explore the CLI: !{OMNI} run –help • Bundled demo agents: !{OMNI} polly -p “review this repo” –no-session !{OMNI} debby -p “brainstorm 3 names for a coffee shop” –no-session • YAML schema: https://github.com/omnigent-ai/omnigent/blob/main/docs/AGENT_YAML_SPEC.md • Policies: https://github.com/omnigent-ai/omnigent/blob/main/docs/POLICIES.md “””) We add the tutorial directory to PYTHONPATH, define the currency-related question, and execute the Omnigent agent through a non-interactive subprocess. We capture the generated response, display diagnostic output when execution fails, and provide a debug command for examining runner issues. We finish by printing useful next steps for exploring Omnigent’s CLI, bundled agents, YAML specification, and policy documentation. In conclusion, we created a practical Omnigent multi-agent application that integrates live financial data retrieval, hierarchical agent delegation, automated writing assessment, and policy-based execution controls. We used uv to solve Colab’s ensurepip limitation and maintain a separate Python 3.12 environment without modifying the notebook’s system interpreter. We exposed local Python functions as agent-accessible tools, defined the agent and sub-agent behavior through a readable YAML configuration, and enforced hard limits on tool usage and API spending. We also executed the workflow non-interactively, captured both standard output and diagnostic errors, and established

JetBrains Research Open-Sources KotlinLLM. KotlinLLM is an IntelliJ IDEA plugin for Kotlin/JVM projects that adds a language feature called Smart macros. A Smart macro is a regular Kotlin function call whose body is generated Kotlin code. The public API is deliberately small. asLlm<F, T>(from, hint) converts an input of type F into a typed value T, such as a data class, enum, list, or primitive. mockLlm<T>() generates a stateful implementation of an interface T, whose behavior depends on which methods are called on it. Copy CodeCopiedUse a different Browser val issuesApiUrl: String = asLlm(repoInput, hint = “GitHub API URL: get all issues, including closed”) val issues: List<Issue> = asLlm(response, hint = “Return all beginner-friendly issues for this repository”) The runtime loop When a project launches through the KotlinLLM run configuration, the plugin scans for asLlm and mockLlm calls, updates generated bootstrap/provider/parser/mock files, launches the run configuration under JDI, and registers breakpoints on generated regenerate hooks. If generated logic does not match a runtime scenario, execution reaches a hook. The plugin captures runtime values and type information from the suspended frame, the LLM agent submits a code update, and the plugin compiles it and redefines the loaded class before retrying the original call. KotlinLLM targets Kotlin/JVM specifically because the runtime evolution loop depends on JVM class redefinition through JDI. Explainer: how a Smart macro evolves The embed below walks the macro API, animates the nine-step runtime loop, and models why covered scenarios stop costing inference calls. Reported results On an adapted Spring Petclinic Kotlin project with 18 asLlm call sites, 24 of 24 application scenarios completed after Smart macro evolution, with a 100% hot-reload success rate and compilation/redefinition adding roughly 1% of total runtime overhead. A synthetic “GitHub Beginner Issue Radar” parsed real issue data across 20 repositories and 30k+ issues, reaching about 0.89 recall on ground-truth beginner labels. Setup requirements The plugin requires IntelliJ IDEA 2025.2.x, JDK 21, and an OpenAI API key stored in the target project’s .kotlinllm file via Tools > KotlinLLM Settings. It is released under the Apache License 2.0, with runnable examples, the thesis write-up, and the KotlinConf 2026 talk recording in the repository. Is it deployable? Not as a production runtime, at least not yet. JetBrains labels KotlinLLM a research prototype, and it is described it as an experimental IntelliJ IDEA plugin. The plugin is experimental, but its output is deployable. Once behavior has been generated, the target project can compile and run that behavior without another LLM request for the same scenario. You ship plain Kotlin, not a model dependency. Company level: best fit today is R&D groups, platform teams at mid-size to large Kotlin/JVM entities, and startups with tolerance for prototype tooling. Regulated enterprises should treat generated sources as reviewable code, which is exactly how KotlinLLM stores them. Industries: fintech and banking (heavy JVM/Kotlin estates), developer tooling, e-commerce, logistics, and any team parsing messy third-party API payloads. Applications: normalizing semi-structured API responses into typed values, building evolving test doubles, adapting to upstream schema drift, and classification over noisy text fields. Key Takeaways KotlinLLM is a JetBrains Research prototype, not a production runtime. Smart macros generate Kotlin source that is committed, reviewed, and run without the plugin. Covered scenarios trigger no further LLM call, so no added latency or cost. Petclinic evaluation: 24/24 scenarios, 100% hot-reload, ~1% overhead. Apache 2.0, Kotlin/JVM only, IntelliJ IDEA 2025.2.x plus JDK 21. Sources: JetBrains Research blog, the kotlinllm-plugin README, and InfoWorld The post JetBrains Open-Sources KotlinLLM: Smart Macros That Generate Kotlin Source Code at Runtime and Hot-Reload It Through JDI appeared first on MarkTechPost.