Committee

arXiv:2607.19336v1 Announce Type: cross Abstract: Agentic systems large language model (LLM) based architectures capable of reasoning, planning, acting, and coordinating with tools and other agents are rapidly transitioning from research prototypes to production scale deployments across domains such as software engineering, scientific discovery, and finance. While academic work has emphasized benchmarks and algorithmic innovation, deployment raises new challenges around robustness, safety, and reliability. This tutorial brings together researchers and practitioners to explore advances in reasoning and planning, multi agent coordination, and evaluation, highlighting open challenges arising from deployment experience. Through applied case studies in pharmaceutical discovery and financial systems, we analyze common design patterns that make agentic systems successful, and discuss practical mitigation strategies for failure modes, such as verification pipelines, fallback mechanisms, and human in the loop supervision. Attendees will gain a comprehensive view of the field along with concrete design patterns, evaluation checklists, and templates for safe and reliable deployment across industries.

Cisco Foundation AI has released Antares, a family of security small language models (SLMs) built for one narrow security task. The task is vulnerability localization. Given a vulnerability description and a repository, find the files containing the flaw. Two models are open-weight and available now on Hugging Face, Antares-350M and Antares-1B. Both are Apache 2.0. Cisco team also shipped the Vulnerability Localization Benchmark (VLoc Bench), a 500-task agentic evaluation, under the same license. The main result is not a new state of the art. It is that a 1B model reaches 0.209 File F1. GPT-5.5 reaches 0.229, and a 753B open-weight model reaches 0.186. The problem Antares is scoped to Software security depends on connecting external vulnerability knowledge to internal source code. That knowledge lives in public databases, advisories, and Common Weakness Enumerations. The code lives in repositories that are large, modular, and dependency-rich. Connecting the two is expensive. Devs search unfamiliar code, follow naming conventions, inspect call paths, and compare candidate files. Cisco’s framing is that this first triage step is where the cost concentrates. Antares does not replace the application security toolchain. Cisco is explicit about this. Dev teams still need dependency scanning, secret scanning, dynamic testing, container checks, threat modeling, and expert review. Understanding the Models Antares consists of three decoder-only transformers at 350M, 1B, and 3B parameters. All three initialize from IBM Granite 4.0 checkpoints. They share a tokenizer and architecture: grouped-query attention, SwiGLU MLPs, RMSNorm, RoPE, and shared input/output embeddings. Model Params Base checkpoint Context Layers / hidden / KV heads Status Antares-350M 350M Granite 4.0 350M 32K 28 / 1024 / 4 Open weights Antares-1B 1.6B Granite 4.0 1B 128K 40 / 2048 / 4 Open weights Antares-3B 3B Granite 4.0 Micro 128K Not published Not released How the Agent Loop Works Antares is not evaluated as a standalone sequence model. It runs inside a constrained loop with three tools. The model receives a CWE category description and nothing else. No advisory text, no file hints, no severity details. It then issues read-only terminal commands against a Docker sandbox with networking disabled. Command output is truncated to 2,000 characters before entering the transcript. The budget is 15 terminal calls per task. The model terminates by calling submit_vulnerable_files with a ranked list, or submit_no_vulnerability_found. The submission itself does not count against the budget. Output is a ranked list of file paths plus the exploration trace that produced it. What VLoc Bench measures VLoc Bench draws 500 tasks from 290 unique real-world repositories. Sources are public GitHub Security Advisories across six ecosystems: npm, pip, Maven, Go, Rust, and Composer. It covers 147 unique CWE categories, and 78% of entries carry assigned CVE identifiers. Ground truth is derived from the security patch. Files modified in the fix are labels, with tests, docs, and configuration excluded. The benchmark has two phases: Phase A gives the model the vulnerable snapshot and scores File F1. Phase B gives the patched snapshot and scores True Negative Rate, testing whether the model raises a false alarm on fixed code. Results: task-specific training beats parameter scale The pattern in the data is a capability cliff, not a scaling curve. Antares-3B reaches 0.223 File F1, just under GPT-5.5 (xhigh) at 0.229. Antares-1B reaches 0.209, above GLM-5.2 at 753B parameters, which scores 0.186. Antares-350M reaches 0.135, above Gemma-4-31B at 0.101 and Gemini 2.5 Flash at 0.102. Antares-1B also records the highest recall of any evaluated system at 0.224. Static analysis tools were run under the same evaluation. Semgrep scores 0.086 File F1, CodeQL scores 0.023, and Horusec scores 0.020. Cisco’s reading is that rule-based scanners recover some vulnerable files but cannot adaptively inspect repository context. Where the capability comes from The untrained Granite 4.0 base checkpoints score 0.001, 0.000, and 0.000 File F1 under the identical protocol. They have tool-calling ability and still produce degenerate output inside an agentic loop. Supervised fine-tuning does the heavy lifting. It lifts the three scales to 0.108, 0.188, and 0.198. The SFT corpus is 71.5% cybersecurity reasoning, 15.4% code search trajectories, and 13.1% deep research and general reasoning. All reasoning traces come from a single teacher, GPT-OSS-120B, to avoid cross-teacher distribution shift. GRPO then adds 11% to 25%, with the largest relative gain at 350M. Rewards are verifiable and computed programmatically from trajectory text, with no learned reward model. Components cover localization quality, submission behavior, tool-use compliance, exploration, and malformed-output penalties. The variance effect may matter more than the mean. GRPO cuts run-to-run standard deviation by 42% to 65%. One GRPO evaluation run is a more reliable estimate than one SFT run. There is also a scale-dependent split in learned strategy. After GRPO, the 350M and 1B models use 87% to 89% search commands and submit more files. The 3B model settles at 52% search and 37% read, and submits fewer files at higher precision. The reward never prescribed either policy. Deployment Key Takeaways Antares-1B hits 0.209 File F1 on VLoc Bench, above GLM-5.2 at 753B parameters and Gemini 3 Pro. The Granite 4.0 base checkpoints score ~0.000 under the same protocol, so post-training supplies essentially all the capability. GRPO adds 11-25% File F1 and cuts run-to-run variance 42-65%, which matters more for repeatable CI scans. A full 500-task sweep costs under $1 on one H100, against $12.50 for GLM-5.2 and $141 for GPT-5.5. The strongest variant, Antares-3B, is not released, and Antares has no published Phase B false-alarm numbers. Check out the Models on Hugging Face, Benchmark, GitHub Repo and Mentioned Technical Report. The post Cisco Foundation AI Releases Antares: 350M and 1B Open-Weight Models That Localize Known Vulnerabilities Inside Real Codebases appeared first on MarkTechPost.

Four open source projects dominate LLM fine-tuning today. Unsloth, Axolotl, TRL, and LLaMA-Factory all wrap the same underlying PyTorch and Hugging Face stack. They diverge on where they spend engineering effort. Unsloth rewrites kernels. Axolotl composes parallelism strategies. TRL defines the trainer APIs the others build on. LLaMA-Factory optimizes for breadth of model coverage and zero-code operation. This comparison covers three axes engineers actually hit: training throughput, peak VRAM, and multi-GPU scaling. Understand each framework TRL is the reference implementation layer. It ships SFTTrainer, DPOTrainer, GRPOTrainer, KTOTrainer, RewardTrainer, and RLOOTrainer. Axolotl and LLaMA-Factory both call into it. The current stable release line is v1.8.0. Unsloth replaces parts of the modeling code with hand-written Triton kernels. Backpropagation steps are manually derived rather than autograd-generated. Hugging Face’s own writeup notes accuracy degradation is 0% versus standard QLoRA, because no approximations are introduced. Axolotl is a YAML-driven wrapper over Transformers, PEFT, TRL, Accelerate, and DeepSpeed. Its differentiator is composability of parallelism strategies, not kernel work. LLaMA-Factory is an ACL 2024 system demonstration paper with a Gradio web UI called LlamaBoard. The repository covers 100+ LLMs and VLMs. Speed Unsloth: kernel-level gains on a single GPU Unsloth’s published benchmarks show 2x training speed for Llama 3.1 8B and Llama 3.3 70B. The setup used the Alpaca dataset, batch size 2, and gradient accumulation 4. QLoRA ran at rank 32 on all linear layers. The MoE results are larger. Unsloth fine-tuned unsloth/gpt-oss-20b-BF16 on an NVIDIA B200. It reports 712.33 ms per step at 8K context, versus 5,226.86 ms for Transformers v5. That is a 7.3x gap. At 4K the gap is 4.82x, and at 1K it is only 1.37x. The trend direction is model-dependent, and Unsloth’s docs scope this claim to gpt-oss. There, the speedup grows with sequence length, credited to Flex Attention and the MoE kernels. Qwen3-30B-A3B on B200 runs the other way. Its reported speedup falls from 1.7x at 1K to 1.1x at 16K. Memory savings move the opposite direction, rising from about 2% to 15%. Qwen3-30B-A3B on H100 reaches up to 1.77x. GLM-4.7-Flash on RTX PRO 6000 reaches 2.1x. A collaboration with AMD measured Llama-3.1-8B LoRA SFT at 2.07 s/step. TRL plus FlashAttention-2 took 2.87 s/step, a 1.39x gap with matching loss curves. Axolotl: kernels borrowed, parallelism native Axolotl added custom Triton kernels and autograd functions for LoRA in February 2025, explicitly citing Unsloth as inspiration. They are opt-in through lora_mlp_kernel, lora_qkv_kernel, and lora_o_kernel. Recent release notes add SonicMoE LoRA support. It delivers up to 1.45x speedup and 30% memory reduction over a grouped_mm baseline. That figure is for Qwen3.5-35B-A3B 8-bit LoRA on a single H100 SXM. Axolotl also ships FlashAttention 2/3/4, xFormers, Flex Attention, SageAttention, Liger Kernel, Cut Cross Entropy, and ScatterMoE. TRL: the baseline everyone measures against TRL is usually the reference point rather than the winner on raw single-GPU throughput. It compensates with breadth of memory and speed levers documented in Reducing Memory Usage and Speeding Up Training. Those levers include packing, padding-free batching, truncation, Liger Kernel, and vLLM sleep mode for GRPO. TRL also has a first-party Unsloth integration, so the two are not mutually exclusive. LLaMA-Factory: speed by delegation LLaMA-Factory does not write its own kernels. It exposes other people’s work through config flags. Setting use_unsloth: true activates the Unsloth patch. The project’s changelog reports 170% relative speed from that path. Unsloth’s long-sequence training is listed at 117% speed and 50% memory. It also supports enable_liger_kernel: true and FlashAttention-2 via flash_attn: fa2. VRAM Reported memory floors Unsloth publishes a VRAM requirements table sorted by parameter count. It lists 6 GB for an 8B model in 4-bit QLoRA and 41 GB for 70B. LoRA at 16-bit costs 22 GB and 164 GB for the same models. LLaMA-Factory’s README hardware table covers the same regime for 4-bit QLoRA. It lists 6 GB at 7B, 24 GB at 30B, and 48 GB at 70B. Full bf16 fine-tuning of 70B is listed at 600 GB. Both tables describe minimums. Batch size, sequence length, and optimizer choice move the real number. Context length is the sharper differentiator Peak VRAM at a fixed context is less interesting than the maximum context a given VRAM budget allows. Unsloth’s context length benchmarks for Llama 3.1 8B QLoRA at rank 32 and batch size 1 are stark. GPU VRAM Unsloth context Transformers + FA2 context 8 GB 2,972 OOM 16 GB 40,724 2,551 24 GB 78,475 5,789 48 GB 191,728 15,502 80 GB 342,733 28,454 Unsloth attributes this to its gradient checkpointing algorithm combined with Apple’s Cut Cross Entropy. For Llama 3.3 70B on an 80 GB A100, it reports 89,389 tokens. The FA2 baseline reaches 6,916. The MoE memory story MoE training is where memory behavior has shifted most in 2026. Unsloth reports gpt-oss-20b fine-tuning inside 12.8 GB, while Qwen3-30B-A3B at 16-bit LoRA needs 63 GB. Its B200 gpt-oss run used 47.43 GB at 8K context where Transformers v5 used 73.80 GB. At 16K, Transformers v5 went out of memory and Unsloth used 55.13 GB. The mechanism is a split-LoRA formulation. PEFT materializes the LoRA delta across all experts before the MoE matmul. Unsloth reorders the operations instead, which is mathematically identical but avoids the materialization. Axolotl attacks the same problem differently. Its MoE expert quantization quantizes expert weights during model loading, freeing the original bf16 tensor immediately. The reason is a Transformers v5 change. Expert layers moved from nn.Linear to fused nn.Parameter 3D tensors. bitsandbytes could no longer quantize them on load. Axolotl’s docs report GLM-4.7-Flash QLoRA dropping from roughly 127 GiB to roughly 23 GiB reserved memory with quantize_moe_experts: true. Multi-GPU This is where the ranking inverts. Unsloth’s single-GPU lead does not carry over. Axolotl: the deepest parallelism matrix Axolotl’s multi-GPU guide offers three mutually exclusive sharding strategies: DeepSpeed ZeRO stages 1 through 3, FSDP, and DDP. FSDP2 is the recommended path, and FSDP1 is deprecated. On top of those, its N-D Parallelism guide composes data, tensor, context, and expert parallelism through PyTorch’s DeviceMesh. The documented support matrix confirms FSDP+TP, HSDP+TP, FSDP+CP,

When NASA’s Nancy Grace Roman Space Telescope launches, as early as the end of next month, it will attempt one of astronomy’s most precise disappearing acts to date. The telescope will carry the first space-bound “active” coronagraph, an instrument that effectively erases most of the light from a star during photography. It will allow astronomers to take the first pictures of planets orbiting other stars that are similar to those in our solar system. Ultimately, it could pave the way for a future mission that could snap the first photos of Earth-like worlds. “I hope it’s remembered for it being that critical stepping stone for … finding Earth 2.0,” says Brandon Creager, the instrument’s lead mechanical engineer at NASA’s Jet Propulsion Laboratory (JPL). Named after Nancy Grace Roman, NASA’s first chief of astronomy, this new telescope will carry a roughly 300-megapixel wide-field camera that will enable it to capture images about 100 times larger than the Hubble Space Telescope’s widest exposures at a similar resolution. These capabilities will help astronomers unpack the mysterious identities of dark matter and dark energy—and to detect around 100,000 new exoplanets, planets outside our solar system, whose presence can be inferred from the way they distort the starlight of more distant stars. Javier Viaña, a research scientist at Harvard who has had two projects selected for Roman’s highly competitive first year of observing, compares the leap to moving from “interviewing a handful of people” to “conducting a global census.” Another camera will use the coronagraph, blocking out a star’s light as it observes one stellar system at a time. The instrument will allow astronomers an unprecedented look at the space around stars, enabling them to see smaller, dimmer, and more close-in exoplanets. “It’s giving us the ability to see planets that we haven’t been able to physically see before,” says Creager. The anatomy of a vanishing trick Coronagraphs in space aren’t new. But earlier incarnations, such as those currently aboard Hubble and the James Webb Space Telescope, use a stationary system to block a star’s blinding light. The approach does help, but it’s a bit like putting your thumb over a flashlight while searching a dark room for a firefly. Though the bulb vanishes, stray glare can still escape and overwhelm the light of the insect. Inside a telescope, that glare can come from light leaking around the edges of machinery or from minuscule imperfections in mirrors and coatings that can scatter starlight into speckles. All this can hide, or even impersonate, a planet. Roman’s coronagraph, however, will attempt something completely unseen in space telescopes until this year: Before each observation, it will measure that leftover light and try to suppress it, a technique known as active wavefront control. The telescope is able to do this because it contains two deformable mirrors. Each has a 48-by-48 checkerboard of actuators (tiny pistons) beneath a thin, deformable sheet of glass. Applying a small amount of voltage makes the actuators contract and tug their patches of mirror slightly backward, like thousands of microscopic fingers delicately sculpting a surface. The effect is very subtle: Each patch of mirror can deform by up to 0.5 micrometers, or about one-fourth the size of an E. coli bacterium, and in increments as small as approximately 10 picometers. That’s about a tenth the diameter of a hydrogen atom, says Ilya Poberezhskiy, the instrument’s project systems engineer at JPL. The actuators allow the mirrors to create an “active wavefront,” where each component is moved to the perfect position to cancel out incoming waves of unwanted light—a bit like a pair of noise-canceling headphones, but for light instead of sound. The “canceled-out” light creates a “doughnut-shaped region around the star where we suppress starlight and where we’re hoping to see exoplanets,” says Poberezhskiy. Compared with current space-based coronagraphs, the system is expected to improve sensitivity to exoplanets against the glare of their host stars by a factor of up to 1,000, revealing planets that would have been far too faint to detect before. Like Hubble and JWST, Roman also uses masks, patterned plates placed in the path of the light that are designed to block the photons that run into them. One tool in Roman’s mask arsenal is “silicon grass,” a thicket of microscopic spikes on some masks that can be used in certain configurations to absorb photons so they don’t bounce around the telescope and accidentally reach a detector. Light entering the forest bounces deeper and deeper between the blades and gets trapped instead of reflecting back toward the camera. “Once the light gets into there, it never gets out,” Poberezhskiy says. The mirrors and masks form a succession of gates and hedges to guide as much of the preserved planetary light as possible toward the final detector. Alien Jupiters This elaborate setup could open a new chapter in the direct imaging of exoplanets. Nearly all exoplanets photographed so far are oversize youngsters that are nothing like the residents of our solar system: several times the mass of Jupiter, still glowing with the heat left over from their birth, and orbiting tens or hundreds of times farther from their star than the Earth is from the sun. This is because they are relatively easy to see. Their size, warmth, and distance from their parent star makes them shine brightly in infrared light, far away from the worst of the stellar glare. Roman, however, could directly image a true Jupiter analogue—a planet similar to Jupiter in mass and circling a sunlike star a few times farther out than Earth is from our sun. Unlike the hot Jupiters we can see now, this one would be a much more mature gas giant like ours, primarily reflecting its parent star’s light after billions of years of cooling instead of heavily emitting its own. Astronomers have been able to infer the existence of such planets from the gravitational wobble they impart to the star. Roman instead will collect starlight reflected from the planet itself. “We’re not looking

In this tutorial, we explore NVIDIA’s srt-slurm framework and learn how we use srtctl to convert declarative YAML configurations into reproducible SLURM benchmark workflows for distributed LLM serving. We set up the project in Google Colab, inspect its internal architecture, define a cluster configuration, dry-run built-in and custom recipes, and model a disaggregated prefill-and-decode deployment for DeepSeek-R1. We also generate parameter sweeps, interact with the typed Python API, validate expanded configurations, and analyze simulated benchmark results through a throughput-versus-latency Pareto frontier. Although Colab does not provide a real SLURM environment, we use it as a practical development workspace to understand, validate, and prepare production-grade benchmark recipes before we submit them to an actual GPU cluster. Copy CodeCopiedUse a different Browser import os, sys, subprocess, textwrap, json, shutil, importlib from pathlib import Path def run(cmd, check=True, quiet=False): “””Run a shell command, stream output.””” print(f”n$ {cmd}”) r = subprocess.run(cmd, shell=True, text=True, capture_output=True) out = (r.stdout or “”) + (r.stderr or “”) if not quiet: print(out[-6000:]) if check and r.returncode != 0: raise RuntimeError(f”Command failed ({r.returncode}): {cmd}”) return out def section(title): print(“n” + “═”*78 + f”n {title}n” + “═”*78) section(“1. Install srt-slurm”) REPO = Path(“/content/srt-slurm”) if Path(“/content”).exists() else Path.cwd()/”srt-slurm” if not REPO.exists(): run(f”git clone –depth 1 https://github.com/NVIDIA/srt-slurm.git {REPO}”, quiet=True) run(f”{sys.executable} -m pip install -q -e {REPO}”, quiet=True) sys.path.insert(0, str(REPO / “src”)) importlib.invalidate_caches() os.chdir(REPO) run(“srtctl –help”) We prepare the Colab environment by importing the required modules and defining reusable helper functions for command execution and section formatting. We clone the NVIDIA srt-slurm repository, install it in editable mode, and expose its source directory to the active Python runtime. We then switch to the repository directory and verify that the srtctl command-line interface is installed correctly. Copy CodeCopiedUse a different Browser section(“2. Repository architecture”) print(textwrap.dedent(“”” src/srtctl/ cli/ submit.py (apply/dry-run/preflight/monitor), do_sweep, interactive core/ schema.py (typed config), sweep.py, slurm.py (sbatch gen), validation.py, health.py, topology.py, fingerprint.py backends/ sglang.py, trtllm.py, vllm.py, mocker.py ← engine adapters frontends/ Dynamo / router frontends templates/ Jinja2 → sbatch + orchestrator scripts recipes/ ready-made benchmarks per platform (gb200-fp4, h100, b200-fp8, qwen3-32b, dsv4-pro, mocker smoke tests, …) analysis/ srtlog (log parsers) + Streamlit dashboard (Pareto, latency…) docs/ sweeps.md, profiling.md, analyzing.md, config-reference.md “””)) for d in [“recipes”, “docs”]: print(f”{d}/ →”, “, “.join(sorted(p.name for p in (REPO/d).iterdir()))[:300]) section(“3. Cluster configuration (srtslurm.yaml)”) (REPO/”srtslurm.yaml”).write_text(textwrap.dedent(“”” cluster: “colab-demo” default_account: “demo-account” default_partition: “gpu” default_time_limit: “01:00:00” gpus_per_node: 4 use_gpus_per_node_directive: true use_segment_sbatch_directive: true containers: dynamo-sglang: “/containers/dynamo-sglang.sqsh” lmsysorg+sglang+v0.5.5.post2.sqsh: “/containers/sglang-v0.5.5.sqsh” model_paths: deepseek-r1: “/models/DeepSeek-R1” “””)) print((REPO/”srtslurm.yaml”).read_text()) We inspect the repository structure to understand how srtctl organizes its command-line tools, schemas, backends, templates, recipes, and analysis components. We then create a local srtslurm.yaml file containing simulated cluster defaults, container aliases, GPU settings, and model paths. We use this configuration to resolve recipe references in Colab without requiring access to an actual SLURM cluster. Copy CodeCopiedUse a different Browser section(“4. Dry-run: mocker smoke test → generated sbatch script”) run(“srtctl dry-run -f recipes/mocker/agg.yaml”, check=False) section(“5. Custom disaggregated recipe (prefill/decode split)”) (REPO/”my-disagg.yaml”).write_text(textwrap.dedent(“”” name: “colab-disagg-demo” model: path: “deepseek-r1” container: “lmsysorg+sglang+v0.5.5.post2.sqsh” precision: “fp8” resources: gpu_type: “gb200” gpus_per_node: 4 prefill_nodes: 1 decode_nodes: 2 prefill_workers: 1 decode_workers: 2 backend: prefill_environment: { PYTHONUNBUFFERED: “1” } decode_environment: { PYTHONUNBUFFERED: “1” } sglang_config: prefill: served-model-name: “deepseek-ai/DeepSeek-R1” model-path: “/model/” trust-remote-code: true kv-cache-dtype: “fp8_e4m3” tensor-parallel-size: 4 disaggregation-mode: “prefill” decode: served-model-name: “deepseek-ai/DeepSeek-R1” model-path: “/model/” trust-remote-code: true kv-cache-dtype: “fp8_e4m3” tensor-parallel-size: 4 disaggregation-mode: “decode” benchmark: type: “sa-bench” isl: 1024 osl: 1024 concurrencies: [64, 128, 256] req_rate: “inf” “””)) run(“srtctl dry-run -f my-disagg.yaml”, check=False) We dry-run the built-in mocker recipe to examine how srtctl validates configurations and generates SLURM submission artifacts without executing a real benchmark. We then define an advanced DeepSeek-R1 recipe that separates prefill and decode workloads across independent node and worker pools. We validate this disaggregated SGLang configuration through another dry run and inspect how the serving parameters are translated into job scripts. Copy CodeCopiedUse a different Browser section(“6. Parameter sweep (grid search) — dry-run + expansion on disk”) run(“srtctl dry-run -f examples/example-sweep.yaml”, check=False) sweep_dirs = sorted((REPO/”dry-runs”).glob(“example-sweep_sweep_*”)) if sweep_dirs: latest = sweep_dirs[-1] print(“Per-job configs generated by the sweep expander:”) for p in sorted(latest.rglob(“config.yaml”)): print(” “, p.relative_to(REPO)) section(“7. Programmatic use of srtctl’s Python API”) import yaml from srtctl.core.config import load_config from srtctl.core.sweep import generate_sweep_configs, expand_template from srtctl.core.schema import BenchmarkType, Precision, GpuType cfg = load_config(“my-disagg.yaml”) print(f”Loaded : {cfg.name}”) print(f”Model : {cfg.model.path} ({cfg.model.precision}) on {cfg.resources.gpu_type}”) print(f”Layout : {cfg.resources.prefill_nodes}P + {cfg.resources.decode_nodes}D nodes, ” f”{cfg.resources.gpus_per_node} GPUs/node”) print(f”Bench : {cfg.benchmark.type} isl={cfg.benchmark.isl} osl={cfg.benchmark.osl} ” f”concurrencies={cfg.benchmark.concurrencies}”) print(f”Enums : benchmarks={[b.value for b in BenchmarkType]}”) print(f” precisions={[p.value for p in Precision]}, gpus={[g.value for g in GpuType]}”) raw_sweep = yaml.safe_load(Path(“examples/example-sweep.yaml”).read_text()) jobs = generate_sweep_configs(raw_sweep) print(f”nSweep expands to {len(jobs)} jobs:”) for job_cfg, params in jobs: pf = job_cfg[“backend”][“sglang_config”][“prefill”] print(f” {params} → chunked-prefill-size={pf[‘chunked-prefill-size’]}, ” f”max-total-tokens={pf[‘max-total-tokens’]}”) print(“nTemplate substitution:”, expand_template({“flag”: “{x}”, “n”: “{y}”}, {“x”: 4096, “y”: 2})) We execute the example parameter sweep and inspect the individual job configurations created from its Cartesian search space. We load our custom recipe through the typed Python API and examine its model, precision, GPU topology, benchmark settings, and supported enumeration values. We also programmatically expand sweep templates and verify how each parameter combination affects the generated backend configuration. Copy CodeCopiedUse a different Browser section(“8. Analysis: Pareto frontier from (simulated) benchmark results”) import numpy as np, matplotlib.pyplot as plt rng = np.random.default_rng(0) def simulate(variant, base_tps, base_itl): rows = [] tps_gpu = base_tps * c / (c + 90) * rng.uniform(.97, 1.03) itl = base_itl * (1 + c/220) * rng.uniform(.97, 1.03) rows.append({“variant”: variant, “concurrency”: c, “tok_s_gpu”: tps_gpu, “itl_ms”: itl}) return rows results = simulate(“chunked=4096”, 260, 9.5) + simulate(“chunked=8192”, 300, 11.5) print(json.dumps(results[:3], indent=2), “…”) plt.figure(figsize=(8, 5)) for variant in (“chunked=4096”, “chunked=8192”): pts = [(r[“itl_ms”], r[“tok_s_gpu”], r[“concurrency”]) for r in results if r[“variant”] == variant] xs, ys, cs = zip(*pts) plt.plot(xs, ys, “o-“, label=variant) for x, y, c in pts: plt.annotate(str(c), (x, y), fontsize=7, xytext=(3, 3), textcoords=”offset points”) plt.xlabel(“Inter-token latency (ms/token) → worse”) plt.ylabel(“Throughput (tokens/s/GPU) → better”) plt.title(“Pareto frontier: sweep variants (points labeled by concurrency)”) plt.legend(); plt.grid(alpha=.3); plt.tight_layout(); plt.show() We simulate benchmark observations for two chunked-prefill variants across increasing concurrency levels. We calculate representative throughput per GPU and inter-token latency values to model the saturation and latency growth commonly observed in distributed