YouZum

Committee

AI, Committee, Nachrichten, Uncategorized

Supercooled kidneys have been transplanted into pigs in a “landmark achievement”

When it comes to organ donation, time is everything. As soon as an organ has been carefully removed from a donor’s body, it starts to deteriorate. Surgeons have a matter of hours to get it into a recipient. Leave it too long and the organ will become unusable. In most cases, organs will be kept on ice during that time, at around 4 °C (39 °F). They cannot be frozen—in previous attempts, ice has formed, causing all kinds of damage. Matthew Powell Palm at Texas A&M University and his colleagues have an alternative solution—a device that allows organs to be cooled to -4 °C (25 °F) without forming any ice. Now, in new research with pig organs, his team has shown that kidneys, at least, can be supercooled and preserved in the device for days. Once rewarmed, the organs have been successfully transplanted into animals, and they seem to do better than organs kept on ice. The work represents “a landmark achievement,” says Kevin Myer, president and CEO of LifeGift, an organ procurement organization based in Texas, who was not involved in the research. Cooling organs Powell Palm hopes this approach could ultimately help ease the organ shortage crisis. Today, there are more than 104,000 people waiting for a kidney transplant in the US alone. It is estimated that 17 people die every day in the US while waiting for a transplant. That’s partly due to a lack of donated kidneys, but it’s also because many of those that are available never make it to a recipient. In some years, around one in three donated kidneys end up being discarded, often because they end up too degraded to use by the time they reach a recipient. Kidneys can be stored on ice for around 24 hours or placed in devices that aim to mimic the conditions of the body, also for up to around 24 hours. That’s not always long enough to find a suitable recipient and transport the organ, says Myer. Scientists around the world have been working on ways to store organs for longer by cooling them to even chillier temperatures. Cooling an organ slows its metabolism—the colder you go, the greater the effect, and the longer you can store it. We’ve long been able to successfully cryopreserve eggs, sperm, and embryos, but it’s much harder to freeze large organs. Teams have been exploring various temperatures and cryoprotectants (chemicals that essentially work like antifreeze), but so far no one has been able to freeze human organs for transplantation.   As a thermodynamicist, Powell Palm explored another approach. By keeping an organ submerged at a constant pressure, it should be possible to prevent the formation of ice at temperatures a little below 0 °C, without the need for cryoprotectants (which might have side effects and would need to be approved before being used in human transplants).  To test this theory, Powell Palm and his colleagues have created a device that does just that. The device itself is essentially a hermetically sealed chamber with a transparent lid. At its base is a device that monitors the organ’s temperature and checks for the formation of ice. Organs are submerged in a solution that is already commonly used to preserve them for transplant. “I always describe this as low-tech high science,” says Powell Palm. “A lot of work has gone into understanding the … kinetics at play in this system, but ultimately … it’s quite simple.” Supercooled kidneys To test their device, Powell Palm and his colleagues first removed single kidneys from pigs. The organs were flushed with the same commonly used solution to remove the blood, just as transplant organs are. The team then kept some kidneys on ice for either two hours or 24 hours, to mimic standard conditions used in human transplantation. They also put some of the removed kidneys in their device for 24, 48, or 72 hours. The stored kidneys were then each transplanted back into the original donor pigs. Each pig’s second kidney was removed in the same procedure, leaving each animal with only the kidney that had been stored, and reimplanted. Once the 24-hour supercooled kidneys were transplanted, they immediately began producing urine—a key indication that they were working. The team members also measured other markers of kidney function and found that the organs appeared to be working normally within about 10 days of being transplanted. A kidney that was supercooled for 72 hours recovers once it is transplanted back into a pig.COURTESY RONALD SELLERS, POWELL-PALM LAB, TEXAS A&M UNIVERSITY That’s slower than kidneys stored on ice for two hours but much quicker than kidneys kept on ice for 24 hours, says Powell Palm. The organs that were kept supercooled for 48 and 72 hours performed similarly, he says. “Even at three days—triple the clinical standard—we’re getting recovery that is faster than … [what has been] the gold standard for the last three decades,” he says. “So we’re really, really pumped about this.” “It is impressive,” says Heidi Yeh, a transplant surgeon at Mass General Brigham for Children, who also researches organ preservation technologies. “Often kidneys that have been stored for 48 hours [in other studies] take a week or two before they start working again.” Organs that grow The supercooled organs seem to work well in the long term, too. Over a 30-day period, the pigs grew by around 30%—and the kidneys grew with them, almost doubling in size to compensate for both the pigs’ growth and the lack of a second kidney. The team monitored one of the pigs for 200 days before removing and analyzing its kidney. Even at that point the organ looked healthy, says Powell Palm. He and his colleagues presented the findings at the American Transplant Congress in Boston last month. Earlier this year, researchers in Canada showed they could also cool pig kidneys to below-zero temperatures and transplant them into pigs. The team’s protocol included the use of a cryoprotectant, and organs were stored for up to 48

Supercooled kidneys have been transplanted into pigs in a “landmark achievement” Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Andrew Ng Just Released OpenWorker: An Open-Source, Local-First Desktop AI Coworker That Returns Finished Deliverables Instead of Chat

Andrew Ng has announced OpenWorker, an open-source desktop agent that produces finished work rather than conversation. OpenWorker asks the user for an outcome, not a prompt: a polished document, a Slack reply containing the actual numbers, an updated calendar, a triaged inbox. It then breaks that outcome into steps, works across local files and connected apps, and checks in before anything consequential. The architecture is four layers, and all of them run on your machine The repository contains 119 Python files (~32,400 lines) under coworker/, 149 TypeScript/TSX files under surfaces/gui/, and 78 backend test modules. The stack breaks down as follows: Desktop shell — a Tauri 2 native window wrapping a React 18 UI. The bundle identifier is com.openworker.desktop, and the shell supervises the Python server itself. Local agent server — Python 3.10+ on FastAPI and uvicorn, binding to 127.0.0.1:8765 by default. The example config caps a turn at 12 modeltool iterations. Capability and connector layer — vetted local tools (files, git, ripgrep-backed search, shell, todo) plus hosted integrations plus MCP. Model router — one interface over native, OpenAI-compatible, reseller and local providers. The engine is built on aisuite, Andrew Ng’s provider-agnostic LLM library. Bring your own model, from a deliberately small curated list There is no OpenWorker inference service. The user pastes an API key, or points the app at a local runtime. The curated model matrix contains exactly 30 entries. Native providers cover OpenAI (GPT-5.6 Sol/Terra/Luna and GPT-5.5), Anthropic (Claude Fable 5, Opus 4.8, Sonnet 4.6, Haiku 4.5) and Google (Gemini 3.1 Pro, 3.6 Flash, 2.5 Pro, 2.5 Flash). OpenAI-compatible vendors add GLM-5.2, DeepSeek V4, Kimi K2.6, MiniMax M2.5, Qwen3 Max, Grok 4.3 and Mistral Large. Open-weight models arrive through Together AI and Fireworks, and fully local models through Ollama, which requires no key at all. The permission engine is the actual engineering story Most desktop agent projects treat approvals as a UI afterthought. OpenWorker treats them as a typed layer. Every tool call is classified into one of four risk classes: read (no side effects), write_local (mutates the workspace, path-scoped), exec (runs commands), and external (side effects off the machine). Five permission modes then decide what happens: discuss and plan are read-only, interactive is the default and asks before writes, commands and external actions, auto allows everything while remaining path-scoped, and custom auto-approves a user-listed set of tools. Two design decisions stand out. First, unattended mode does not raise the autonomy ceiling — it only changes where the human is reached. Prompts that would appear inline are routed to an Inbox, and the session suspends until answered. Second, task-scoped standing rules are restricted to external risk only. Shell commands ask forever, by design. The built-in ops persona also instructs the model to treat content from tools, logs, the web, files and incoming messages as untrusted data rather than instructions. That is an explicit prompt-injection posture, written into the shipped persona. Privacy: local-first Model calls go directly from the machine to the configured provider. Conversations, connector tokens and model keys stay local, and the secret store is designed so that secrets never enter the model’s context, prompts or traces. The only cloud component is an optional broker that handles OAuth handshakes for one-click connectors, using Auth0 Authorization Code with PKCE. Connector tokens are handed straight to the machine and are never stored in the cloud. The app is fully functional signed out, using manually pasted credentials. Key Takeaways OpenWorker is Andrew Ng’s MIT-licensed desktop AI coworker that returns finished deliverables, not chat replies. The stack is a Tauri 2 + React shell over a local Python FastAPI agent server built on aisuite. Model access is bring-your-own-key across 30 curated tool-calling models, plus fully local Ollama. A typed risk engine (read/write_local/exec/external) gates every action across five permission modes. Check out the GitHub Repo, the project site, and the announcement. All credit for this research goes to the researchers and developers of this project. The post Andrew Ng Just Released OpenWorker: An Open-Source, Local-First Desktop AI Coworker That Returns Finished Deliverables Instead of Chat appeared first on MarkTechPost.

Andrew Ng Just Released OpenWorker: An Open-Source, Local-First Desktop AI Coworker That Returns Finished Deliverables Instead of Chat Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Cisco Foundation AI Releases Antares: 350M and 1B Open-Weight Models That Localize Known Vulnerabilities Inside Real Codebases

Cisco Foundation AI has released Antares, a family of security small language models (SLMs) built for one narrow security task. The task is vulnerability localization. Given a vulnerability description and a repository, find the files containing the flaw. Two models are open-weight and available now on Hugging Face, Antares-350M and Antares-1B. Both are Apache 2.0. Cisco team also shipped the Vulnerability Localization Benchmark (VLoc Bench), a 500-task agentic evaluation, under the same license. The main result is not a new state of the art. It is that a 1B model reaches 0.209 File F1. GPT-5.5 reaches 0.229, and a 753B open-weight model reaches 0.186. The problem Antares is scoped to Software security depends on connecting external vulnerability knowledge to internal source code. That knowledge lives in public databases, advisories, and Common Weakness Enumerations. The code lives in repositories that are large, modular, and dependency-rich. Connecting the two is expensive. Devs search unfamiliar code, follow naming conventions, inspect call paths, and compare candidate files. Cisco’s framing is that this first triage step is where the cost concentrates. Antares does not replace the application security toolchain. Cisco is explicit about this. Dev teams still need dependency scanning, secret scanning, dynamic testing, container checks, threat modeling, and expert review. Understanding the Models Antares consists of three decoder-only transformers at 350M, 1B, and 3B parameters. All three initialize from IBM Granite 4.0 checkpoints. They share a tokenizer and architecture: grouped-query attention, SwiGLU MLPs, RMSNorm, RoPE, and shared input/output embeddings. Model Params Base checkpoint Context Layers / hidden / KV heads Status Antares-350M 350M Granite 4.0 350M 32K 28 / 1024 / 4 Open weights Antares-1B 1.6B Granite 4.0 1B 128K 40 / 2048 / 4 Open weights Antares-3B 3B Granite 4.0 Micro 128K Not published Not released How the Agent Loop Works Antares is not evaluated as a standalone sequence model. It runs inside a constrained loop with three tools. The model receives a CWE category description and nothing else. No advisory text, no file hints, no severity details. It then issues read-only terminal commands against a Docker sandbox with networking disabled. Command output is truncated to 2,000 characters before entering the transcript. The budget is 15 terminal calls per task. The model terminates by calling submit_vulnerable_files with a ranked list, or submit_no_vulnerability_found. The submission itself does not count against the budget. Output is a ranked list of file paths plus the exploration trace that produced it. What VLoc Bench measures VLoc Bench draws 500 tasks from 290 unique real-world repositories. Sources are public GitHub Security Advisories across six ecosystems: npm, pip, Maven, Go, Rust, and Composer. It covers 147 unique CWE categories, and 78% of entries carry assigned CVE identifiers. Ground truth is derived from the security patch. Files modified in the fix are labels, with tests, docs, and configuration excluded. The benchmark has two phases: Phase A gives the model the vulnerable snapshot and scores File F1. Phase B gives the patched snapshot and scores True Negative Rate, testing whether the model raises a false alarm on fixed code. Results: task-specific training beats parameter scale The pattern in the data is a capability cliff, not a scaling curve. Antares-3B reaches 0.223 File F1, just under GPT-5.5 (xhigh) at 0.229. Antares-1B reaches 0.209, above GLM-5.2 at 753B parameters, which scores 0.186. Antares-350M reaches 0.135, above Gemma-4-31B at 0.101 and Gemini 2.5 Flash at 0.102. Antares-1B also records the highest recall of any evaluated system at 0.224. Static analysis tools were run under the same evaluation. Semgrep scores 0.086 File F1, CodeQL scores 0.023, and Horusec scores 0.020. Cisco’s reading is that rule-based scanners recover some vulnerable files but cannot adaptively inspect repository context. Where the capability comes from The untrained Granite 4.0 base checkpoints score 0.001, 0.000, and 0.000 File F1 under the identical protocol. They have tool-calling ability and still produce degenerate output inside an agentic loop. Supervised fine-tuning does the heavy lifting. It lifts the three scales to 0.108, 0.188, and 0.198. The SFT corpus is 71.5% cybersecurity reasoning, 15.4% code search trajectories, and 13.1% deep research and general reasoning. All reasoning traces come from a single teacher, GPT-OSS-120B, to avoid cross-teacher distribution shift. GRPO then adds 11% to 25%, with the largest relative gain at 350M. Rewards are verifiable and computed programmatically from trajectory text, with no learned reward model. Components cover localization quality, submission behavior, tool-use compliance, exploration, and malformed-output penalties. The variance effect may matter more than the mean. GRPO cuts run-to-run standard deviation by 42% to 65%. One GRPO evaluation run is a more reliable estimate than one SFT run. There is also a scale-dependent split in learned strategy. After GRPO, the 350M and 1B models use 87% to 89% search commands and submit more files. The 3B model settles at 52% search and 37% read, and submits fewer files at higher precision. The reward never prescribed either policy. Deployment Key Takeaways Antares-1B hits 0.209 File F1 on VLoc Bench, above GLM-5.2 at 753B parameters and Gemini 3 Pro. The Granite 4.0 base checkpoints score ~0.000 under the same protocol, so post-training supplies essentially all the capability. GRPO adds 11-25% File F1 and cuts run-to-run variance 42-65%, which matters more for repeatable CI scans. A full 500-task sweep costs under $1 on one H100, against $12.50 for GLM-5.2 and $141 for GPT-5.5. The strongest variant, Antares-3B, is not released, and Antares has no published Phase B false-alarm numbers. Check out the Models on Hugging Face, Benchmark, GitHub Repo and Mentioned Technical Report. The post Cisco Foundation AI Releases Antares: 350M and 1B Open-Weight Models That Localize Known Vulnerabilities Inside Real Codebases appeared first on MarkTechPost.

Cisco Foundation AI Releases Antares: 350M and 1B Open-Weight Models That Localize Known Vulnerabilities Inside Real Codebases Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Agents in the Wild: Where Research Meets Deployment

arXiv:2607.19336v1 Announce Type: cross Abstract: Agentic systems large language model (LLM) based architectures capable of reasoning, planning, acting, and coordinating with tools and other agents are rapidly transitioning from research prototypes to production scale deployments across domains such as software engineering, scientific discovery, and finance. While academic work has emphasized benchmarks and algorithmic innovation, deployment raises new challenges around robustness, safety, and reliability. This tutorial brings together researchers and practitioners to explore advances in reasoning and planning, multi agent coordination, and evaluation, highlighting open challenges arising from deployment experience. Through applied case studies in pharmaceutical discovery and financial systems, we analyze common design patterns that make agentic systems successful, and discuss practical mitigation strategies for failure modes, such as verification pipelines, fallback mechanisms, and human in the loop supervision. Attendees will gain a comprehensive view of the field along with concrete design patterns, evaluation checklists, and templates for safe and reliable deployment across industries.

Agents in the Wild: Where Research Meets Deployment Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Unsloth vs Axolotl vs TRL vs LLaMA-Factory: A Fine-Tuning Framework Comparison on Speed, VRAM, and Multi-GPU

Four open source projects dominate LLM fine-tuning today. Unsloth, Axolotl, TRL, and LLaMA-Factory all wrap the same underlying PyTorch and Hugging Face stack. They diverge on where they spend engineering effort. Unsloth rewrites kernels. Axolotl composes parallelism strategies. TRL defines the trainer APIs the others build on. LLaMA-Factory optimizes for breadth of model coverage and zero-code operation. This comparison covers three axes engineers actually hit: training throughput, peak VRAM, and multi-GPU scaling. Understand each framework TRL is the reference implementation layer. It ships SFTTrainer, DPOTrainer, GRPOTrainer, KTOTrainer, RewardTrainer, and RLOOTrainer. Axolotl and LLaMA-Factory both call into it. The current stable release line is v1.8.0. Unsloth replaces parts of the modeling code with hand-written Triton kernels. Backpropagation steps are manually derived rather than autograd-generated. Hugging Face’s own writeup notes accuracy degradation is 0% versus standard QLoRA, because no approximations are introduced. Axolotl is a YAML-driven wrapper over Transformers, PEFT, TRL, Accelerate, and DeepSpeed. Its differentiator is composability of parallelism strategies, not kernel work. LLaMA-Factory is an ACL 2024 system demonstration paper with a Gradio web UI called LlamaBoard. The repository covers 100+ LLMs and VLMs. Speed Unsloth: kernel-level gains on a single GPU Unsloth’s published benchmarks show 2x training speed for Llama 3.1 8B and Llama 3.3 70B. The setup used the Alpaca dataset, batch size 2, and gradient accumulation 4. QLoRA ran at rank 32 on all linear layers. The MoE results are larger. Unsloth fine-tuned unsloth/gpt-oss-20b-BF16 on an NVIDIA B200. It reports 712.33 ms per step at 8K context, versus 5,226.86 ms for Transformers v5. That is a 7.3x gap. At 4K the gap is 4.82x, and at 1K it is only 1.37x. The trend direction is model-dependent, and Unsloth’s docs scope this claim to gpt-oss. There, the speedup grows with sequence length, credited to Flex Attention and the MoE kernels. Qwen3-30B-A3B on B200 runs the other way. Its reported speedup falls from 1.7x at 1K to 1.1x at 16K. Memory savings move the opposite direction, rising from about 2% to 15%. Qwen3-30B-A3B on H100 reaches up to 1.77x. GLM-4.7-Flash on RTX PRO 6000 reaches 2.1x. A collaboration with AMD measured Llama-3.1-8B LoRA SFT at 2.07 s/step. TRL plus FlashAttention-2 took 2.87 s/step, a 1.39x gap with matching loss curves. Axolotl: kernels borrowed, parallelism native Axolotl added custom Triton kernels and autograd functions for LoRA in February 2025, explicitly citing Unsloth as inspiration. They are opt-in through lora_mlp_kernel, lora_qkv_kernel, and lora_o_kernel. Recent release notes add SonicMoE LoRA support. It delivers up to 1.45x speedup and 30% memory reduction over a grouped_mm baseline. That figure is for Qwen3.5-35B-A3B 8-bit LoRA on a single H100 SXM. Axolotl also ships FlashAttention 2/3/4, xFormers, Flex Attention, SageAttention, Liger Kernel, Cut Cross Entropy, and ScatterMoE. TRL: the baseline everyone measures against TRL is usually the reference point rather than the winner on raw single-GPU throughput. It compensates with breadth of memory and speed levers documented in Reducing Memory Usage and Speeding Up Training. Those levers include packing, padding-free batching, truncation, Liger Kernel, and vLLM sleep mode for GRPO. TRL also has a first-party Unsloth integration, so the two are not mutually exclusive. LLaMA-Factory: speed by delegation LLaMA-Factory does not write its own kernels. It exposes other people’s work through config flags. Setting use_unsloth: true activates the Unsloth patch. The project’s changelog reports 170% relative speed from that path. Unsloth’s long-sequence training is listed at 117% speed and 50% memory. It also supports enable_liger_kernel: true and FlashAttention-2 via flash_attn: fa2. VRAM Reported memory floors Unsloth publishes a VRAM requirements table sorted by parameter count. It lists 6 GB for an 8B model in 4-bit QLoRA and 41 GB for 70B. LoRA at 16-bit costs 22 GB and 164 GB for the same models. LLaMA-Factory’s README hardware table covers the same regime for 4-bit QLoRA. It lists 6 GB at 7B, 24 GB at 30B, and 48 GB at 70B. Full bf16 fine-tuning of 70B is listed at 600 GB. Both tables describe minimums. Batch size, sequence length, and optimizer choice move the real number. Context length is the sharper differentiator Peak VRAM at a fixed context is less interesting than the maximum context a given VRAM budget allows. Unsloth’s context length benchmarks for Llama 3.1 8B QLoRA at rank 32 and batch size 1 are stark. GPU VRAM Unsloth context Transformers + FA2 context 8 GB 2,972 OOM 16 GB 40,724 2,551 24 GB 78,475 5,789 48 GB 191,728 15,502 80 GB 342,733 28,454 Unsloth attributes this to its gradient checkpointing algorithm combined with Apple’s Cut Cross Entropy. For Llama 3.3 70B on an 80 GB A100, it reports 89,389 tokens. The FA2 baseline reaches 6,916. The MoE memory story MoE training is where memory behavior has shifted most in 2026. Unsloth reports gpt-oss-20b fine-tuning inside 12.8 GB, while Qwen3-30B-A3B at 16-bit LoRA needs 63 GB. Its B200 gpt-oss run used 47.43 GB at 8K context where Transformers v5 used 73.80 GB. At 16K, Transformers v5 went out of memory and Unsloth used 55.13 GB. The mechanism is a split-LoRA formulation. PEFT materializes the LoRA delta across all experts before the MoE matmul. Unsloth reorders the operations instead, which is mathematically identical but avoids the materialization. Axolotl attacks the same problem differently. Its MoE expert quantization quantizes expert weights during model loading, freeing the original bf16 tensor immediately. The reason is a Transformers v5 change. Expert layers moved from nn.Linear to fused nn.Parameter 3D tensors. bitsandbytes could no longer quantize them on load. Axolotl’s docs report GLM-4.7-Flash QLoRA dropping from roughly 127 GiB to roughly 23 GiB reserved memory with quantize_moe_experts: true. Multi-GPU This is where the ranking inverts. Unsloth’s single-GPU lead does not carry over. Axolotl: the deepest parallelism matrix Axolotl’s multi-GPU guide offers three mutually exclusive sharding strategies: DeepSpeed ZeRO stages 1 through 3, FSDP, and DDP. FSDP2 is the recommended path, and FSDP1 is deprecated. On top of those, its N-D Parallelism guide composes data, tensor, context, and expert parallelism through PyTorch’s DeviceMesh. The documented support matrix confirms FSDP+TP, HSDP+TP, FSDP+CP,

Unsloth vs Axolotl vs TRL vs LLaMA-Factory: A Fine-Tuning Framework Comparison on Speed, VRAM, and Multi-GPU Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Shape-shifting mirrors on NASA’s new space telescope could unveil Jupiters like our own

When NASA’s Nancy Grace Roman Space Telescope launches, as early as the end of next month, it will attempt one of astronomy’s most precise disappearing acts to date. The telescope will carry the first space-bound “active” coronagraph, an instrument that effectively erases most of the light from a star during photography. It will allow astronomers to take the first pictures of planets orbiting other stars that are similar to those in our solar system. Ultimately, it could pave the way for a future mission that could snap the first photos of Earth-like worlds. “I hope it’s remembered for it being that critical stepping stone for … finding Earth 2.0,” says Brandon Creager, the instrument’s lead mechanical engineer at NASA’s Jet Propulsion Laboratory (JPL). Named after Nancy Grace Roman, NASA’s first chief of astronomy, this new telescope will carry a roughly 300-megapixel wide-field camera that will enable it to capture images about 100 times larger than the Hubble Space Telescope’s widest exposures at a similar resolution. These capabilities will help astronomers unpack the mysterious identities of dark matter and dark energy—and to detect around 100,000 new exoplanets, planets outside our solar system, whose presence can be inferred from the way they distort the starlight of more distant stars. Javier Viaña, a research scientist at Harvard who has had two projects selected for Roman’s highly competitive first year of observing, compares the leap to moving from “interviewing a handful of people” to “conducting a global census.” Another camera will use the coronagraph, blocking out a star’s light as it observes one stellar system at a time. The instrument will allow astronomers an unprecedented look at the space around stars, enabling them to see smaller, dimmer, and more close-in exoplanets. “It’s giving us the ability to see planets that we haven’t been able to physically see before,” says Creager. The anatomy of a vanishing trick Coronagraphs in space aren’t new. But earlier incarnations, such as those currently aboard Hubble and the James Webb Space Telescope, use a stationary system to block a star’s blinding light. The approach does help, but it’s a bit like putting your thumb over a flashlight while searching a dark room for a firefly. Though the bulb vanishes, stray glare can still escape and overwhelm the light of the insect. Inside a telescope, that glare can come from light leaking around the edges of machinery or from minuscule imperfections in mirrors and coatings that can scatter starlight into speckles. All this can hide, or even impersonate, a planet. Roman’s coronagraph, however, will attempt something completely unseen in space telescopes until this year: Before each observation, it will measure that leftover light and try to suppress it, a technique known as active wavefront control. The telescope is able to do this because it contains two deformable mirrors. Each has a 48-by-48 checkerboard of actuators (tiny pistons) beneath a thin, deformable sheet of glass. Applying a small amount of voltage makes the actuators contract and tug their patches of mirror slightly backward, like thousands of microscopic fingers delicately sculpting a surface. The effect is very subtle: Each patch of mirror can deform by up to 0.5 micrometers, or about one-fourth the size of an E. coli bacterium, and in increments as small as approximately 10 picometers. That’s about a tenth the diameter of a hydrogen atom, says Ilya Poberezhskiy, the instrument’s project systems engineer at JPL. The actuators allow the mirrors to create an “active wavefront,” where each component is moved to the perfect position to cancel out incoming waves of unwanted light—a bit like a pair of noise-canceling headphones, but for light instead of sound. The “canceled-out” light creates a “doughnut-shaped region around the star where we suppress starlight and where we’re hoping to see exoplanets,” says Poberezhskiy. Compared with current space-based coronagraphs, the system is expected to improve sensitivity to exoplanets against the glare of their host stars by a factor of up to 1,000, revealing planets that would have been far too faint to detect before. Like Hubble and JWST, Roman also uses masks, patterned plates placed in the path of the light that are designed to block the photons that run into them. One tool in Roman’s mask arsenal is “silicon grass,” a thicket of microscopic spikes on some masks that can be used in certain configurations to absorb photons so they don’t bounce around the telescope and accidentally reach a detector. Light entering the forest bounces deeper and deeper between the blades and gets trapped instead of reflecting back toward the camera. “Once the light gets into there, it never gets out,” Poberezhskiy says. The mirrors and masks form a succession of gates and hedges to guide as much of the preserved planetary light as possible toward the final detector. Alien Jupiters This elaborate setup could open a new chapter in the direct imaging of exoplanets. Nearly all exoplanets photographed so far are oversize youngsters that are nothing like the residents of our solar system: several times the mass of Jupiter, still glowing with the heat left over from their birth, and orbiting tens or hundreds of times farther from their star than the Earth is from the sun. This is because they are relatively easy to see. Their size, warmth, and distance from their parent star makes them shine brightly in infrared light, far away from the worst of the stellar glare. Roman, however, could directly image a true Jupiter analogue—a planet similar to Jupiter in mass and circling a sunlike star a few times farther out than Earth is from our sun. Unlike the hot Jupiters we can see now, this one would be a much more mature gas giant like ours, primarily reflecting its parent star’s light after billions of years of cooling instead of heavily emitting its own. Astronomers have been able to infer the existence of such planets from the gravitational wobble they impart to the star. Roman instead will collect starlight reflected from the planet itself. “We’re not looking

Shape-shifting mirrors on NASA’s new space telescope could unveil Jupiters like our own Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

The Download: NASA’s new space telescope and OpenAI’s autonomous hacker

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Shape-shifting mirrors on NASA’s new space telescope could unveil Jupiters like our own When NASA’s Nancy Grace Roman Space Telescope launches, as early as the end of next month, it will attempt one of astronomy’s most precise disappearing acts to date. It will carry the first space-bound “active” coronagraph, an instrument that effectively erases most of the light from a star during photography. The technology will allow astronomers to take the first pictures of planets orbiting other stars that are similar to those in our solar system. Ultimately, it could pave the way for a future mission that could snap the first photos of Earth-like worlds. “I hope it’s remembered for it being that critical stepping stone for … finding Earth 2.0,” says Brandon Creager, the instrument’s lead mechanical engineer. Read the full story on the space telescope that could transform the search for distant planets. —Eshan Raul MIT Technology Review Narrated: PsiQuantum has a plan to make a massive quantum computer out of light The machine that could change the world will be housed in a room that looks like a data center crossed with an ice cream factory.  Inside, some 100 stainless-steel cabinets each hold hundreds of chips. On those chips, thousands of light particles will fly through a maze of optical switches and beam splitters. Each photon must be accounted for, because precisely measuring where it ends up will help answer questions that current computers might take millions of years to solve. This computer, as described, does not exist. It’s the brainchild of a company called PsiQuantum, founded in 2016 by four physicists from UK universities. In a crowded field of deep-pocketed competitors with similarly fantastical visions, the company aims to be the first to build a useful quantum machine. —James O’Donnell This is our latest story to be turned into an MIT Technology Review Narrated podcast, which we publish each week on Spotifyand Apple Podcasts. Just navigate to MIT Technology Review Narrated on either platform, and follow us to get all our new content as it’s released. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 OpenAI says one of its models carried out an autonomous hackIt escaped its testing sandbox and breached AI research platform Hugging Face. (Reuters $)+ OpenAI described it as a cybersecurity test that went badly wrong. (WSJ $)+ The hack is among the first known cyberattacks by an AI acting on its own. (FT $)+ Even simple AI attacks are cause for alarm, though. (MIT Technology Review) 2 France has become the first EU country to ban social media for under-15sIts parliament approved the ban, which President Macron championed. (NYT $)+ He pledged to enforce it by September, the start of the school year. (Guardian)+ But critics say it’s unconstitutional and impossible to enforce. (NPR) 3 The US and China will hold talks over AI in SeptemberTreasury Secretary Scott Bessent will lead the US side. (Reuters $)+ Chinese models have Trump’s AI world at war with itself. (MIT Technology Review) 4 Publishers are considering cutting Google off as AI reshapes searchNews outlets are weighing lost traffic against AI exposure. (WSJ $) 5 Samsung is in talks to invest €1 billion in MistralThe French AI firm is positioning itself as an alternative to US models. (FT $)+ It’s Europe’s leading AI firm, but US peers dwarf its $20 billion valuation. (Reuters $) 6 Amazon pushed up rivals’ prices, leaked records allegeInternal emails reveal tactics that allegedly reshaped online pricing. (Guardian) 7 Trump has tapped a Big Tech critic to lead the DOJ’s antitrust divisionAdam Candeub has called for tougher federal competition enforcement. (FT $) 8 New drilling methods could unlock geothermal energy almost anywhereThey aim to unlock Earth’s enormous heat reserves. (New Scientist $)+ AI is uncovering hidden geothermal energy resources. (MIT Technology Review) 9 AI researchers have proposed a “Genie coefficient” for measuring AI risksIt would track the gap between intent and action. (IEEE Spectrum)+ We need better ways to evaluate AI. (MIT Technology Review) 10 Japan’s AI boom has two unlikely winners: a toilet maker and an MSG giantThey’re supplying critical chipmaking materials. (CNBC) Quote of the day “He’s an analog man in a digital AI world, and I think that’s incredibly appealing.” —Paul Dergarabedian, a movie industry analyst at Comscore, tells Fortune that Christopher Nolan’s commitment to human filmmaking provides an attractive counterweight to Hollywood’s embrace of AI. One More Thing DANA SMITH Taiwan’s “silicon shield” could be weakening Taiwan produces the majority of the world’s semiconductors and more than 90% of the most advanced chips needed for AI applications. Many believe that’s helped deter China from invading the island. But now some Taiwan specialists and citizens are worried that this “silicon shield” is cracking. Facing pressure from Washington, TSMC—the world’s largest chipmaker—is expanding manufacturing abroad. In Taiwan, there are worries that this will dilute the company’s power at home, making the US and other countries less inclined to defend the island. Find out why Taiwan’s chipmaking dominance could be key to its future security. —Johanna M. Costigan We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Thrifty filmmakers have masterfully recreated Star Wars on a $10 budget.+ A man discovered squirrels hug and kiss their loved ones in the privacy of their homes.+ Toronto’s floating waterfront store is reimagining one of the most familiar spaces across cultures.+ These animations of Sesame Street characters performing classic tracks like Underworld’s “Born Slippy” will brighten up your day.

The Download: NASA’s new space telescope and OpenAI’s autonomous hacker Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Advancing next-gen AI with materials science innovation

The conversation about AI often centers on algorithms, computing power, or huge investments in new semiconductor fabrication plants and hyperscale data centers. But beneath each of these advances is another layer of innovation that makes them possible: advanced materials. Every new generation of AI technology demands more processing power, more memory, greater energy efficiency, and higher reliability. Every increase in computing performance increases the physical demands placed on the systems that make and run AI. Delivering these gains depends not only on advances in chip design and system architecture, but on advances in the materials that enable them to perform under extreme conditions. As AI continues to push the physical limits of semiconductors and data center infrastructure, advanced materials are no longer simply supporting innovation in this area; they are defining the limits of what is possible. Performance first Advanced materials exist to solve performance challenges. As AI raises the bar, these challenges are becoming more demanding. Manufacturing a semiconductor chip today requires thousands of tightly controlled process steps, with almost no room for error. Tiny variations in temperature or chemical instability can create defects that reduce yield and drive up manufacturing costs. With every new generation of semiconductor chips, manufacturers seek advanced materials that can deliver greater purity, higher chemical and plasma resistance, and better stability under increasingly harsh operating conditions. These are familiar engineering challenges being pushed to new extremes. And it’s here that materials innovation makes the difference with continuous advances in polymers, elastomers, specialty fluids, and other advanced materials that make each new generation of technology possible. For materials companies, it’s not about reinventing semiconductor manufacturing but about ensuring the materials supporting the industry continue to evolve alongside it. This same principle applies beyond the semiconductor fabrication floor. As AI workloads become more demanding, the physical infrastructure that powers them is evolving rapidly. Increasing computing density is transforming data center design, driving the need for more sophisticated thermal management, higher-voltage power architectures, increased data storage, and faster, more reliable data transmission. Every part of the system is under greater pressure, from cooling and power management to critical electronic components, such as connectors, capacitors, and hard disk drives. At Syensqo, we’re building on our expertise in electronic and electrical components, along with insights from other markets, to meet these emerging needs. For example, as data centers shift to higher-voltage architectures and greater power density, many of the materials challenges we face closely mirror those of electric vehicles. Fluid-circulation know-how from semiconductor and automotive coolant systems, for instance, can be adapted to direct liquid-cooling designs for AI servers. By transferring knowledge across markets, we can accelerate new power and thermal management solutions while supporting the reliability required by next-generation AI infrastructure. Whether we’re talking about semiconductor fabrication or hyperscale server farms, the challenge for materials science companies is the same: enabling greater performance without compromising reliability. A new definition of what performance means While performance remains the first priority, the way performance is defined is changing. In addition to meeting the increasingly demanding technical requirements of next-generation semiconductors and data centers, there is now an expectation that these materials are developed and manufactured more responsibly. Perfluoroelastomers, for example, are used to seal semiconductor manufacturing equipment. These materials operate under extreme temperatures, aggressive plasma, and highly reactive chemicals. To make the process more sustainable, at Syensqo, our next generation of perfluoroelastomers use a fluorosurfactant-free manufacturing process. Our goal was to make a better-performing material, produced in a better way, ensuring manufacturers no longer have to choose between higher performance and a more responsible way of producing the materials that enable it. This approach reflects a broader reality across the industry. New materials aren’t adopted simply because they are new. Qualification can take years, and manufacturers only make changes when a material solves a genuine engineering challenge or enables new technology. Performance remains the price of entry. The difference today is that the definition of performance has expanded. Success increasingly depends on delivering technical excellence through more responsible manufacturing from the outset. Accelerating the pace of discovery As the performance bar rises, the way we innovate must evolve with it. Developing advanced materials has traditionally involved a lengthy process of hypothesis, synthesis, testing, and iteration. While this process remains unchanged, new digital tools are helping researchers move through these cycles faster. By helping researchers identify the most promising candidates earlier, AI can reduce the number of physical experiments required and accelerate the earliest stages of materials discovery. AI isn’t replacing scientific expertise. It’s helping scientists apply that expertise more effectively, allowing them to spend less time searching for answers and more time solving the industry’s toughest challenges. At Syensqo, we’re putting this approach into practice through use of several AI tools, including the Microsoft Discovery platform, which are helping researchers identify and evaluate promising molecular candidates for next-generation heat transfer fluids, used in semiconductor manufacturing and data centers. AI helps our researchers rapidly identify and evaluate promising molecular candidates based on the properties they need to achieve. This allows us to focus laboratory work where it has the greatest potential to deliver results, accelerating discovery and reducing the time needed to turn promising materials into solutions customers can qualify and deploy. The journey from laboratory discovery to a qualified material will always require scientific expertise, rigorous testing, and close collaboration with customers. But by accelerating the earliest stages of discovery, AI can help materials innovation keep pace with the evolving needs of industries such as semiconductors, electronics, and data centers. Progress is earned The future of artificial intelligence will depend on better algorithms, more powerful chips, and larger computing infrastructure. But sustaining that progress will also require advances in the materials that make those technologies possible. Whether in semiconductor manufacturing or AI infrastructure, progress is earned. Every new generation of technologies raises the bar, and every new material must prove it can deliver the performance, reliability, and efficiency needed before it earns its place. For materials companies, that remains

Advancing next-gen AI with materials science innovation Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

The Download: Chinese AI divides the White House, and a record copyright payout

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. China’s AI models have Trump’s AI world at war with itself Last weekend, several current and former advisers to President Donald Trump on AI publicly lobbed insults at the country’s leading AI companies. David Sacks branded Anthropic’s models “lobotomized” and “woke.” Emil Michael, a top Pentagon official, called OpenAI’s new head of strategic futures a “supreme village idiot.” It began because no one can agree on what to do about Kimi, a free, open-source model that Chinese AI company Moonshot launched last week. It appears to rival the intelligence of models from OpenAI and Anthropic, which are very much not free.  Every time a new smart, free model from China gets released, US companies see less reason to fork out money for models from Anthropic or OpenAI. That’s creating economic and political problems for the president—and dividing the top AI strategists in his orbit.  Read the full story on why no one can agree what to do about Kimi. —James O’Donnell This article is from The Algorithm, our weekly AI newsletter. Sign up to receive it in your inbox every Monday. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 Anthropic’s record $1.5 billion copyright settlement has been approvedThe plaintiffs said Anthropic used pirated works to train Claude. (Reuters $)+ And won the largest known copyright payout in history. (Engadget)+ Yet many authors and creators still don’t view it as a win. (TechCrunch)+ But AI copyright anxiety could limit creativity. (MIT Technology Review) 2 The Trump administration is weighing a ban on Chinese AI modelsThe launch of Kimi K3 has revived calls for restrictions. (Axios)+ But officials are divided on the proposals. (Fast Company)+ China’s bet on open-source is paying off. (MIT Technology Review) 3 China is mulling tighter export controls on AI models and chipsIt wants to stop the West from acquiring its tech and startups. (FT $)+ Beijing has held talks with tech firms about potential restrictions. (Reuters $) 4 Trump’s AI safety head has resigned after just three monthsChris Fall had led CAISI, the federal AI Safety Institute, since April. (Axios)+ No reason was given for his exit. (CNBC) 5 Google is working on a new chip to run Gemini models more efficiently The chip, called “Frozen V2,” may be deployed in 2028. (Information $)+ Alphabet stock popped on the report. (CNBC) 6 New Orleans police have explored arming drones with weaponsA draft drone manual paves the way for weaponised quadcopters. (404 Media)+ Shoplifters could soon be chased by drones. (MIT Technology Review) 7 The EU has handed AliExpress a record fine over unsafe product salesThe €550 million fine is the largest-ever under the Digital Services Act. (BBC)+ Alibaba has vowed to appeal the fine. (SCMP) 8 Election advice from AI chatbots is “inaccurate and unreliable”That’s the conclusion from tests in Hungary earlier this year. (Guardian) 9 Red light therapy is showing promise for healing and healthy agingBetter skin and reduced vision loss are also on the cards. (Economist $) 10 Neill Blomkamp’s new horror clip is all AI-generated—and it sucksThe acclaimed director wants to make “a full feature in this format.” (Gizmodo) Quote of the day “This would be a terribly self-defeating form of intervention if it were to happen.”  —Tech investor Chamath Palihapitiya slams plans to restrict Chinese AI models in a post on X. One More Thing AKILAH TOWNSEND Inside Chicago’s surveillance panopticon Early on the morning of September 2, 2024, four people were shot and killed on a westbound train in Chicago. Police swiftly activated a digital dragnet—a surveillance network that connects thousands of cameras across the city—and arrested the suspect just 90 minutes later. Law enforcement and security advocates say this vast monitoring system protects public safety and works well. But activists and many residents say it’s a surveillance panopticon that creates a chilling effect on behavior and violates guarantees of privacy and free speech. Go inside the surveillance network that’s dividing Chicago. —Rod McCullom We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + NASA has shared a stunning timelapse video of the Psyche spacecraft’s view of Mars.+ This comparison of American and European Urbanism shows good city design is a choice.+ Musician Luca Stricagnoli recently performed a marvellous acoustic guitar medley of Prodigy songs.+ Two Australian paddleboarders saved a stranded wallaby after it was swept out to sea—and caught the whole rescue on video.

The Download: Chinese AI divides the White House, and a record copyright payout Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Validating Distributed LLM Serving Benchmarks with NVIDIA srt-slurm, SLURM Recipes, Parameter Sweeps, and Pareto Analysis

In this tutorial, we explore NVIDIA’s srt-slurm framework and learn how we use srtctl to convert declarative YAML configurations into reproducible SLURM benchmark workflows for distributed LLM serving. We set up the project in Google Colab, inspect its internal architecture, define a cluster configuration, dry-run built-in and custom recipes, and model a disaggregated prefill-and-decode deployment for DeepSeek-R1. We also generate parameter sweeps, interact with the typed Python API, validate expanded configurations, and analyze simulated benchmark results through a throughput-versus-latency Pareto frontier. Although Colab does not provide a real SLURM environment, we use it as a practical development workspace to understand, validate, and prepare production-grade benchmark recipes before we submit them to an actual GPU cluster. Copy CodeCopiedUse a different Browser import os, sys, subprocess, textwrap, json, shutil, importlib from pathlib import Path def run(cmd, check=True, quiet=False): “””Run a shell command, stream output.””” print(f”n$ {cmd}”) r = subprocess.run(cmd, shell=True, text=True, capture_output=True) out = (r.stdout or “”) + (r.stderr or “”) if not quiet: print(out[-6000:]) if check and r.returncode != 0: raise RuntimeError(f”Command failed ({r.returncode}): {cmd}”) return out def section(title): print(“n” + “═”*78 + f”n {title}n” + “═”*78) section(“1. Install srt-slurm”) REPO = Path(“/content/srt-slurm”) if Path(“/content”).exists() else Path.cwd()/”srt-slurm” if not REPO.exists(): run(f”git clone –depth 1 https://github.com/NVIDIA/srt-slurm.git {REPO}”, quiet=True) run(f”{sys.executable} -m pip install -q -e {REPO}”, quiet=True) sys.path.insert(0, str(REPO / “src”)) importlib.invalidate_caches() os.chdir(REPO) run(“srtctl –help”) We prepare the Colab environment by importing the required modules and defining reusable helper functions for command execution and section formatting. We clone the NVIDIA srt-slurm repository, install it in editable mode, and expose its source directory to the active Python runtime. We then switch to the repository directory and verify that the srtctl command-line interface is installed correctly. Copy CodeCopiedUse a different Browser section(“2. Repository architecture”) print(textwrap.dedent(“”” src/srtctl/ cli/ submit.py (apply/dry-run/preflight/monitor), do_sweep, interactive core/ schema.py (typed config), sweep.py, slurm.py (sbatch gen), validation.py, health.py, topology.py, fingerprint.py backends/ sglang.py, trtllm.py, vllm.py, mocker.py ← engine adapters frontends/ Dynamo / router frontends templates/ Jinja2 → sbatch + orchestrator scripts recipes/ ready-made benchmarks per platform (gb200-fp4, h100, b200-fp8, qwen3-32b, dsv4-pro, mocker smoke tests, …) analysis/ srtlog (log parsers) + Streamlit dashboard (Pareto, latency…) docs/ sweeps.md, profiling.md, analyzing.md, config-reference.md “””)) for d in [“recipes”, “docs”]: print(f”{d}/ →”, “, “.join(sorted(p.name for p in (REPO/d).iterdir()))[:300]) section(“3. Cluster configuration (srtslurm.yaml)”) (REPO/”srtslurm.yaml”).write_text(textwrap.dedent(“”” cluster: “colab-demo” default_account: “demo-account” default_partition: “gpu” default_time_limit: “01:00:00” gpus_per_node: 4 use_gpus_per_node_directive: true use_segment_sbatch_directive: true containers: dynamo-sglang: “/containers/dynamo-sglang.sqsh” lmsysorg+sglang+v0.5.5.post2.sqsh: “/containers/sglang-v0.5.5.sqsh” model_paths: deepseek-r1: “/models/DeepSeek-R1” “””)) print((REPO/”srtslurm.yaml”).read_text()) We inspect the repository structure to understand how srtctl organizes its command-line tools, schemas, backends, templates, recipes, and analysis components. We then create a local srtslurm.yaml file containing simulated cluster defaults, container aliases, GPU settings, and model paths. We use this configuration to resolve recipe references in Colab without requiring access to an actual SLURM cluster. Copy CodeCopiedUse a different Browser section(“4. Dry-run: mocker smoke test → generated sbatch script”) run(“srtctl dry-run -f recipes/mocker/agg.yaml”, check=False) section(“5. Custom disaggregated recipe (prefill/decode split)”) (REPO/”my-disagg.yaml”).write_text(textwrap.dedent(“”” name: “colab-disagg-demo” model: path: “deepseek-r1” container: “lmsysorg+sglang+v0.5.5.post2.sqsh” precision: “fp8” resources: gpu_type: “gb200” gpus_per_node: 4 prefill_nodes: 1 decode_nodes: 2 prefill_workers: 1 decode_workers: 2 backend: prefill_environment: { PYTHONUNBUFFERED: “1” } decode_environment: { PYTHONUNBUFFERED: “1” } sglang_config: prefill: served-model-name: “deepseek-ai/DeepSeek-R1” model-path: “/model/” trust-remote-code: true kv-cache-dtype: “fp8_e4m3” tensor-parallel-size: 4 disaggregation-mode: “prefill” decode: served-model-name: “deepseek-ai/DeepSeek-R1” model-path: “/model/” trust-remote-code: true kv-cache-dtype: “fp8_e4m3” tensor-parallel-size: 4 disaggregation-mode: “decode” benchmark: type: “sa-bench” isl: 1024 osl: 1024 concurrencies: [64, 128, 256] req_rate: “inf” “””)) run(“srtctl dry-run -f my-disagg.yaml”, check=False) We dry-run the built-in mocker recipe to examine how srtctl validates configurations and generates SLURM submission artifacts without executing a real benchmark. We then define an advanced DeepSeek-R1 recipe that separates prefill and decode workloads across independent node and worker pools. We validate this disaggregated SGLang configuration through another dry run and inspect how the serving parameters are translated into job scripts. Copy CodeCopiedUse a different Browser section(“6. Parameter sweep (grid search) — dry-run + expansion on disk”) run(“srtctl dry-run -f examples/example-sweep.yaml”, check=False) sweep_dirs = sorted((REPO/”dry-runs”).glob(“example-sweep_sweep_*”)) if sweep_dirs: latest = sweep_dirs[-1] print(“Per-job configs generated by the sweep expander:”) for p in sorted(latest.rglob(“config.yaml”)): print(” “, p.relative_to(REPO)) section(“7. Programmatic use of srtctl’s Python API”) import yaml from srtctl.core.config import load_config from srtctl.core.sweep import generate_sweep_configs, expand_template from srtctl.core.schema import BenchmarkType, Precision, GpuType cfg = load_config(“my-disagg.yaml”) print(f”Loaded : {cfg.name}”) print(f”Model : {cfg.model.path} ({cfg.model.precision}) on {cfg.resources.gpu_type}”) print(f”Layout : {cfg.resources.prefill_nodes}P + {cfg.resources.decode_nodes}D nodes, ” f”{cfg.resources.gpus_per_node} GPUs/node”) print(f”Bench : {cfg.benchmark.type} isl={cfg.benchmark.isl} osl={cfg.benchmark.osl} ” f”concurrencies={cfg.benchmark.concurrencies}”) print(f”Enums : benchmarks={[b.value for b in BenchmarkType]}”) print(f” precisions={[p.value for p in Precision]}, gpus={[g.value for g in GpuType]}”) raw_sweep = yaml.safe_load(Path(“examples/example-sweep.yaml”).read_text()) jobs = generate_sweep_configs(raw_sweep) print(f”nSweep expands to {len(jobs)} jobs:”) for job_cfg, params in jobs: pf = job_cfg[“backend”][“sglang_config”][“prefill”] print(f” {params} → chunked-prefill-size={pf[‘chunked-prefill-size’]}, ” f”max-total-tokens={pf[‘max-total-tokens’]}”) print(“nTemplate substitution:”, expand_template({“flag”: “{x}”, “n”: “{y}”}, {“x”: 4096, “y”: 2})) We execute the example parameter sweep and inspect the individual job configurations created from its Cartesian search space. We load our custom recipe through the typed Python API and examine its model, precision, GPU topology, benchmark settings, and supported enumeration values. We also programmatically expand sweep templates and verify how each parameter combination affects the generated backend configuration. Copy CodeCopiedUse a different Browser section(“8. Analysis: Pareto frontier from (simulated) benchmark results”) import numpy as np, matplotlib.pyplot as plt rng = np.random.default_rng(0) def simulate(variant, base_tps, base_itl): rows = [] tps_gpu = base_tps * c / (c + 90) * rng.uniform(.97, 1.03) itl = base_itl * (1 + c/220) * rng.uniform(.97, 1.03) rows.append({“variant”: variant, “concurrency”: c, “tok_s_gpu”: tps_gpu, “itl_ms”: itl}) return rows results = simulate(“chunked=4096”, 260, 9.5) + simulate(“chunked=8192”, 300, 11.5) print(json.dumps(results[:3], indent=2), “…”) plt.figure(figsize=(8, 5)) for variant in (“chunked=4096”, “chunked=8192”): pts = [(r[“itl_ms”], r[“tok_s_gpu”], r[“concurrency”]) for r in results if r[“variant”] == variant] xs, ys, cs = zip(*pts) plt.plot(xs, ys, “o-“, label=variant) for x, y, c in pts: plt.annotate(str(c), (x, y), fontsize=7, xytext=(3, 3), textcoords=”offset points”) plt.xlabel(“Inter-token latency (ms/token) → worse”) plt.ylabel(“Throughput (tokens/s/GPU) → better”) plt.title(“Pareto frontier: sweep variants (points labeled by concurrency)”) plt.legend(); plt.grid(alpha=.3); plt.tight_layout(); plt.show() We simulate benchmark observations for two chunked-prefill variants across increasing concurrency levels. We calculate representative throughput per GPU and inter-token latency values to model the saturation and latency growth commonly observed in distributed

Validating Distributed LLM Serving Benchmarks with NVIDIA srt-slurm, SLURM Recipes, Parameter Sweeps, and Pareto Analysis Beitrag lesen »

We use cookies to improve your experience and performance on our website. You can learn more at Datenschutzrichtlinie and manage your privacy settings by clicking Settings.

Privacy Preferences

You can choose your cookie settings by turning on/off each type of cookie as you wish, except for essential cookies.

Allow All
Manage Consent Preferences
  • Always Active

Save
de_DE