Nachrichten

Prime Intellect has released prime-rl version 0.6.0. The framework targets reinforcement learning on trillion-parameter Mixture-of-Experts (MoE) models. It focuses on heavy agentic workloads, like long-horizon software-engineering tasks. The research team trained GLM-5 on SWE tasks at up to 131k sequence length. Step times stayed under five minutes. The batch size was 256 rollouts. The run used only 28 H200 nodes. TL;DR prime-rl 0.6.0 trains trillion-parameter MoE models on agentic RL workloads. GLM-5 trained on SWE at 131k sequence length, sub-5-minute steps, 28 H200 nodes. Asynchronous RL disaggregates trainer and inference for independent optimization. Inference uses FP8, Wide EP, P/D disaggregation, KV offloading, and router replay. Training uses 3-D parallelism (FSDP, EP, CP) plus block-scaled FP8. What is prime-rl 0.6.0? prime-rl is an open framework for asynchronous reinforcement learning. It post-trains large open-source models on agentic tasks. Version 0.6.0 extends this to trillion-parameter MoE scale. The example model in the announcement is zai-org/GLM-5.1. The optimizations also apply to other large MoE models. Examples include moonshotai/Kimi-K2.7-Code and nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16. A full GLM-5.1 run starts with one command on a Slurm cluster. Copy CodeCopiedUse a different Browser uv run rl @ examples/glm5_llmd/rl.toml –output-dir /shared/outputs/glm5-llmd Role of asynchronous RL Agentic tasks have long-tail outliers. Some coding rollouts run for hours. Waiting for them before each policy update would idle GPUs. Asynchronous RL avoids this. The trainer and inference systems are disaggregated. They run and scale independently. The inference policy updates as soon as the optimizer step finishes. There is one synchronization point: the policy update. prime-rl pushes new weights as soon as they exist. Already-dispatched rollouts keep their active prefix cache. So a single rollout may mix tokens from several policy versions. New rollouts behave differently. They repopulate their own KV cache, even when prefixes match. A KV-cache salt forces this. Requests from too old a policy are dropped. The max_off_policy_steps value controls that threshold. Inference optimizations Inference is usually the throughput bottleneck in an RL system. prime-rl optimizes for throughput, while keeping latency bounded. FP8 inference: Lower precision speeds up prefill and decode. prime-rl uses FP8 with DeepEP and DeepGEMM kernels. Wide Expert Parallelism: Wide EP spreads experts across ≥32 GPUs. It pairs with a large data-parallel rank, for example 32. Each GPU holds separate experts and serves as an endpoint. Synchronization happens per-layer, through dispatch and combine operations. Prefill and Decode Disaggregation: Some modelenv pairs hit a 4:1 prefill:decode token ratio. Shared workers would inflate end-to-end latency. That reduces the benefits of PipelineRL. P/D disaggregation separates prefill and decode workers. Long tool outputs then stop throttling decode workers. KV cache management: High concurrency needs large KV cache space. prime-rl supports tiered offloading to CPU and disk. vLLM native offloading creates one pool per worker. Mooncake Store instead pools RAM and disk across all nodes centrally. Request routing: prime-rl ships a fork of vllm-router by default. It also supports the NVIDIA Dynamo router as a drop-in. Routers score workers using KV cache reuse, queue depth, and live load. Router replay (R3): Trainerinference mismatch silently kills training. Router replay captures inference routing decisions. It replays them directly on the trainer. This cuts KL mismatch by roughly an order of magnitude. Routed experts have shape [num_layers, top_k, seq_len]. This payload can grow to hundreds of GB. At scale, the data rate reaches tens of Gbps. So prime-rl treats it as an opaque payload. Optimized PyTorch operations handle the processing. Training optimizations The trainer builds on torchtitan, a PyTorch-native training codebase. It relies on 3-D parallelism: FSDP, CP, and EP. The GLM-5 case study uses all three. Strategy What it shards Primary use Key detail FSDP (FSDP2) Parameters, gradients, optimizer states Baseline memory amortization Gathers weights on demand per layer via fully_shard Expert Parallelism (EP) Experts within a layer Shrinks active layer memory all2all dispatch/combine; torch-native or DeepEP Context Parallelism (CP) The sequence dimension Long-context activation memory Ulysses (default) or Ring Attention EP exists because layers stay huge after FSDP. With 78 layers and 800B params in float32, one layer’s all-gather needs roughly 40GB. Overlapping one layer pushes that near 80GB. Setting EP=8 dispatches tokens instead of gathering full experts. torch-native all2all is slightly faster within one node. DeepEP wins when EP spans multiple nodes. CP matters at 131k+ sequence length. There, activations dominate memory, not parameters. GLM-5 uses DSA, which neither Ulysses nor Ring Attention parallelizes directly. So prime-rl ships a custom context-parallel implementation for it. FP8 training. prime-rl uses DeepGEMM block-scaled FP8, as proposed by DeepSeek V3. This rarely raises throughput, due to quantization overhead. Its real value is matching trainer and inference precision. That reduces KL mismatch and stabilizes training. Interactive Explainer Use cases with examples Long-horizon SWE agents: Train a model on real repository issues. Rollouts can span 100s of turns and tool calls. P/D disaggregation keeps decode latency predictable here. 1T-scale post-training on fewer nodes: The GLM-5 run fit on 28 H200 nodes. Wide EP and KV offloading raise concurrency and throughput. Stable agentic RL at scale: Router replay and FP8 training both reduce trainerinference KL mismatch. Lower mismatch means steadier training. Check out the Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Prime Intellect Releases prime-rl 0.6.0 to Train Trillion-Parameter MoE Models on Agentic RL Workloads appeared first on MarkTechPost.

India is home to about 60% of the world’s wild Asian elephants, and around 80% of the animals’ habitat lies outside protected areas, according to the Ministry of Environment, Forest, and Climate Change. That brings people and wildlife into close contact, and clashes can turn lethal: There have been some 3,000 human casualties in the last five years and over 1,000 elephant deaths since 2014. In places where elephants tend to wander, warnings from ground-based patrols can sometimes take hours to reach populated areas like villages and farms, so they have failed to prevent much of the damage. In response, state forest departments, NGOs, and locals are beginning to design, test, and deploy a range of artificially intelligent systems that can cut response and warning times to minutes—or even seconds.  Kanika Gupta is an independent journalist and documentary filmmaker based in New Delhi.

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here. For those of you enjoying your summer unaware of Anthropic’s latest feud with the US government, here’s a recap: In April the company said it had built an AI model called Mythos that was so good at working with code it could pose a global cybersecurity threat. Anthropic gave access to a small group of cybersecurity experts so they could see what they were up against. Then it released a modified version called Fable which it said was safer to the public on Tuesday, June 9. That Friday, the federal government told the company it was a threat to national security and placed export controls on the new release. Anthropic revoked access to both models hours later. People worried about catastrophic effects of AI—broadly labeled “doomers”—have said for years that the technology poses a threat to humanity and published proposals for how the government should intervene in its development. The doomers just got their government intervention—not over a bioweapon or rogue AI, but in response to an AI model that’s basically just really good at coding. And the result so far looks less like a safety plan than like a superficial reaction. There’s plenty to dissect about what happened in those few days that led to such drastic action from the government, and it’s notable that Amazon CEO Andy Jassy was the one who told government officials that Fable would be dangerous (Amazon is both invested in Anthropic and building its own competing AI models). It’s also possible this will be a short-lived ban from the government that doesn’t survive legal scrutiny (it’s not clear that Anthropic’s offering access to Fable really counts as “exporting” it, for example).  But there are ripple effects happening already.  For one, this is making a whole lot of people not want to rely on American AI companies. TheFrench politician Bruno Retailleau described it as a “wake-up call” that should motivate Europe to build more AI. But any vision of turning Paris into Silicon Valley—touted by many other European leaders following the shutdown of Anthropic’s models—is complicated by one big thing: China.  Open-source models from China are very capable and incredibly cheap, and they can be downloaded to run on anyone’s servers with no rules or guardrails. (This makes them attractive to companies that don’t want access turned off on the basis of a decision from the White House—but equally attractive to cybercriminals, the type that Anthropic hoped to fend off by building safety guardrails into its models.)  It’s possible that companies, including those in the US and Europe, will decide that working with Chinese models is just easier, as the skyrocketing of shares in the Chinese startup Zhipu suggests. Playing this forward, is it possible the government’s next drastic decision will be to say that US companies using models from China pose a threat to national security? I wouldn’t write it off.  Second, it’s possible that shutting off access to Anthropic’s models will leave the country morevulnerable to cybersecurity attacks, not less. Leading cybersecurity experts have said as much in an open letter to the government, writing that access to Anthropic’s models was helping researchers prepare defenses, and that the company’s models are no more dangerous than other leading models that are widely available. Such is the risk of applying the concept of nonproliferation to software—trying to control and restrict dangerous AI models in the manner of the uranium used for nuclear weapons.  The third thing worth watching is how US lawmakers will react. Remember that following Anthropic’s last feud with the government over how the Pentagon could or could not use its models, a slate of new bills was introduced that would define the limits of military AI. Right now, the biggest players shaping how AI gets used are the companies and the White House. There’s been much talk about more federal AI regulation, and polling suggests most Americans want it. Lawmakers are still figuring out whether to form rules on how kids use chatbots and are far from a clear answer on the extent to which the government should vet the safety of AI models. But with every drastic action from the White House, the pressure for regulations rises. To state the obvious, predictions are hard when the administration’s attitudes toward AI  change with the wind. When President Trump took office, he threw out the restrictive rulebook for how to make AI safe and promised to get out of the way of tech companies. The White House has now called the most valuable AI startup a risk to national security once in the spring, and again in summer. What will fall bring?

In this tutorial, we build a Prefab application that demonstrates how to create interactive dashboards entirely in Python. We use Prefab’s component-based Python interface to design a polished operations dashboard with reactive state, charts, tables, filters, forms, tabs, alerts, metrics, and client-side actions. We generate realistic pipeline monitoring data, connect it to live UI controls, and export the final app as a static HTML dashboard that we can preview directly inside Google Colab. Through this workflow, we learn how Prefab lets us move from Python data logic to a modern React-powered user interface without having to write frontend code manually. Installing Prefab in Colab Copy CodeCopiedUse a different Browser import os import sys import base64 import subprocess from pathlib import Path from IPython.display import HTML, display, FileLink PREFAB_VERSION = “0.20.2” APP_PATH = Path(“/content/prefab_advanced_tutorial_app.py”) HTML_PATH = Path(“/content/prefab_advanced_dashboard.html”) subprocess.check_call([ sys.executable, “-m”, “pip”, “install”, “-q”, f”prefab-ui=={PREFAB_VERSION}”, ]) APP_CODE = “” We set up the Colab environment by importing the required Python utilities and defining the Prefab version, app path, and HTML export path. We install the pinned prefab-ui package so that the tutorial runs consistently without version-related issues. We also initialize an empty APP_CODE string, which we use to build the complete Prefab application step by step. Copy CodeCopiedUse a different Browser APP_CODE += r”’ Generating Synthetic Operations Data Copy CodeCopiedUse a different Browser import random from collections import Counter, defaultdict from datetime import date, timedelta from prefab_ui.actions import AppendState, OpenLink, PopState, SetState, ShowToast, ToggleState from prefab_ui.app import PrefabApp from prefab_ui.components import ( Alert, AlertDescription, AlertTitle, Badge, Button, Card, CardContent, CardDescription, CardFooter, CardHeader, CardTitle, Code, Column, DataTable, DataTableColumn, Form, Grid, H2, Input, Markdown, Mermaid, Metric, Muted, Progress, Ring, Row, Slider, Small, Switch, Tab, Tabs, Text ) from prefab_ui.components.charts import ( BarChart, ChartSeries, LineChart, PieChart, RadarChart, ScatterChart, Sparkline ) from prefab_ui.components.control_flow import Else, ForEach, If from prefab_ui.rx import EVENT, STATE random.seed(42) TODAY = date.today() DATES = [TODAY – timedelta(days=29 – i) for i in range(30)] REGIONS = [“All”, “APAC”, “EMEA”, “NA”, “LATAM”] PIPELINES = [ “Customer 360 ETL”, “Invoice OCR”, “LLM Triage”, “Risk Scoring”, “Forecast Sync”, “Warehouse Load”, ] OWNERS = [“Data Platform”, “AI Apps”, “Revenue Ops”, “Risk Engineering”] STATES = [“Completed”, “Completed”, “Completed”, “Completed”, “Late”, “Failed”] PRIORITIES = [“P0”, “P1”, “P2”, “P3”] runs = [] daily_region_rows = [] for d in DATES: for region in REGIONS[1:]: region_bias = { “APAC”: 0.96, “EMEA”: 0.94, “NA”: 0.97, “LATAM”: 0.91, }[region] volume = random.randint(32, 78) failures = 0 late = 0 total_cost = 0.0 total_latency = 0.0 total_revenue = 0.0 for i in range(volume): pipeline = random.choice(PIPELINES) owner = random.choice(OWNERS) state = random.choices( STATES, weights=[ region_bias * 10, 6, 4, 3, 1.2, max(0.2, (1 – region_bias) * 16), ], k=1, )[0] duration = max( 12, int( random.gauss(95, 35) + (20 if state == “Late” else 0) + (45 if state == “Failed” else 0) ), ) cost = round(max(0.09, random.lognormvariate(-1.15, 0.55) + duration / 1800), 2) revenue = round(random.uniform(1.2, 8.5) * (1.3 if state == “Completed” else 0.6), 2) priority = random.choices(PRIORITIES, weights=[1, 3, 7, 10], k=1)[0] if state == “Failed”: failures += 1 if state == “Late”: late += 1 total_cost += cost total_latency += duration total_revenue += revenue if d >= TODAY – timedelta(days=10) and (state in {“Failed”, “Late”} or random.random() < 0.05): runs.append({ “run_id”: f”{d.strftime(‘%m%d’)}-{region[:2]}-{len(runs)+1:04d}”, “date”: d.strftime(“%Y-%m-%d”), “pipeline”: pipeline, “owner”: owner, “region”: region, “state”: state, “priority”: priority, “duration_s”: duration, “cost_usd”: cost, “revenue_k”: revenue, “sla_gap”: round(max(0, duration – 120) / 60, 1), }) daily_region_rows.append({ “date”: d.strftime(“%b %d”), “region”: region, “runs”: volume, “failures”: failures, “late”: late, “success_rate”: round(100 * (volume – failures – late * 0.35) / volume, 1), “avg_latency”: round(total_latency / volume, 1), “cost_usd”: round(total_cost, 2), “revenue_k”: round(total_revenue, 1), }) runs = sorted( runs, key=lambda r: (r[“priority”], r[“state”] != “Failed”, -r[“duration_s”]) )[:80] def aggregate_daily(rows): by_date = defaultdict(lambda: { “date”: “”, “runs”: 0, “failures”: 0, “late”: 0, “cost_usd”: 0.0, “revenue_k”: 0.0, “latency_weighted”: 0.0, }) for r in rows: bucket = by_date[r[“date”]] bucket[“date”] = r[“date”] bucket[“runs”] += r[“runs”] bucket[“failures”] += r[“failures”] bucket[“late”] += r[“late”] bucket[“cost_usd”] += r[“cost_usd”] bucket[“revenue_k”] += r[“revenue_k”] bucket[“latency_weighted”] += r[“avg_latency”] * r[“runs”] out = [] for d in [x.strftime(“%b %d”) for x in DATES]: b = by_date[d] if b[“runs”]: b[“success_rate”] = round(100 * (b[“runs”] – b[“failures”] – b[“late”] * 0.35) / b[“runs”], 1) b[“avg_latency”] = round(b[“latency_weighted”] / b[“runs”], 1) b[“cost_usd”] = round(b[“cost_usd”], 2) b[“revenue_k”] = round(b[“revenue_k”], 1) del b[“latency_weighted”] out.append(dict(b)) return out def aggregate_regions(rows): by_region = defaultdict(lambda: { “region”: “”, “runs”: 0, “failures”: 0, “late”: 0, “cost_usd”: 0.0, “revenue_k”: 0.0, “latency_weighted”: 0.0, }) for r in rows: b = by_region[r[“region”]] b[“region”] = r[“region”] b[“runs”] += r[“runs”] b[“failures”] += r[“failures”] b[“late”] += r[“late”] b[“cost_usd”] += r[“cost_usd”] b[“revenue_k”] += r[“revenue_k”] b[“latency_weighted”] += r[“avg_latency”] * r[“runs”] out = [] for region in REGIONS[1:]: b = by_region[region] b[“success_rate”] = round(100 * (b[“runs”] – b[“failures”] – b[“late”] * 0.35) / b[“runs”], 1) b[“avg_latency”] = round(b[“latency_weighted”] / b[“runs”], 1) b[“cost_usd”] = round(b[“cost_usd”], 2) b[“revenue_k”] = round(b[“revenue_k”], 1) b[“roi”] = round(b[“revenue_k”] / max(1, b[“cost_usd”]), 1) del b[“latency_weighted”] out.append(dict(b)) return out def make_status_rows(table_rows): counts = Counter(r[“state”] for r in table_rows) return [{“state”: k, “count”: v} for k, v in counts.items()] def make_pipeline_rows(table_rows): counts = Counter(r[“pipeline”] for r in table_rows) return [{“pipeline”: k, “count”: v} for k, v in counts.most_common()] def make_kpis(region, daily_rows, table_rows): runs_count = sum(r[“runs”] for r in daily_rows) failures = sum(r[“failures”] for r in daily_rows) late = sum(r[“late”] for r in daily_rows) cost = sum(r[“cost_usd”] for r in daily_rows) revenue = sum(r[“revenue_k”] for r in daily_rows) return { “region”: region, “runs”: runs_count, “success_rate”: round(100 * (runs_count – failures – late * 0.35) / max(1, runs_count), 1), “avg_latency”: round(sum(r[“avg_latency”] * r[“runs”] for r in daily_rows) / max(1, runs_count), 1), “cost_usd”: round(cost, 2), “revenue_k”: round(revenue, 1), “roi”: round(revenue / max(1, cost), 1), “open_issues”: len(table_rows), “p0p1”: sum(1 for r in table_rows if r[“priority”] in {“P0”, “P1”}), “failure_rate”: round(100 * failures / max(1, runs_count), 2), “spark”: [r[“success_rate”] for r in daily_rows[-14:]], } DAILY_BY_REGION = {“All”: aggregate_daily(daily_region_rows)} REGION_ROWS = aggregate_regions(daily_region_rows) for region in REGIONS[1:]: DAILY_BY_REGION[region] = [r for r in daily_region_rows if r[“region”] == region] RUNS_BY_REGION = { region: [r for r in

MoonMath AI team has released a bf16 forward attention kernel for AMD’s MI300X GPU. It is written in HIP, not hand-written assembly. The code is open-source under the MIT license. The MoonMath.ai team reports it beats AITER v3, AMD’s own optimized kernel, on every tested shape. Bare-metal access came from HotAisle, an AMD cloud provider. Attention is the fused softmax(QKᵀ/√d)·V operation inside every transformer. The MI300X is AMD’s CDNA3 data-center GPU, with the ISA target (gfx942). This kernel runs on that hardware only. TL;DR MoonMath.ai open-sources a bf16 forward attention kernel for AMD MI300X, written in HIP, not assembly (MIT). It beats AMD’s AITER v3 on every shape and rounding mode — geomean 1.18×/1.15×/1.08×, up to 1.26×. The core trick: one-instruction asm wrappers let you pick the opcode while the compiler allocates registers. Most of the speedup is memory placement — K in LDS, V hot in L1, Q and accumulators in registers. A real SGLang PR used it to speed up Wan2.1 video diffusion by 1.23×, with no quality regression. Understanding Kernel A kernel is a small program that runs directly on the GPU’s many cores to perform one specific computation—here, the attention math—as fast as the hardware allows. The kernel computes forward attention in bf16 on MI300X only. It takes inputs in either BSHD or BHSD layout, with no transpose. Head dimension is fixed at 128. It supports any sequence length, including cross-attention. There are real limits. There is no causal mask, no GQA, and no varlen batching. Outputs are bf16, and it runs on gfx942 hardware exclusively. Numerics are tightly controlled. All three rounding modes match AITER’s per-mode rounding rule. Every finite output sits within 1 bf16 ULP of AITER. NaN and Inf handling is bit-identical, and results are deterministic. The Core Trick: One-Instruction asm Wrappers The core technique avoids a familiar dilemma. Compiler intrinsics keep code tidy but let the compiler reorder or rename operands. Raw inline assembly gives control but forces manual register and address management. MoonMath wraps exactly one instruction in a __device__ __forceinline__ function. Extended asm constraints describe the operands. The research team picks the opcode. The compiler still allocates registers and tracks data flow. Copy CodeCopiedUse a different Browser // in/out tied to the SAME VGPR → no accumulator rename, no v_mov copy. __device__ __forceinline__ void asm_mfma(bf16x4_t a, bf16x4_t b, fp32x4_t& c) { asm volatile(“v_mfma_f32_16x16x16_bf16 %0, %1, %2, %0” : “+v”(c) : “v”(a), “v”(b)); } The “+v”(c) constraint ties the accumulator input and output to the same VGPR. No copy instruction is emitted. This keeps the kernel close to ordinary HIP. It still steers the machine one instruction at a time. The Architecture: Eight Waves, Two Groups, Two Barriers A CDNA3 compute unit has four SIMD units. The textbook block is four waves. MoonMath instead runs eight waves per block, in two groups of four. The two groups run the same Q*K, softmax, O += P*V sequence. They are offset by a phase. While one group saturates the matrix core, the other runs softmax and issues loads. Then they swap, so the matrix core never idles. There are two s_barriers per iteration. One sits at the phase handoff. One sits at the iteration boundary. Per-counter waits handle the rest of the synchronization. This echoes FlashAttention-3’s matmul and softmax alternation. It does not copy FA3’s producer and consumer warp split. On CDNA3, every memory move is already asynchronous, so a dedicated producer wave is unnecessary. Where Data Lives, and Why 16×16×16 Most of the speedup comes from memory placement. K streams from HBM into LDS, double-buffered, shared by all eight waves. V stays hot in L1, read on every PV matmul. Q and accumulators live in registers. The research team picked the 16×16×16 MFMA over 32×32×8. Both shapes have identical throughput. The smaller tile accumulates into 4 fp32 elements per lane, against 16. Lower accumulator pressure leaves room for deeper prefetch and a third Q tile. Decision Choice Reason Waves per block 8 (two groups of 4) Plan the pipeline directly; share one K copy MFMA shape 16×16×16 bf16 Same throughput, lower VGPR pressure, better power efficiency K placement LDS, double-buffered, 32 KiB Shared by all 8 waves, swapped per iteration V placement L1, resident, prefetched Reread across PV, kept hot deliberately Q + accumulators VGPRs Read every iteration, never reloaded Two later wins close the gap. A third Q tile (3Q) raises data reuse per loaded K and V tile. A Flash-Decoding-style tail KV split rescues the stranded fractional round across MI300X’s 304 CUs. These wins cascade. Moving V to L1 freed the LDS that the third Q tile then fills. Benchmark Tests ran on MI300X in bf16, head dimension 128. Each shape was measured at three rounding modes. RTNE rounds to nearest even. RTNA rounds to nearest, ties away from zero. RTZ truncates toward zero. Shape (B, H, S, D) Round Ours (ms) AITER v3 (ms) vs AITER vs MAX (2, 24, 8192, 128) RTNE 3.083 3.792 1.23× 1.37× (2, 24, 16384, 128) RTNE 11.670 14.691 1.26× 1.54× (4, 16, 16384, 128) RTZ 15.055 16.183 1.07× 1.47× (2, 24, 32768, 128) RTNA 44.440 52.363 1.18× 1.57× (1, 16, 131072, 128) RTNE 232.517 269.278 1.16× 1.46× Geomeans across the sweep favor MoonMath. Versus AITER, it scores 1.18× (RTNE), 1.15× (RTNA), and 1.08× (RTZ). Versus Modular MAX, geomeans run 1.44× to 1.49×, and per-shape speedups reach 1.59×. RTZ is AITER’s own fastest mode and the tightest race. The (4, 16, 16384) RTZ shape moved from 0.95× to 1.07×. The tail KV split is what closed that final gap. Interactive Explainer Use Cases The kernel installs with pip and exposes a small API. It launches on the caller’s stream, so it overlaps inside larger pipelines. Copy CodeCopiedUse a different Browser import torch import moonmath_attention as ma # PyTorch’s ROCm build uses the “cuda” device string on AMD GPUs q = torch.randn(2, 8192, 24, 128, dtype=torch.bfloat16, device=”cuda”) k = torch.randn(2, 8192, 24, 128, dtype=torch.bfloat16, device=”cuda”) v = torch.randn(2, 8192, 24, 128, dtype=torch.bfloat16, device=”cuda”) out