Uncategorized

Day 100 in production isn’t really about chunking strategies anymore.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. NASA’s new dark energy space telescope can also detect killer asteroids   At the end of August, NASA is set to launch the Nancy Grace Roman Space Telescope from Kennedy Space Center in Florida. Its quest is to help us better understand how the universe works, from the glue-like dark matter that keeps galaxies together to the elusive dark energy that drives the expansion of the cosmos.  But Roman could also serve another purpose: defending Earth from killer asteroids. In September, a multi-institutional team of planetary scientists and astronomers will outline how it’s uniquely placed to scan asteroids, and provide information about their trajectories, sizes and compositions. Read our story about how this new space telescope might boost planetary defense. —Robin George Andrews  MIT Technology Review Narrated: The inevitable weakness of metrics  —Bryan Gardiner There are plenty of useful things a metric can reveal. There are even more that it can obscure or corrupt. Like a lot of people bitten by the self-quantifying bug, I started gathering personal data to pursue a nebulous collection of goals and desires. I wanted to feel better physically and emotionally, get outside more, and bring order to the messiness and uncertainty of my daily existence. But external metrics and data can never capture what’s truly important. Worse, they inevitably redefine your core sense of what’s important, whether you’re aware of the trap or not. Read the full story. This is our latest story to be turned into an MIT Technology Review Narrated podcast, which we publish each week on Spotify and Apple Podcasts. Just navigate to MIT Technology Review Narrated on either platform, and follow us to get all our new content as it’s released. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 The US is considering banning Chinese data center componentsThe news comes just after the FCC announced curbs on other high-tech Chinese hardware. (Reuters $)+ Texas is now requiring data centers to pass an audit before connecting to the grid. (The Verge $)+ Trump’s AI protectionism has come for robotics. (MIT Technology Review) 2 SpaceX rocket has crashed into the moon The collision, at about 5,400 miles per hour, will likely leave a new crater. (Guardian)+ SpaceX posted a loss of over $500 million in its first quarterly earnings report since its IPO. (WP $)+ The company’s future plans rely on the success of its huge Starship rockets. (WSJ $)+ Meanwhile, SpaceX’s spending on AI is soaring. (NYT $) 3 The UK’s AI Safety Institute says it’s uncovered more AI hacksIncluding an Anthropic model setting up fake accounts mimicking real people. (BBC)+ The White House has a new cybersecurity framework…but it’s keeping details under wraps. (Wired $)+ Here’s why AI agents lie and cheat to reach their goals. (MIT Technology Review) 4 The future’s looking bright for French AI lab MistralTurmoil in the US is creating opportunities for European AI that it hopes to seize. (Wired $) 5 People are betting on wildfires on prediction markets Which, as US senators warn, sure seems like it will incentivize arson. (Ars Technica)+ Prediction markets are also starting to put the accuracy of weather predictions at risk. (MIT Technology Review)  6 Can Reddit save itself from being swamped by AI search spam?It’s becoming increasingly influential, and brands are desperate to secure mentions. (The Verge $) 7 Grassroots groups around the world are trying to “humanize” AIPart of their work is educating people so they can have a say on their futures. (Rest of World) + The World Bank says poorer countries have “less to fear” from AI than rich ones. (FT $) 8 A Chinese physical AI startup has run into controversy Spirit AI stands accused of “benchmark hacking” after it briefly overtook Nvidia on an AI leaderboard in June. (SCMP$)+ How to build a better AI benchmark. (MIT Technology Review) 9 Pluto’s atmosphere may be starting to collapseAnd after the loss of its planet status, too. Really adds insult to injury. (Gizmodo) 10 Even Microsoft is begging its engineers to stop using AI so muchKinda telling that even Big Tech firms are struggling to find ROI for their spending. (404 Media) Quote of the day “We’re essentially trying to criminalize government scientists giving advice to political leaders, if that is advice the political leaders don’t like or it creates an outcome the political leaders didn’t want.”  —Ashish Jha, former COVID response coordinator, tells Axios why scientists are becoming increasingly wary of taking US government jobs. One More Thing CHRIS LABROOY Meet the Vitalists: the hardcore longevity enthusiasts who believe death is “wrong”  Last April, an excited crowd gathered at a compound in Berkeley, California, for a three-day event called the Vitalist Bay Summit. It was part of a longer, two-month residency that hosted various events to explore tools—from drug regulation to cryonics—that might be deployed in the fight against death.  One of the main goals, though, was to spread the word of Vitalism, a somewhat radical movement established a few years ago. Consider it longevity for the most hardcore adherents—a sweeping mission to which nothing short of total devotion will do. Vitalists don’t just want to live longer. They believe that defeating death should be humanity’s top concern. Read our story to learn more about them, and their philosophy.  —Jessica Hamzelou We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + This yellow labrador’s love of walking is a joy to behold.+ Bowling legend Phyllis is still rolling perfect games at 95 years old.+ Here are some evidence-based ways to cut back on life’s many distractions.+ Humans since 1982 turns clock faces into mesmerizing kinetic art that still functions as timepieces.

Meta AI has released Muse Code (in beta), a terminal coding agent in beta, powered by its new Muse Spark 1.2 model. Meta positions the pair as its next step toward the frontier, with larger models on the way. Muse Code targets complex software engineering across large repositories: it plans changes, writes code, and validates the results. A set of async background agents stays alive for the whole session instead of spawning per task. A local append-only event log records every model call, tool run, approval, and edit, which Meta calls replay-exact and restart-safe. Muse Spark 1.2 was co-trained with the harness itself. Meta also published a kernel-optimization case study running 1,000+ tool calls over as long as 24 hours. Is it deployable Yes. Muse Code ships in beta for macOS and Linux via curl -fsSL https://dev.meta.ai/install.sh | bash. Muse Spark 1.2 is available in Muse Code and the Meta Model API, with expanded global access. The launch post does not mention downloadable weights, so treat this as a hosted dependency. Company level: The API path fits any size. The Muse Code path fits teams already running agents in sandboxes with review gates. Industries: Software and SaaS, developer tooling, fintech engineering, GPU and inference infrastructure, semiconductors and HPC. Applications: Repository-scale refactors and migrations, long-running bug triage, test generation, and GPU kernel optimization. Async background agents Muse Code runs a simple agent loop plus a set of async background agents. These specialized agents remain active throughout each session. They are not spawned for individual tasks, which Meta says avoids redundant information gathering. They carry out next steps and choose when to report back to the main agent. Meta states this persistence reduces latency and steering on difficult, multi-step tasks. Runtime design Muse Code uses a local event log. Every model call, tool run, approval, and edit is appended to it. Meta calls this single source of truth replay-exact and restart-safe. After a crash, the agent resumes precisely where it stopped, letting long-running tasks survive failures. Bundled skills Three default skills ship with the agent. /plan turns a task into an approval-gated plan. /grill stress-tests that plan until it holds up. /goal works toward successful completion of the specified objective. What changed in Muse Spark 1.2 Muse Spark 1.2 is a coding-focused update to Muse Spark 1.1. Meta reports gains in code generation, complex debugging, codebase understanding, and end-to-end developer workflows. The research team significantly scaled up training compute on coding tasks and expanded environment diversity. The model keeps its strength in other areas, including general agents. Three important training details: Co-training with the harness: Muse Spark 1.2 was co-trained with Muse Code. Training included rejection-sampled harness trajectories and recipe optimizations for goals, compaction, and subagents. The Muse Code toolset was integrated to maximize harness compatibility. Long-horizon training: Training covered whole-repository generation, large end-to-end projects, and auto-research. The model uses planning, goal conditioning, and context compaction to sustain progress. Self-improvement: Muse Spark 1.1 generated challenging coding environments and instruction-following templates, then graded candidate solutions against those requirements. That produced a scalable training dataset for 1.2. https://research.meta.ai/blog/introducing-muse-code-and-muse-spark-1-2 Evaluation Meta’s methodology report is unusually specific. Terminal-Bench 2.1 uses all 89 tasks, pass@1 over five attempts. DeepSWE v1.1 covers 113 tasks across 91 repositories and five languages. Meta Internal Coding Bench holds 440 tasks derived from real internal pull requests. Runs execute in isolated Daytona cloud sandboxes. Comparisons include Grok 4.5, Claude Opus 5, GPT-5.6 Terra, Gemini 3.6 Flash, and Kimi K3, each with its own agent product. Meta notes its harness may not be tuned for third-party models. For reference, Meta’s model page lists Muse Spark 1.1 at 80.0 on Terminal-Bench 2.1. Case study: kernel optimization Meta tested iterative GPU kernel optimization over 1,000+ tool calls, running up to 24 hours. The model writes, compiles, profiles, and progressively improves kernels against a provided baseline. Benchmarks covered KDA and MLA kernels on NVIDIA Hopper GPUs. For KDA, the baseline is the FLA Triton implementation, with third-party kernel libraries prohibited. Muse Spark 1.2 paired a chunk-parallel preparation kernel with a sequential inter-chunk scan. For MLA, the reference is PyTorch at batch size 1, 64 heads, sequence length 8192, and latent dimension 512. The model built a two-kernel Triton pipeline that reuses the shared KV latent as both K and V. Key Takeaways Muse Code is a beta terminal coding agent for macOS and Linux, powered by Muse Spark 1.2. Persistent async background agents replace per-task spawning to cut redundant information gathering. An append-only local event log makes the runtime replay-exact and restart-safe after crashes. Muse Spark 1.2 was co-trained with the harness and trained on long-horizon, repository-scale work. Kernel case study ran 1,000+ tool calls over 24 hours on NVIDIA Hopper KDA and MLA kernels. Check out the Technical details, Model (Muse Spark 1.2) and Evaluation Methodology. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Meta AI Releases Muse Code (Beta): A Terminal Coding Agent Powered by the New Muse Spark 1.2 Model appeared first on MarkTechPost.

Day 100 in production isn’t really about chunking strategies anymore.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Trump’s AI protectionism has come for robotics   —James O’Donnell  Humanoid robots usually elicit more cringe than awe: They stumble, kick children, and despite advances are still worse at using their hands than my toddler. It’s a nascent industry, and such robots are more commonly seen in viral videos than real workplaces or homes.  It was a surprise, then, when last week the Federal Trade Commission issued a sweeping ban on foreign imports of advanced robots, including humanoids, quadrupeds, and wheeled robots.  The decision should be understood not as another chapter in the old China trade playbook, but as evidence that the Trump administration is expanding its protection of the AI industry beyond today’s leading labs. It is now willing to step in on behalf of an emerging robotics sector that is still barely finding its footing. Read our analysis to understand the ban’s potential impact. This story is from The Algorithm, our weekly newsletter all about the latest goings-on in the world of AI. Sign up to receive it in your inbox every Monday. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 ICE collected nearly one million people’s DNA last year Most of them have never been convicted of any crime. (Wired $) 2 China is poised to win the technologies of the futureIt’s already a high-tech hardware powerhouse, but it increasingly has its sights set on software too. (New Yorker $)+ US tech and political leaders are freaking out, but few seem to agree how to respond. (Vox $) + What’s next for Chinese open-source AI. (MIT Technology Review) 3 AI “tokenomics” is a burgeoning new field Businesses are pouring a lot of money into AI. Now they want to see what they’re getting in return. (NYT $)+ Why it’s proving so hard to make AI pay. (BBC)+ The US economy is becoming more and more reliant on the AI boom. (WSJ $) 4 Eli Lilly is letting people apply to try an unapproved obesity drugRetatrutide is still in clinical trials, but certain patients can gain early access. (STAT $) + Montana’s plan to become an experimental medical hub just pushed forward. (MIT Technology Review) 5 Inside the one US town that wants a data center Jay in Maine is a reminder that politics is all about the local. (The Atlantic $)+ How data centers broke US politics. (Wired $) 6 Flock license plate readers can have a shockingly high error rateIn one California town, Flock misread license plates in 71% of the alerts it sent to police. (BI $)+ A leaked guide shows how Flock teaches cops to promote its tech. (404 Media $) 7 A drone explosion on a beach in Russia killed seven people It seems to have been caused by Russian forces shooting down a Ukrainian drone. (CNN)+ A US company won a $100 million deal to give Ukrainian drones an AI upgrade. (Ars Technica)+ Europe’s drone-filled vision for the future of war. (MIT Technology Review) 8 How car headlights became so brightLots of modern cars still blind other drivers on the road. That could change soon though, thanks to new tech. (Ars Technica) 9 A $2 million crime novel deal collapsed over AI use concernsAnd the agent and the writer involved have, erm…rather differing accounts of what happened. (Guardian) 10 AI matchmaking services are on the rise Online daters hope it might prove better at finding them love than fruitless swiping. (WSJ $) Quote of the day “Just a little too pleasant to be human.” —North Carolina resident Kristen Charpentier tells Wired how she could tell she was talking to AI when ordering at a Dairy Queen drive-thru. One More Thing GETTY IMAGES How to have a child in the digital age  Before journalist and culture critic Amanda Hess even got pregnant with her first child, in 2020, the internet knew she was trying. She saw pregnancy ads way before a doctor.    Hess’s experience is pretty typical these days, but still raises some big questions. How do we retain control over our bodies when corporations have access to our most personal information? What happens when people stop relying on friends and family for advice on having a kid and instead go online?  Read our interview with Hess to learn what she has to say.  —Alison Arieff We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Brighten your day with this eight-minute video of the world’s cutest baby animals.+ Intertapes has assembled a remarkable treasure trove of cassette tapes found around the world.+ Armchair basketball GMs can finally put their talents to the test in this game that lets you draft an elite NBA team.+ Here’s a hilarious look at the lost recordings of Robin Williams voicing the Genie in Disney’s “Aladdin.”

In this article, you will learn how static, dynamic, and continuous batching work in LLM inference, and why the differences between them matter at production…

This chapter is divided into eight parts; they are: • Metrics for LLM Inference • Measuring a Single Request • Warmup and Synchronization • Measuring GPU Work with CUDA Events • Measuring Memory Usage • Measuring Concurrent Requests • Multiple GPUs and Multiple Machines • Cost per Token The most common inference metrics are: • Latency:  How long a request takes from start to finish.

Cursor Research has open-sourced Mixture-of-Kittens (MoK), the mixture-of-experts training megakernel behind its Composer models. MoK fuses every MoE communication and computation step into a single deterministic kernel. Cursor team reports up to 2.37x higher throughput than the strongest public baseline. It already powers Composer training across tens of thousands of GPUs. Is it deployable Yes, but the hardware floor is high. MoK is on GitHub under Apache-2.0. It requires NVIDIA Blackwell SM100 or SM103 GPUs, which means GB200 NVL72 or GB300 NVL72 racks. It also needs Python 3.12+, PyTorch 2.10+, and CUDA toolkit 13.0+. Inter-GPU buffers rely on PyTorch symmetric memory. That limits realistic adopters to organizations that own or rent NVL72 capacity. Frontier labs, funded model startups, GPU neoclouds, and national computing centers fit. Single-node teams and 8-GPU shops do not. Applications are narrow but high-value. They include pretraining and post-training of DeepSeek-V3-style MoE models. Determinism also makes it useful for on-policy RL post-training and internal ablations. Relevant industries are AI model development, cloud GPU infrastructure, code-generation tooling, and quantitative research. MoE layer as the bottleneck Cursor’s earlier work covered the compute side. The research team wrote its own MXFP8 and NVFP4 training kernels and a ‘warp decode’ path for MoE inference. Those assumed inter-GPU communication was handled separately. In production, communication became the limiting factor. The MoE layer can consume more than half of end-to-end training time. Moving to GB300 NVL72s changed the problem again. A rack is 72 GPUs inside one NVLink domain, which allows fine-grained overlap. But the integrated Grace CPUs are slow relative to the GPUs. CPU-GPU synchronization therefore has to be minimized aggressively. Three design decisions that matter Communication direction is chosen per operation: Existing approaches such as DeepEP lean on push-based transfers. Cursor’s microbenchmarks show push moves fewer total bytes in one direction. That leaves the reverse NVLink lane mostly idle. Pull-based dispatch delivers up to 29% higher NVLink bandwidth utilization under expert imbalance. It also eliminates cross-GPU completion signals. Push dispatch signalling measured 103 µs against 18 µs for pull, roughly 5.8x. MoK therefore uses pull-based forward dispatch and push-based forward combine. The backward pass mirrors this with pull reverse-combine and push reverse-dispatch. One schedule table serves all four, costing under 3% of MoE runtime. Overlap granularity sits between the extremes: Comet is fine-grained; DeepEP is coarse-grained. Cursor team argues the optimum is in the middle and workload-dependent. The heuristic targets at least two full SM waves per expert-grouped GEMM. For Kimi 2.5 shapes, the base model for Composer 2.5, the floor is 2,368 tokens. Measured latency matches that estimate closely. A ring token buffer removes the CPU from the loop: The alternatives are dropping tokens or asking the CPU to size buffers. MoK instead cycles a fixed ring buffer of a few hundred megabytes. It does so at minibatch granularity, interleaving dispatch and combine at macrobatch boundaries. The ring is walked in reverse to minimize forward activation replay during backward. MoK is built as a megakernel and is fully deterministic. It supports BF16 and MXFP8 precision modes. Scheduling runs through Blackwell’s Cluster Launch Control, so inter-rack RDMA does not serialize behind it. Router weight gradients use a SonicMoE-style calculation fused into the SwiGLU backward. https://cursor.com/blog/mixture-of-kittens Results Layer benchmarks ran in a single NVL72 rack at EP degree 64. Each GPU held 2,048 tokens before routing. Baselines were NCCL+PyTorch, DeepEP+PyTorch, DeepEP+TransformerEngine, and HybridEP+Megatron. Shapes covered Kimi K2.7 Code, GLM-5.2, Qwen3.5-397B-A17B, and DeepSeek-V4-Pro. Against the fastest baseline, MoK is up to 2.37x faster for MXFP8 forward. The other figures are 1.78x MXFP8 backward, 1.92x BF16 forward, and 1.58x BF16 backward. End-to-end testing used 512 GPUs across several GB300 NVL72 racks. Tokens per second per GPU rose from 760.9 to 1,070.2, a 1.41x gain. Key Takeaways MoK fuses all MoE communication and computation into one deterministic megakernel for NVL72 racks. Pull dispatch plus push combine cuts signalling from 103 µs to 18 µs. A ring token buffer drops zero tokens and removes CPU-GPU synchronization entirely. Up to 2.37x over the fastest public baseline; 1.41x end-to-end on 512 GPUs. Apache-2.0, but it demands Blackwell SM100/SM103, CUDA 13.0+, and PyTorch 2.10+. Check out the GitHub Repo and Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Cursor Open-Sources Mixture-of-Kittens (MoK): A Deterministic MoE Training Megakernel for GB300 NVL72 Racks appeared first on MarkTechPost.