Committee

Large hybrid MoE models like Nemotron-3-Super are accurate but expensive to serve. Their active parameters, KV cache, and Mamba state cap how many users a node can hold at a given per-user token rate. NVIDIA AI team has released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super. The parent model has 120.7B total and 12.8B active parameters. The compressed model has 75.3B total and 9.3B active parameters. The deployment target was fixed before the architecture search began. Target one was 2x server throughput at 100 tokens per second per user. Target two was 8 concurrent 1M-token requests on a single H100. Three checkpoints on Hugging Face: BF16, FP8, and NVFP4. TL;DR 120.7B/12.8B active compresses to 75.3B/9.3B active, with the 88-block hybrid layout preserved. 8xB200 total throughput rises 1.60x to 2.14x over Super at matched NVFP4 and matched user throughput. Single-H100 1M-token concurrency goes 1 to 8, driven by a 70 GB to 44.5 GB weight drop. Iterative Puzzle beats single-step Puzzle by 0.57 average points at the same compression target. Arena-Hard-V2 (-4.2) and SWE-Bench (-2.6) are the real costs; RULER and AA-LCR barely move. Nemotron-Labs-3-Puzzle-75B-A9B Nemotron-3-Super is a hybrid Mamba-Transformer MoE model. Puzzle-75B-A9B preserves the parent’s block layout exactly. It has 88 blocks: 40 Mamba, 40 MoE, and 8 attention blocks. What changed is capacity inside those blocks: Quantity Super Puzzle-75B-A9B Ratio Total parameters 120.7B 75.3B 62.4% Active parameters 12.8B 9.3B 73.1% Mamba SSM state size 128 96 75% MoE routed expert intermediate size 2688 1280-2688 Mean 59.9% Activated routed experts per token 22 4-18 Mean 50% Active routed expert capacity (relative) 100% 8.7%-62.3% Mean 30.9% The number of routed experts, the shared expert size, and the MoE latent size are unchanged. Attention layers were left untouched. The proposed research’s stated reason is that Nemotron-3-Super is already very KV-cache efficient. Mamba layers were pruned uniformly, because inference frameworks do not support a different SSM state size per layer. https://arxiv.org/pdf/2607.04371 The result is not a uniformly scaled-down teacher. The above figure shows the allocation across depth. Puzzle preserved capacity in selected middle and late layers, and cut hard elsewhere. Benchmark and Performance The below table reports Pareto-optimal total throughput on a single 8xB200 node, with single-step decoding. Scenario (in/out) UT floor Super (tok/s) Puzzle-75B-A9B (tok/s) Boost 50K / 2K >= 100 5,128 8,210 1.60x 50K / 2K >= 125 3,784 6,412 1.69x 50K / 2K >= 150 2,532 4,523 1.79x 8K / 64K >= 100 20,939 42,601 2.03x 8K / 64K >= 125 13,074 27,918 2.14x 8K / 64K >= 150 8,522 18,047 2.12x Both models were served at matched NVFP4 weights, FP8 KV cache, and FP16 Mamba state. The gap therefore reflects compression, not a change in numeric format. The prefill-heavy 50K/2K regime gains least. The decode-heavy 8K/64K regime gains most. On a single 8xH100 node at UT = 100, the gains are smaller. They are 1.91x on 50K/2K and 1.82x on 8K/64K. Both models there use FP8 weights, FP8 KV cache, and FP32 Mamba state. On a single H100 at 1M context, the binding constraint flips from compute to memory. Super’s NVFP4 weights occupy about 70 GB of the 80 GB HBM budget. Each 1M-token request adds about 4 GB of KV cache. Effective concurrency is therefore 1. Puzzle-75B-A9B’s NVFP4 weights occupy about 44.5 GB. Attention layout is unchanged, so per-request KV cost is unchanged. Concurrency at 1M rises to 8. Aggregate decode throughput at that concurrency is roughly 4x Super’s single-request throughput. Prefill of a 990K-token prompt is about 1.2x faster. How Iterative Puzzle Works Puzzle is a decomposed neural architecture search framework, implemented here as Puzzletron. It defines a discrete search space of alternative layer implementations. Each alternative gets a quality score. A mixed-integer program then selects one alternative per layer under a deployment constraint. Three pruning techniques form the search space: Intermediate channel pruning: Channels inside each routed expert are ranked by contribution to the expert’s output. All experts within one MoE layer are pruned to a uniform size, for kernel compatibility. Top-k reduction: The number of experts a token is routed to varies per layer, up to the parent’s k=22. Mamba SSM pruning: The SSM state size drops from 128 to 96 channels. The SSM result is measured. Dropping 128 channels to 96 speeds the SSM kernel 1.2x to 1.3x during decode. This holds at batch sizes between 8 and 512. Channels were ranked by estimated contribution to the Mamba layer output. The estimate averaged over 67M tokens of validation data. Appendix A shows this beats random channel selection under aggressive pruning. The original formulation assumes replacement quality impacts are approximately additive. Each candidate block is scored inside the unmodified parent. That ignores higher-order interactions between replacements. Iterative Puzzle alternates bounded compression with short knowledge distillation recovery. It builds a sequence M0, M1, … MR instead of jumping to the target. Scores are recomputed against the current compressed model, not the original parent. Three stages were used: MoE weights to 75% of teacher capacity, Mamba SSM state to 75%. Healed for 24B tokens. MoE weights to 60% of teacher capacity. Healed for 43.2B tokens. Activated routed-expert budget to 50%, allocated heterogeneously. Healed for 52.8B tokens. https://arxiv.org/pdf/2607.04371 The above table compares this against a single-step Puzzle baseline at the same target. The three-step procedure averages 69.05 across ten benchmarks, against 68.48. Gains appear on MMLU-Pro, GPQA, HLE, AA-LCR, LiveCodeBench, SciCode, and RULER-256K. IFBench-Instruction fell 0.2 points and IFBench-Prompt fell 0.5. Recovery: Distillation, RL, and Verbosity Knowledge distillation ran on 30% pretraining data and 70% SFT data from Nemotron-3-Nano. During the Puzzle phase, KD used a 32K sequence length. Recovery then trained at 128K, and scaled to 512K. The budget was up to 100B tokens, with a 16M-token global batch, in Megatron-LM. RL post-training adopted Stage 2 of the Nemotron-3-Super RL pipeline, focused on software engineering. Phase 2.1 did single-step tool-use comparison. Phase 2.2 moved to end-to-end sandbox RL, where agents run up to 200 turns. Both phases used a KL penalty of 0. The team swept learning

The default assumption in most LLM developer communities is that you start with raw API calls and graduate to a framework as your project grows.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Four nuclear reactors hit a big milestone in the US —Casey Crownhart I was really looking forward to July 4, and not just because I love a poolside barbecue. This year the American holiday also marked a big symbolic deadline for US nuclear power. Last year the Trump administration set a goal to see three new microreactors achieve criticality, a technical milestone establishing that a reactor can sustain a chain reaction, by the nation’s 250th birthday. And just in time, not just three, but four reactors did so. It’s a positive sign for nuclear technologies at a time of increasing need for electricity and emissions-free energy sources. But achieving criticality doesn’t mean a reactor is ready to provide electricity for the grid (or at all, for that matter). Here’s what the milestone could mean for nuclear power in the US—and where the four companies might go next. This story is from The Spark, our weekly climate tech newsletter. Sign up to receive it in your inbox every Wednesday. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 China plans to let its top AI firms buy Nvidia H200 chipsAlibaba, ByteDance, and DeepSeek are set to get permission. (Information $)+ China had previously withheld approval despite US authorization. (Reuters $) 2 NATO is building a network to stop Russian attackers in their tracksIt will use sensors, drones, satellites, and AI to detect them. (Business Insider)+ Troops are donning odd camouflage to elude drones. (Economist $)+ The US wants cheaper drones as Iran’s wrecking its Reapers. (Ars Technica) 3 Researchers have a new idea to fight future El Niños: dimming the sunDeflecting solar energy could cool the ocean and mitigate the risks. (Wired $)+ But there could be unexpected consequences. (New Scientist $)+ And geoengineering as a field is getting a reality check. (MIT Technology Review) 4 Meta is patenting an AI device that records users to analyse emotionsIt ostensibly aims to tailor workout plans to the user’s mood. (404 Media)+ AI memory is privacy’s next frontier. (MIT Technology Review) 5 Chipmakers are going vertical as Moore’s Law slowsThey’re stacking transistors to keep chips advancing. (Economist $)+ IBM is betting on the technique. (MIT Technology Review) 6 Ivy League students suspected of AI cheating saw scores fall in personFrom 96% all the way down to 48%. (Ars Technica)+ AI giants want to take over the classroom. (MIT Technology Review) 7 A new study says parents’ phone addictions damage bonds with kidsIt can exacerbate “insecure attachment” for life. (Bloomberg $)+ And make children more anxious and avoidant. (Gizmodo) 8 A judge approved Musk’s $1.5 million Twitter settlement with the SECDespite what she called “serious misgivings” and “red flags.” (Reuters $)+ Musk was accused of skirting stock disclosure rules. (Fortune) 9 Shoebox-sized “detector satellites” could find nuclear bombs in spaceCubesats carrying the detector could sense a bomb’s radiation. (Space)+ Russia is suspected of developing space-based nukes. (Reuters $) 10 A World Cup match drove Google Search traffic to a new recordThe milestone came after Argentina’s comeback against Egypt. (CNBC) Quote of the day “I talk about it on Tic Tac.” —President Donald Trump tells the public where to find his insights on the dangers of communism, Gizmodo reports. One More Thing Robots are bringing new life to extinct species Paleontologists aren’t easily deterred by evolutionary dead ends or a sparse fossil record. And in the last few years, they’ve developed a new trick for turning back time and studying prehistoric animals: building experimental robotic models of them.  In the absence of a living specimen, an ambling, flying, swimming, or slithering automaton is the next best thing for studying the behavior of extinct organisms. Learning more about how they moved can in turn shed light on their lives, such as their historic ranges and feeding habits. Scientists can simply sit back and observe their behavior in different environments.  Read the full story on the rise of paleo-inspired robots—and four examples that are shedding light on creatures of yore. —Shi En Kim We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Georgia Hill’s monochrome artworks are filled with visual harmony.+ AI has salvaged text from a papyrus scroll burned to a crisp when Mount Vesuvius erupted 2,000 years ago.+ Rare images taken by a Japanese space probe show a near-Earth asteroid resembling a cuddly snowman.+ “Another One Bites the Bee Gees” smoothly merges two classic tracks with a 4/4 time signature into the perfect song for applying CPR.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Four nuclear reactors hit a big milestone in the US —Casey Crownhart I was really looking forward to July 4, and not just because I love a poolside barbecue. This year the American holiday also marked a big symbolic deadline for US nuclear power. Last year the Trump administration set a goal to see three new microreactors achieve criticality, a technical milestone establishing that a reactor can sustain a chain reaction, by the nation’s 250th birthday. And just in time, not just three, but four reactors did so. It’s a positive sign for nuclear technologies at a time of increasing need for electricity and emissions-free energy sources. But achieving criticality doesn’t mean a reactor is ready to provide electricity for the grid (or at all, for that matter). Here’s what the milestone could mean for nuclear power in the US—and where the four companies might go next. This story is from The Spark, our weekly climate tech newsletter. Sign up to receive it in your inbox every Wednesday. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 China plans to let its top AI firms buy Nvidia H200 chipsAlibaba, ByteDance, and DeepSeek are set to get permission. (Information $)+ China had previously withheld approval despite US authorization. (Reuters $) 2 NATO is building a network to stop Russian attackers in their tracksIt will use sensors, drones, satellites, and AI to detect them. (Business Insider)+ Troops are donning odd camouflage to elude drones. (Economist $)+ The US wants cheaper drones as Iran’s wrecking its Reapers. (Ars Technica) 3 Researchers have a new idea to fight future El Niños: dimming the sunDeflecting solar energy could cool the ocean and mitigate the risks. (Wired $)+ But there could be unexpected consequences. (New Scientist $)+ And geoengineering as a field is getting a reality check. (MIT Technology Review) 4 Meta is patenting an AI device that records users to analyse emotionsIt ostensibly aims to tailor workout plans to the user’s mood. (404 Media)+ AI memory is privacy’s next frontier. (MIT Technology Review) 5 Chipmakers are going vertical as Moore’s Law slowsThey’re stacking transistors to keep chips advancing. (Economist $)+ IBM is betting on the technique. (MIT Technology Review) 6 Ivy League students suspected of AI cheating saw scores fall in personFrom 96% all the way down to 48%. (Ars Technica)+ AI giants want to take over the classroom. (MIT Technology Review) 7 A new study says parents’ phone addictions damage bonds with kidsIt can exacerbate “insecure attachment” for life. (Bloomberg $)+ And make children more anxious and avoidant. (Gizmodo) 8 A judge approved Musk’s $1.5 million Twitter settlement with the SECDespite what she called “serious misgivings” and “red flags.” (Reuters $)+ Musk was accused of skirting stock disclosure rules. (Fortune) 9 Shoebox-sized “detector satellites” could find nuclear bombs in spaceCubesats carrying the detector could sense a bomb’s radiation. (Space)+ Russia is suspected of developing space-based nukes. (Reuters $) 10 A World Cup match drove Google Search traffic to a new recordThe milestone came after Argentina’s comeback against Egypt. (CNBC) Quote of the day “I talk about it on Tic Tac.” —President Donald Trump tells the public where to find his insights on the dangers of communism, Gizmodo reports. One More Thing Robots are bringing new life to extinct species Paleontologists aren’t easily deterred by evolutionary dead ends or a sparse fossil record. And in the last few years, they’ve developed a new trick for turning back time and studying prehistoric animals: building experimental robotic models of them.  In the absence of a living specimen, an ambling, flying, swimming, or slithering automaton is the next best thing for studying the behavior of extinct organisms. Learning more about how they moved can in turn shed light on their lives, such as their historic ranges and feeding habits. Scientists can simply sit back and observe their behavior in different environments.  Read the full story on the rise of paleo-inspired robots—and four examples that are shedding light on creatures of yore. —Shi En Kim We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Georgia Hill’s monochrome artworks are filled with visual harmony.+ AI has salvaged text from a papyrus scroll burned to a crisp when Mount Vesuvius erupted 2,000 years ago.+ Rare images taken by a Japanese space probe show a near-Earth asteroid resembling a cuddly snowman.+ “Another One Bites the Bee Gees” smoothly merges two classic tracks with a 4/4 time signature into the perfect song for applying CPR.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Why worms (and microbes) are catching on as a manure pollution solution Anthony Agueda, a third-generation California dairy farmer, pulls a rake through a bed of dark, wet wood chips to reveal a half-dozen squirming red earthworms. There are likely hundreds of thousands more wriggling just under the surface. The worms and microbes are part of a “vermifiltration” system that cleans manure wastewater. The approach may dramatically cut methane, nitrous oxide, and water pollution. Vermifiltration is just one of a variety of methods that farmers, companies, and scientists are employing to drive down manure pollution as the livestock industry faces growing pressure to address the environmental harms from one of the smelliest parts of the business. Explore how the humble earthworm could reshape the future of sustainable farming. —James Temple MIT Technology Review Narrated: geoengineering gets a reality check Solar geoengineering, the controversial idea that we could deliberately intervene in the climate system to counteract global warming, is moving beyond computer simulations and into the practical engineering challenges required to make it real. Researchers are now working on aircraft, materials, and other systems for solar geoengineering. But as they delve into these details, they’re finding that even early deployment would require significant new infrastructure, time, and investment. —James Temple This is our latest story to be turned into an MIT Technology Review Narrated podcast, which we publish each week on Spotify and Apple Podcasts. Just navigate to MIT Technology Review Narrated on either platform, and follow us to get all our new content as it’s released. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 The Trump administration has lifted restrictions on OpenAI’s GPT 5.6The green light came after additional testing and meetings. (Axios)+ OpenAI subsequently said it will launch widely tomorrow. (Bloomberg $) + The rollout had been delayed due to security concerns. (Verge)+ Does AI know too much? (MIT Technology Review) 2 China is looking at curbing overseas access to its top AI modelsAlibaba, ByteDance, and Z.ai attended meetings about the plan. (Reuters $)+ Beijing is also weighing the security risks of open-weight AI. (SCMP)+ And has issued a “backdoor” security alert over Claude Code. (CNBC) 3 European NATO allies have unveiled a $50 billion high-tech missile planThey will engineer stealth and high-speed hypersonic weapons. (BBC)+ Which can strike targets at least 300 km away. (Reuters $)+ The Dutch and British are also developing amphibious ships. (Bloomberg $) 4 Meta is testing “super sensing” AI glasses that record every momentIt plans to disable privacy LEDs that alert people when they’re “on.” (FT $)+ It’s also released an AI image generator. (NYT $)+ Which lets anyone use your Instagram photos in AI images. (Wired $) 5 China’s DeepSeek is developing its own AI chip, sources sayIt could reduce the company’s reliance on Nvidia and Huawei. (Bloomberg $)+ DeepSeek V4 was a win for Chinese chipmakers. (MIT Technology Review) 6 Wikipedia is fighting to survive the internet’s next eraIt’s under attack from MAGA, AI raids, and repressive regimes. (NYT $)+ AI has given Wikipedia a language problem. (MIT Technology Review) 7 SpaceX plans to launch its first model coproduced with CursorThe new frontier model could arrive as soon as this week. (Information $)+ It’s built with AI startup Cursor, which SpaceX is buying for $60 billion. (FT $) 8 A new academic “humanizer” tool can erase signs of AI-written textBut researchers are very divided over its potential impact. (Nature $) 9 Scientists have detected a mystery chemical on Pluto and TitanIt appears to absorb light in a way we don’t currently understand. (Wired $) 10 A Waymo robotaxi reportedly called the cops on drinking teensOfficers then approached the vehicle with guns drawn. (404 Media) Quote of the day “Parents do you know where your teens are? Waymo does!”  —Local police post on Facebook that a Waymo in California called the cops on two teenagers for “drinking and shooting from the vehicle.” One More Thing MICHAEL BYERS Your boss is watching Dora Manriquez has spent nine years driving for Uber and Lyft, where every ride she accepts or rejects is tracked by the apps she relies on for work. Having found herself unable to score enough better-­paying rides, she has had to file for bankruptcy.  App-based employers aren’t the only ones keeping a very close eye on workers today. Jobs today—whether in an office, a warehouse, or your car—can mean constant electronic surveillance with little transparency, and potentially with livelihood-ending consequences if your productivity flags. All that data is shifting the relationships between workers and managers—and protections are lagging. Read the full story on the widening power imbalance it’s created. —Rebecca Ackermann We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + Literary worlds collide in this marvellous Dr Seuss/Stephen King mashup.+ A daring snorkeler saved a dolphin from a suckerfish—and then celebrated with its whole pod.+ This clever musical project seamlessly constructs an original song from vocal snippets of 50 artists singing US city names.+ A long-lost wallet from 1970 was recently unearthed, creating a cute time capsule from its owner’s high school years.

Anthony Agueda, a third-generation California dairy farmer, pulls a rake through a bed of dark, wet wood chips on his family’s land in Hickman, a tiny town in the state’s agricultural heartland. He reaches down with both hands and pulls up a clump of muck, turning it over to reveal a half-dozen squirming red earthworms. There are likely hundreds of thousands more wriggling just under the surface of the three-foot mound of wood and crushed river rock before us, which stretches across the equivalent of six football fields. These natural materials form a biofilter that may dramatically cut the methane, nitrous oxide, and water pollution generated by the massive amounts of manure that hundreds of Holstein cows produce each day. Agueda’s family business, the Alberto Dairy, was one of the first cattle operations in California to adopt this approach to manure treatment, developed and patented by the Chilean company BioFiltro. Eight more of these so-called vermifiltration systems are already operating on US dairies, according to the company, while another 16 are under construction or set to be next year, nearly all of them in California.  Vermifiltration is just one of a variety of methods that farmers, companies, and scientists are employing to drive down manure pollution as the livestock industry faces growing pressure to address the environmental harms from one of the smelliest parts of the business. California, easily the nation’s largest milk producer, has established a handful of programs to promote their adoption, including one initiative that has funneled more than a billion dollars to farms. Researchers stress that much more work needs to be done to determine the most effective approaches, the trade-offs between them, and their success over the long term, under actual farm conditions. Agueda says that he and his family recognized the need to adopt new practices as environmental rules tightened. They were drawn to vermifiltration because it’s simple and relatively cheap compared with other, higher-tech options. “California daily farmers are constantly facing more and more regulation,” says Agueda, standing alongside one of the farm’s free-stall barns. “This makes me excited, because it shows how we are part of the solution.” The growing manure problem Manure is responsible for a significant portion of the climate pollution from livestock operations. The World Resources Institute estimates that manure management on dairy and swine farms accounts for 1.6% of the US’s greenhouse-gas emissions. Globally, manure storage and processing makes up about 10% of the livestock industry’s contributions to climate change.  “Farms have become larger in the past two decades or so, so there’s much more manure—and that has to be stored somewhere,” says Swati Hegde, the organization’s global manager of agricultural methane. Typically, cattle and swine farms spray manure into lagoons or tanks, creating a foul-smelling, low-oxygen slurry in which microorganisms known as methanogens thrive. They gobble up hydrogen, carbon dioxide, and other compounds and produce methane as a by-product. Other microbes in the mix produce smaller amounts of nitrous oxide. A pair of Holstein cows poke their heads through the rails of a free-stall barn at the Alberto Dairy.JOE PROUDMAN/UC DAVIS Both are particularly potent greenhouse gases, with as much as 30 to nearly 275 times the warming power of carbon dioxide, respectively, over a century. The slurry is often spread onto fields to add nutrients to the soil. When it’s done excessively or improperly, this part of the practice can pollute soil or groundwater with drug residues, pathogens like salmonella and E. coli, and nitrates. Nitrates that leach into drinking water have been linked to a variety of human health risks. And those that flow into rivers, lakes, and coastal waters can spawn algae blooms that poison fish, block sunlight, suck up oxygen, or form large coastal dead zones devoid of marine life. Policy drivers A number of regions, nations, and states have passed regulations or offered subsidies designed to limit the pollution from livestock manure, but so far, most of the major initiatives have focused on water contamination rather than greenhouse-gas emissions. The European Union, for instance, restricts the amount of manure that farmers can apply to fields and requires member nations to monitor nitrate levels in ground and surface water. The US’s Clean Water Act requires large livestock operations to obtain permits and develop manure management plans that limit pollution.  But California has arguably done the most to use government policy specifically to drive down the methane emissions from livestock. The dairy industry accounts for about 45% of the state’s pollution from the potent greenhouse gas, and more than half of that comes from manure, according to the government’s estimates.  In 2016, the state enacted a law that requires dairies, landfills, and other businesses to cut methane emissions 40% below 2013 levels by 2030, as part of a broader effort to reduce pollution from powerful but short-lived greenhouse gases. The measure directed the California Air Resources Board, the state’s main climate regulatory agency, to set up various incentive programs to encourage these industries to shift to cleaner practices.  “In terms of bang for your buck, short-term benefits, methane can go a long way toward reaching climate goals,” says Tawny Mata, director of California’s Office of Agricultural Resilience and Sustainability.  Between these various programs—and falling livestock numbers in the state—the dairy sector is on track to reduce annual methane emissions by the equivalent of 5 million metric tons of carbon dioxide by 2030, the state estimates. That would still fall about 4 million tons short of the target under the 2016 law. The downsides of dairy digesters Excluding the decline in herd populations—which has been driven by growing international competition and rising costs—the vast majority of California’s estimated methane reductions come from the use of what are known as anaerobic digesters. This technology entails covering the slurry lagoons to prevent methane from leaking into the air and then piping the biogas into separate vessels, where it’s cleaned and converted into natural gas.  Under California’s Low Carbon Fuel Standard program, dairies that use digesters to produce