YouZum

Nachrichten

AI, Committee, Nachrichten, Uncategorized

Innovating with AI in the special chemicals industry

As companies look to accelerate innovation with artificial intelligence, cloud computing, and advanced modeling, the primary challenge is connecting new technologies to reliable data, scientific expertise, and the practical realities of bringing products to market. For specialty chemicals manufacturer Lubrizol, digital transformation is all about giving researchers the tools to test ideas faster, make better-informed decisions, and develop more sustainable materials without compromising performance. Girish Rao, vice president of information technology at Lubrizol, says the company is building a digital foundation that connects business processes, modern data platforms, and cloud infrastructure to support its research and development (R&D) teams. Their goal is to combine decades of accumulated technical data with the deep expertise of chemists and engineers who understand customer needs as well as the physical constraints of their work. Data scientists can help identify patterns, test new hypotheses, and model possibilities before researchers conduct physical experiments. But, Rao emphasizes, “it’s not about replacing scientific judgment; it is about giving our scientists better tools.” Scientists, data scientists, and IT teams approach problems from different perspectives, prioritizing experimental validity, model performance, and platform security and scalability, respectively. By bringing these groups together in “fusion teams,” Lubrizol aligns them around shared objectives, starting with small pilots to build trust. “Once the trust is built between the team members and they know what value they’re about to create,” Rao says, “things go a lot better.” Cloud infrastructure also gives the company flexibility to access computing resources and modern tools without making the substantial investments required to build and maintain its own data centers. These capabilities could prove especially valuable in advancing sustainability. Digital modeling can help researchers evaluate more formulations, explore materials such as starch and corn-based alternatives, and assess trade-offs across a product’s life cycle. Promising a material in a laboratory is only the first step. Scaling production while meeting performance expectations, regulatory requirements, and financial constraints is still a tightrope walk, but AI’s ability to model multiple variables could help teams navigate those competing demands. Looking to the future, Rao expects AI, automation, advanced modeling, digital twins, and connected data to converge as laboratories become more intelligent and manufacturing plants more efficient. The opportunity, he suggests, lies in bringing these capabilities together responsibly and at scale. “It’s not just a single technology. I think it’s the combination of all of these, and it’ll be the ability to connect all these processes that are going to generate tremendous amounts of data.” This webcast is produced in partnership with Infosys. This content was produced by Insights, MIT Technology Review’s custom content arm, not its editorial staff. It was researched and written by humans, with any AI tools that may have been used limited to production processes under human oversight.

Innovating with AI in the special chemicals industry Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Nancy Pelosi’s replacement is ready to regulate AI

As a California state senator, Scott Wiener authored legislation aimed at curbing the risks of advanced AI. Now, after winning a tight congressional race for former Speaker of the House Nancy Pelosi’s open San Francisco seat, he could play a key role in setting federal AI policy.  I spoke with Wiener on Friday, only a few hours after he had been declared the victor. But he was already thinking about AI risk. The previous day, the AI nonprofit METR had released a report that had been widely interpreted as evidence that Anthropic had achieved recursive self-improvement, a phenomenon in which AI meaningfully contributes to its own advancement and AI progress potentially spins out of control. “It’s very concerning,” Wiener said, citing the report as evidence that we are quickly running out of time to regulate AI. “It was my hope that Congress would take decisive action before [the end of] this year,” he said. “That does not appear to be happening.” Wiener has been increasingly focused on AI in recent years, and he’s made many of the AI safety movement’s concerns his own: During our 20-minute conversation, he used the phrase “recursive self-improvement” half a dozen times, for example.“We need to slow this down,” he says. “Because it’s moving very quickly, and we could sleepwalk into a disaster.” Wiener has already had real influence on AI policy. SB 1047, a bill that he authored in 2024, was set to become the first real piece of frontier AI legislation in the country before it was vetoed by California governor Gavin Newsom; he then sponsored SB 53, a lighter-touch bill that Newsom signed. Both bills have inspired proposed federal legislation.  Wiener’s congressional campaign received over $3 million from Public First Action, a nonprofit in favor of AI regulation that receives substantial funding from Anthropic. That contribution led opponents to criticize him for being too close to the AI industry, but Wiener says that couldn’t be further from the truth.  “Public First supported me because I was the first lawmaker in the country to introduce AI regulation and the first to pass it and get it implemented,” he wrote in a message after our interview. “The attacks on me for [Public First’s] support were shockingly cynical—literally attacking a candidate who actually regulated AI over Big Tech’s objection and falsely claiming the candidate is somehow too close to Big Tech.” Wiener says that if Congress forms a select committee on artificial intelligence—something that Democrats have discussed since September—he’d be eager to serve. And he’s enthusiastic about many of the policy ideas floating around Congress. “I think it’s really important to have a strong, empowered, nimble federal regulator,” he says, noting that Representative Ro Khanna, a fellow Democrat from California, has proposed legislation to establish such a regulator.  He also highlights the importance of whistleblower protections, testing and auditing requirements, and a kill switch requirement, and he wants to see Congress leverage its oversight powers to hold the AI labs to account. “This is about the future of humanity, and we want to make sure that the public has a say and that the public knows what’s going on,” he says. And as a longtime state legislator, he is opposed to broad federal preemption of state AI laws, which some of the proposed federal legislation would require. “States are, as Brandeis says, laboratories,” he notes. “You want the states to be able to continue to be nimble in ways that Congress sometimes is not.” Above all, Wiener says, he wants Congress to pass policies with real teeth. “It’s critically important that when Congress acts, that we do so in a way that is truly meaningful and that meets the moment,” he says. That means taking seriously the possibility of AI catastrophe, something he discusses in the sort of language typically used by AI safety experts. “We’re in a moment that is very dangerous in terms of the risk of loss of control over AI,” he told me. “We need to deal with recursive self-improvement. Once you’re in that realm, that really is potentially a recipe for total loss of control.” He’ll have some allies in the fight against catastrophic AI risk. Senator Bernie Sanders, for instance, has met with many of the AI safety movement’s most influential figures and has spoken repeatedly about AI and existential risk. But there’s still a large gap between Wiener’s Bay Area, AGI-pilled perspective and the national mood on AI, which is anchored by opposition to data centers. The fact that voters approved 40 out of 41 anti-data-center ballot measures on Tuesday might seem to be a good sign for Wiener and others who support AI regulation, but Wiener says he sees this movement as far more tied to local concerns than to bigger questions about AI risk. “Peoople who are really opposed to where AI is going—this is an area where they can actually do something about it,” he says. “But the data center revolt—to me, I think it’s more so that people are upset that there could be something like this in their community, with all the impacts near people’s homes, driving up electric bills and using water. I think that’s the bigger issue for most people, from what I can tell.” Wiener describes himself as only “cautiously optimistic” about Congress’s ability to rein in AI, given its poor track record on regulating the tech industry. He also has concerns about the ability to get legislation past President Donald Trump, who called concerns about catastrophic AI risk a “hoax” in September and has demonstrated a stark unwillingness to collaborate with China, something that Wiener sees as essential to preventing the worst consequences of AI. But he says Trump’s unpredictability provides  some measure of hope, though it’s a fraught kind of hope. “So far what we’ve seen are some problematic things, but not complete disasters,” Wiener says. “I hope that it doesn’t take a disaster to spur the president into acting. And I hope if that’s what it

Nancy Pelosi’s replacement is ready to regulate AI Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

The Download: empty AI climate claims, and an Nvidia investigation

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Opinion: The climate case for AI has not been made  —By Mary Nichols, distinguished counsel at the UCLA School of Law and the former chair of the California Air Resources Board, and Jeremy Tamanini, the founder of Dual Citizen, a sustainability research and advisory firm This week, as countries meet in Antalya, Turkey for the United Nations climate conference, one topic is likely to dominate the conversation: artificial intelligence. The host has already called on countries to commit to ensuring that AI is designed, powered, and deployed in ways that support climate action. But so far, industries around the world aren’t doing nearly enough to harness the technology to actually drive down the greenhouse-gas emissions heating up the planet. AI companies like to make very optimistic claims about AI’s potential to help address climate change. The reality is far less promising. Read the full story. Meet the ecologist revealing the invisible networks that revive rainforests   Tropical forests are our planet’s biggest terrestrial carbon sink, accounting for around 40% of carbon stored on land. Forest ecologist Daisy Dent, a senior scientist at ETH Zurich, is part of an international network of researchers working to decipher the networks of animal and plant interactions that enable some forests to recover after destruction.  The goal is a low-cost, high-impact alternative to typical carbon-­crediting schemes, which plant huge areas with a single species of tree.  For over a decade, her research has helped show that these large “monocrop” programs could be expensive mistakes—and that more resilient forests can rebound naturally in just a few decades, re-creating ecosystems that suck up vast quantities of carbon dioxide.  Learn more about Dent, her work, and her impact on how we understand the role rainforests play in regulating our climate.  —Matthew Ponsford The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 Nvidia is investigating its own bossJensen Huang has admitted to intimacy with a suspected Chinese spy. (NYT $)+ Shares in Nvidia were down 5% in pre-market trading. (CNBC)+ Huang’s children are also employed by Nvidia. (WSJ $) 2 Democrats are planning a blitz of investigations into Big TechReportedly starting with Apple, Meta, and Musk’s companies. (WP $)+ They’re hoping to harness the data center backlash to rein in the AI sector. (Vox $)+ Here’s why the midterms sparked an AI stock selloff. (BBC) 3 The UN annual climate meeting starts todayIt’s the first to be held after confirmation the planet will exceed 1.5 °C warming. (Nature)+ How to study a swiftly melting climate disaster. (MIT Technology Review) 4 Connecting fridges to the internet looks like a bad move nowWe still don’t know who is behind recent hacks, or even if they’re connected. (The Atlantic $)+ Experts say they should be a “wake up call” for the industry. (NPR)+ Are we in the foothills of a hacking epidemic? (The Guardian)+ AI is already making online crimes easier. It could get much worse. (MIT Technology Review) 5 Tesla is axing the Cybertruck RIP, we hardly knew ye. (Wired $)+ The Cybertruck was on our “worst tech flops of 2025” list. (MIT Technology Review) 6 Iran might attack subsea cables if the US escalates againThat’s the message it’s sending via several loitering tankers. (FT $) 7 Chinese AI labs are struggling to poach Silicon Valley workersStartups like MiniMax and Moonshot have an uphill battle as they try to fill new US offices. (Rest of World)+ China’s AI models have Trump’s AI world at war with itself. (MIT Technology Review) 8 Online piracy is back But this time, archivist zeal is at least as big a draw as accessing content for free. (The Verge) + Inside the race to save our online lives from a digital dark age. (MIT Technology Review) 9 How do you land on a moon with almost no gravity?It’s a huge challenge to land on Martian moon Phobos—here’s how Japan’s going to attempt it. (IEEE Spectrum) 10 Meet Philadelphia’s delivery robot saboteurCarolyn Marvin is going to court for blocking them, and it’s turning her into a local hero. (The Guardian) Quote of the day “Leading NVIDIA is the privilege of my life. My responsibility is to serve you and be worthy of your trust. I have fallen short of that responsibility.”  —Jensen Huang apologizes to Nvidia employees for his affair with a suspected Chinese spy in a mass email sent yesterday, the Washington Post reports. One more thing STUART BRADFORD AI is coming for music, too AI models are marching into a creative field that is uniquely vulnerable to disruption: music.   AI-generated creative works—from orchestra performances to heavy metal—are poised to suffuse our lives more thoroughly than any other product of AI has done yet.  The songs are likely to blend into our streaming platforms, party and wedding playlists, soundtracks, and more, whether or not we notice who (or what) made them.  Here’s why AI is set to forever change how we make music.  —James O’Donnell We can still have nice things A place for comfort, fun, and distraction to brighten up your day. (Got any ideas? Drop me a line.) + French bardcore band Corseval has given ABBA a medieval makeover.+ A stunning collection of prehistoric engravings has been found in a Spanish cave.+ This browser game flips a retro classic on its head: you control the asteroids, not the spaceship.+ From a caiman in a wig to a leopard doing a handstand, the Comedy Wildlife Awards shortlist is a delight.

The Download: empty AI climate claims, and an Nvidia investigation Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

The backlash about data centers might really just be about electricity

This story originally appeared in The Algorithm, our weekly newsletter on AI. To get stories like this in your inbox first, sign up here. As voters went to the polls across the US last week, I was in Ashburn, Virginia, part of what’s known as Data Center Alley—the world’s largest data center hub. It’s also where the backlash now sweeping the country arguably first began taking shape. Driving through it is surreal. Concrete monolithic buildings loom along every major road, with names like Skybox and CloudHQ. It’s accepted wisdom in Big Tech that the war against data centers must be about something bigger than just data centers themselves. Two days after the midterms, OpenAI’s chief global affairs officer, Chris Lehane, wrote that data centers “have become a proxy for public unease about affordability, corporate power, and people losing control over their futures.” It’s a culture war, in other words. But spending a few days in Data Center Alley left me wondering if it’s really the proxy that Lehane describes. I came away convinced that the data center backlash is more about electricity than Big Tech admits. The industry can improve its image, but it can’t escape the physical consequences of its enormous appetite for electricity. Here’s why. There are two ways to power the sort of data center that AI companies want: connecting to the grid, or generating electricity onsite. In Virginia’s Loudoun County, where data centers are ubiquitous but have been built at a steadier pace than the breakneck expansion now underway elsewhere in the country, most are connected to the existing grid.  One facility I went inside, operated by Equinix, has massive diesel generators onsite for backup but gets its power from the local utility, Dominion Energy. Much of Dominion’s electricity is generated by natural gas (which is cleaner than coal or oil, but worse than renewables). The average data center operating in this part of Virginia today has a power capacity of 54 megawatts. That’s more than the power draw of the Denver International Airport.  Elena Schlossberg, a prominent activist against data centers in Virginia, told me she’s “always banging my head against the wall” when the industry paints the fight as a culture war. “It was always about power,” she says—that is, electricity. Some Virginia protesters appear more animated by a broader anti-tech populism than Schlossberg, to be fair. But it’s the particular consequences of drawing so much power from the grid that have done the most to shape how the backlash has played out.  Recent protests have led to new rules from the state that data centers must pay fully for the cost of new transmission lines to the grid and make sure they’re not spread among everyone else. Data center developers told me they’re preparing for new rules that would cut them off from grid power during peak demand. Schlossberg also worries about the downstream effects of a strained grid—for example, that data centers will run more diesel generators or that Dominion Energy will expand fossil fuel inputs rather than renewables to meet demand. The overwhelming majority of AI data centers being built elsewhere in the country are not designed this way. The scale desired is so massive that companies are setting up in rural areas where there is virtually no grid infrastructure to connect to. Building there means bringing your own power (so-called “behind the meter” power), in the form of natural-gas-powered turbines. Companies also build diesel generators for backup. Anthropic, for example, has reportedly agreed to pay $3.4 billion for gas turbines in what’s described as the largest power generation purchase by an AI company.  Even if grids were nearby, they’d be unlikely to serve the amount of power demanded. The smallest data centers attractive for AI training are 300 megawatts. It’s a lot of power for one industrial site, but not unprecedented: One data center developer told me he’s eyeing a former steel mill already equipped to draw that much electricity from the grid. But the ideal facilities are more like 6,000 megawatts. That’s the estimated draw of the city of Chicago, which is powered by more than 20 power plants within the city alone (and dozens farther out). The notion of data center companies’ bringing their own power might seem attractive at first, because they wouldn’t be straining the grid. But generating electricity at this scale inevitably affects surrounding communities, and when public utilities build infrastructure of this magnitude, communities have traditionally expected a say in where it goes. Now, private companies are undertaking projects on a similar scale. Natural-gas turbines of any scale release air pollutants. When diesel generators are used, it’s even worse. And in the unprecedented rush to get these facilities operating, companies have become horrible neighbors (recall that Elon Musk violated air quality laws when he set up dozens of turbines to power a data center in Memphis, according to the Environmental Protection Agency). Big Tech’s tendency to chalk the backlash up to a culture war obfuscates one simple fact: There’s only so pleasant you can make a small city of computers to live around. Sure, companies can pursue better deals that raise tax revenues, hire local workers, and get more community involvement, as the protests are leading them to do. But if a small town is asked to welcome a Walt Disney World, an international airport, a steel plant, and the power plants to run them all, should we be surprised by its trepidation? Again, that’s not to say this is the sole reason for opposition to data centers. Fears of job losses, existential risks, and concentrated corporate power all fuel the backlash, as does misinformation (most facilities use far less water than many people think). Nor is consuming enormous amounts of electricity inherently bad: Many of the advances that have improved modern life have required just that. But as one developer recently told me, industry insiders see their ambitions as constrained less by public opposition than by a lack of electricity. That should tell

The backlash about data centers might really just be about electricity Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Attention Variants and Memory Efficiency

Overview This post is divided into eight parts; they are: • The Checkpoint Fixes the Attention Architecture • A Closer Look at Attention Memory • Multi-Head Attention • Multi-Query and Grouped-Query Attention • Sliding-Window and Mixed Attention • Multi-Head Latent Attention • What the Inference Runtime Can Change • From Cache Size to Serving Capacity The Checkpoint Fixes the Attention Architecture The checkpoint fixes the model configuration; the inference engine does not change it.

Attention Variants and Memory Efficiency Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Floating-Point Precision and Its Effects to Model Inference

Overview This post is divided into five parts; they are: • How Floating-Point Numbers Work • Common Precision Formats • Loading a Model at a Different Precision • Mixed Precision and Sensitive Operations • Measuring the Result How Floating-Point Numbers Work A floating-point number contains a sign, an exponent, and a significand (often called the mantissa).

Floating-Point Precision and Its Effects to Model Inference Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

I watched a humanoid robot clean my apartment. It felt like a taste of the future.

I’m not the sort of person who typically pays for a house cleaner. I’m too frugal, and I kind of enjoy putting on a podcast and going to town on my tub with a Magic Eraser. But when I saw an X post about a new humanoid-robot cleaning service in San Francisco, I immediately signed up for the waitlist. I wanted my apartment to feel like The Jetsons, even if just for an hour. I was also motivated by journalistic interest. In order to ensure an authentic experience, I didn’t tell Tau Robotics, the startup behind the X post, that I was a journalist—though I did use my real name, so it’s possible someone looked me up. Tau’s long-term goal is to make autonomous cleaning robots a reality, but they’re not quite there yet: For now, the robots are piloted remotely by humans, who watch what’s going on via cameras mounted in the robots’ heads and wrists. As the robots clean, they collect data that Tau will use to train up their more independent descendants. Every counter successfully wiped and T-shirt successfully folded will help the models that power the robots learn to do better on the next go-round. Because this data is so valuable to Tau, the company charges less for cleaning than one might expect for such a cutting-edge product. At the time I booked, Tau was offering $30 for an hourlong cleaning with a single robot; now the charge is $60 per weekly cleaning with two robots, or $100 for a one-time visit. But even at the cheaper rate of $30 per hour, Tau’s robots aren’t a great bargain. They are thorough but slow, so if you, like me, are tempted to stand around watching them for their entire visit, it would probably be a better use of your time to just take care of the cleaning yourself.  But the robots are physically capable of cleaning, and they’re undeniably cute. Tau has armed each four-and-a-half-foot-tall robot with a 3D-printed “head” with antennae that allow it to communicate with Tau’s headquarters, as well as two 3D-printed claw “hands” (the more humanlike alternatives apparently aren’t durable enough, and claws do surprisingly well for most cleaning tasks). Mine was wearing a Tau Robotics shirt, and grippy socks so that it didn’t scratch hardwood floors. With all the human support they need, the robots are closer to radio-controlled cars than Waymos. Many tasks, such as making the bed and folding laundry, require what’s called kinematic guidance: The human pilot performs the necessary motions, and sensors transmit those movements to the robot, which copies them. These kinematically guided tasks produce especially valuable training data. Tau’s AI models might not have solved laundry folding yet, but learning from the examples of robots that have completed the task in the real world is how they’ll get there. But according to Tau cofounder Cornelia Weinzierl, to whom I came clean as a journalist a few weeks after the cleaning, the robot can already complete some other tasks—such as vacuuming and mopping—without any human help. If Tau succeeds in using all the data it is collecting to train truly autonomous cleaning robots, they could even become San Francisco’s newest tech tourist attraction as Waymos become more commonplace countrywide. The cleaning robot that came to my building last month was already a bit of a spectacle. It arrived in a blue minivan, out of which it was helped by Weinzierl and another Tau employee. That’s when the robot encountered its first obstacle: the eight stairs leading up to the building’s front door. I’d come down to greet Weinzierl and the robot, so I had a front-row seat to its struggles with my steps. It climbed as if it were drunk. Each successful step was as likely as not to be followed by a stumbling backward stagger that erased all its progress—though, impressively, it never entirely lost its footing. When I later discussed my experience with Weinzierl, she explained that the robot sometimes walks strangely because it is operated remotely from Tau’s headquarters, and the short signal transmission delay can affect its coordination. “The amount of work and time that we spend just optimizing, for example, timings and connectivity is crazy,” she said. Inside, the carpeted staircase leading up to my apartment defeated the robot entirely, and Weinzierl had to lug it up the last few steps. This, it turned out, was not typical. As the robot cleaned, it repeatedly stalled out, something Weinzierl attributed to it overheating in the unseasonably hot weather—until she heard from one of her colleagues that the heat had shut down not the robot but the equipment that its human controller was using to watch its progress through my apartment. Without remote assistance, the robot couldn’t clean, and so it, and Weinzierl, departed before my booked hour was up. (Back in September, Weinzierl or another Tau representative remained with the robot throughout its visit, but she says that’s no longer necessary.) You can’t fault Tau on customer service, though. Weinzierl quickly emailed me to apologize, comp the visit, and offer me another on the house (minus whatever price I assign to my data privacy)—and that was all before I told her I was a journalist. I scheduled the next cleaning for about a month later, after a couple of trips out of town, and it went much more smoothly—the robot got up my stairs without much assistance, though still with the drunken stagger. In the hour it was there, it did a creditable job of cleaning my kitchen (though it didn’t brave the stove) and even saved me my most dreaded cleaning task, the toilet. That was its most impressive performance: Three times the head fell off its toilet cleaning wand, and three times it managed to reattach it by poking around with the handle until it made contact at just the right angle. The robot is still rough around the edges, which is to be expected for a product in beta.

I watched a humanoid robot clean my apartment. It felt like a taste of the future. Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning

arXiv:2609.30906v2 Announce Type: replace Abstract: Large language models (LLMs) excel at natural language processing but struggle to interact with external environments. Tool learning provides a promising way to extend LLMs into actionable agents, where tool selection is a critical prerequisite for successful tool use. Existing work often assumes a small or predefined set of tools, leaving large-scale tool selection underexplored. Real-world repositories contain a vast and diverse array of tools, making it difficult for LLMs to effectively search, distinguish, and compose tools under context-length constraints. We identify large-scale tool selection as a new challenge for agentic reinforcement learning, highlighting that existing RL methods for knowledge-based question answering are inadequate for selecting tools while considering compatibility. To address this challenge, we propose ToolSearcher, a novel RL framework for effective multi-turn search and fine-grained optimization in large-scale tool selection. Specifically, we introduce category-constrained tool discrimination to improve the model’s ability to distinguish functionally similar tools, event-level search modeling to explicitly optimize the discovery of target tools during multi-turn search, and trajectory-aligned credit allocation to provide fine-grained reward signals for different stages of the search-selection process. Extensive experiments on large-scale tool selection benchmarks demonstrate that ToolSearcher consistently outperforms a set of strong baselines in challenging settings involving iterative search and complex tool composition.

ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning Beitrag lesen »

AI, Committee, Nachrichten, Uncategorized

Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding

arXiv:2611.07845v1 Announce Type: cross Abstract: Masked diffusion models generate text by reversing a stochastic masking process. Unlike autoregressive LLMs, which generate text one token at a time, they can reveal many tokens in parallel, at the price of an error, since tokens revealed together are drawn independently. We show that this trade-off maps onto the statistical physics of driven many-particle systems. (i) For any data, the error of parallel generation is the entropy production of the generation process relative to the reversed masking process. (ii) For Markov data, generation is a random sequential adsorption process with labelled gaps, which we solve exactly for any number of steps. (iii) For many steps, the error becomes the dissipation of a slowly driven system: the unmasking schedule plays the role of the driving protocol, the number of steps that of its duration, and the error is set by a thermodynamic metric that we obtain in closed form from the mutual information between tokens. Optimal schedules are geodesics of this metric. The exact solution shows how correlations in the text shape both the error and the quality of the output. The longer the correlations, the more the error concentrates at the start of generation. Schedules that start with small steps then gain more, both in path error and in perplexity, and need fewer steps to reach their asymptotic behaviour.

Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding Beitrag lesen »

We use cookies to improve your experience and performance on our website. You can learn more at Datenschutzrichtlinie and manage your privacy settings by clicking Settings.

Privacy Preferences

You can choose your cookie settings by turning on/off each type of cookie as you wish, except for essential cookies.

Allow All
Manage Consent Preferences
  • Always Active

Save
de_DE