Uncategorized

If you have ever stared at thousands of lines of integration test logs wondering which of the sixteen log files actually contains your bug, you are not alone — and Google now has data to prove it. A team of Google researchers introduced Auto-Diagnose, an LLM-powered tool that automatically reads the failure logs from a broken integration test, finds the root cause, and posts a concise diagnosis directly into the code review where the failure showed up. On a manual evaluation of 71 real-world failures spanning 39 distinct teams, the tool correctly identified the root cause 90.14% of the time. It has run on 52,635 distinct failing tests across 224,782 executions on 91,130 code changes authored by 22,962 distinct developers, with a ‘Not helpful’ rate of just 5.8% on the feedback received. https://arxiv.org/pdf/2604.12108 The problem: integration tests are a debugging tax Integration tests verify that multiple components of a distributed system actually communicate to each other correctly. The tests Auto-Diagnose targets are hermetic functional integration tests: tests where an entire system under test (SUT) — typically a graph of communicating servers — is brought up inside an isolated environment by a test driver, and exercised against business logic. A separate Google survey of 239 respondents found that 78% of integration tests at Google are functional, which is what motivated the scope. Diagnosing integration test failures showed up as one of the top five complaints in EngSat, a Google-wide survey of 6,059 developers. A follow-up survey of 116 developers found that 38.4% of integration test failures take more than an hour to diagnose, and 8.9% take more than a day — versus 2.7% and 0% for unit tests. The root cause is structural. Test driver logs usually surface only a generic symptom (a timeout, an assertion). The actual error lives somewhere inside one of the SUT component logs, often buried under recoverable warnings and ERROR-level lines that are not actually the cause. https://arxiv.org/pdf/2604.12108 How Auto-Diagnose works When an integration test fails, a pub/sub event triggers Auto-Diagnose. The system collects all test driver and SUT component logs at level INFO and above — across data centers, processes, and threads — then joins and sorts them by timestamp into a single log stream. That stream is dropped into a prompt template along with component metadata. The model is Gemini 2.5 Flash, called with temperature = 0.1 (for near-deterministic, debuggable outputs) and topp = 0.8. Gemini was not fine-tuned on Google’s integration test data; this is pure prompt engineering on a general-purpose model. The prompt itself is the most instructive part of this research. It walks the model through an explicit step-by-step protocol: scan log sections, read component context, locate the failure, summarize errors, and only then attempt a conclusion. Critically, it includes hard negative constraints — for example: if the logs do not contain lines from the component that failed, do not draw any conclusion. The model’s response is post-processed into a markdown finding with ==Conclusion==, ==Investigation Steps==, and ==Most Relevant Log Lines== sections, then posted as a comment in Critique, Google’s internal code review system. Each cited log line is rendered as a clickable link. Numbers from production Auto-Diagnose averages 110,617 input tokens and 5,962 output tokens per execution, and posts findings with a p50 latency of 56 seconds and p90 of 346 seconds — fast enough that developers see the diagnosis before they have switched contexts. Critique exposes three feedback buttons on a finding: Please fix (used by reviewers), Helpful, and Not helpful (both used by authors). Across 517 total feedback reports from 437 distinct developers, 436 (84.3%) were “Please fix” from 370 reviewers — by far the dominant interaction, and a sign that reviewers are actively asking authors to act on the diagnoses. Among dev-side feedback, the helpfulness ratio (H / (H + N)) is 62.96%, and the “Not helpful” rate (N / (PF + H + N)) is 5.8% — well under Google’s 10% threshold for keeping a tool live. Across 370 tools that post findings to Critique, Auto-Diagnose ranks #14 in helpfulness, putting it in the top 3.78%. The manual evaluation also surfaced a useful side effect. Of the seven cases where Auto-Diagnose failed, four were because test driver logs were not properly saved on crash, and three were because SUT component logs were not saved when the component crashed — both real infrastructure bugs, reported back to the relevant teams. In production, around 20 ‘more information is needed‘ diagnoses have similarly helped surface infrastructure issues. Key Takeaways Auto-Diagnose hit 90.14% root-cause accuracy on a manual evaluation of 71 real-world integration test failures spanning 39 teams at Google, addressing a problem 6,059 developers ranked among their top five complaints in the EngSat survey. The system runs on Gemini 2.5 Flash with no fine-tuning — just prompt engineering. A pub/sub trigger collects logs across data centers and processes, joins them by timestamp, and sends them to the model at temperature 0.1 and topp 0.8. The prompt is engineered to refuse rather than guess. Hard negative constraints force the model to respond with “more information is needed” when evidence is missing — a deliberate trade-off that prevents hallucinated root causes and even helped surface real infrastructure bugs in Google’s logging pipeline. In production since May 2025, Auto-Diagnose has run on 52,635 distinct failing tests across 224,782 executions on 91,130 code changes from 22,962 developers, posting findings in a p50 of 56 seconds — fast enough that engineers see the diagnosis before switching contexts. Check out the Pre-Print Paper here. Also, feel free to follow us on Twitter and don’t forget to join our 130k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Google AI Releases Auto-Diagnose: An Large Language Model LLM-Based System to Diagnose Integration Test Failures at Scale appeared first on MarkTechPost.

The handsome new book Maintenance: Of Everything, Part One, by the tech industry legend Stewart Brand, promises to be the first in a series offering “a comprehensive overview of the civilizational importance of maintenance.” One of Brand’s several biographers described him as a mainstay of both counterculture and cyberculture, and with Maintenance, Brand wants us to understand that the upkeep and repair of tools and systems has profound impact on daily life. As he puts it, “Taking responsibility for maintaining something—whether a motorcycle, a monument, or our planet—can be a radical act.” Radical how? This volume doesn’t say. In an outline for the overall work, Brand says his goal is to “end with the nature of maintainers and the honor owed them.” The idea that maintainers are owed anything, much less honor, might surprise some readers. Actually, maintenance and repair have been hot topics in academia since the mid-2010s. I played some role in that movement as a cofounder of the Maintainers, a global, interdisciplinary network dedicated to the study of maintenance, repair, care, and all the work that goes into keeping the world going. Brand is right, too, that maintainers haven’t gotten the laurels they deserve. Over the past few decades, scholars have shown that work from oiling tools to replacing worn parts to updating code bases all tends to be lower in status than “innovation.” Maintenance gets neglected in many organizational and social settings. (Just look at some American infrastructure!) And as the right-to-­repair movement has shown, companies in pursuit of greater profits have frequently locked us out of being able to do repairs or greatly reduced the maintainable life of their products. It’s hard to think of any other reason to put a computer in the door of a refrigerator. Some of Brand’s earlier work helped inspire those insights. But his new book makes me think he doesn’t see things that way. For Brand, maintenance seems to be a solitary act, profound but more about personal success and fulfillment than tending to a shared world or making it better. Born in 1938, Brand is 87 years old. A sense hangs over the book—with its battles against corrosion, rust, and decay, with its attempts to keep things going even as they inevitably falter—of someone looking over life and pondering its end. Maintenance: Of Everything connects to every stage of Brand’s life. It’s worth reviewing where it falls in that arc. Brand has always been interested in tools and fixing things, but rarely has he focused on the systems that need the most care.  More than a half-century ago, Brand was a member of the Merry Pranksters, a countercultural, LSD-centered hippie collective famously led by Ken Kesey, the author of One Flew Over the Cuckoo’s Nest. In 1966, Brand co-produced the Trips Festival, where bands like the Grateful Dead and Big Brother and the Holding Company performed for thousands amid psychedelic light shows. Brand’s Whole Earth Catalog had a vision that might feel progressive, but its libertarian, rugged-individualist philosophy of remaking civilization alone stood in contrast to more collective social change movements. In some ways, the Trips Festival set a paradigm for the rest of his life’s work. Brand’s biographers have described him as a network celebrity—someone who got ahead by bringing people together, building coalitions of influential figures who could boost his signal. As Kesey put it in 1980, “Stewart recognizes power. And cleaves to it.”  Brand applied this network logic to the undertaking he will always be best remembered for: the Whole Earth Catalog. First published in 1968 and aimed at hippies and members of the nascent back-to-the-land movement, the publication had the motto “Access to tools.” Its pages were full of Quonset huts, geodesic domes, solar panels, well pumps, water filters, and other technologies for life off the grid. It was a vision that might feel progressive or left-leaning, but the libertarian, rugged-individualist philosophy of eschewing corrupt systems and remaking civilization alone stood in contrast to the more collective movements pushing for deep social change at the time—like civil rights, feminism, and environmentalism. That vision also led straight to the empowerment that came with new digital tools, and to Silicon Valley. In 1985, Brand published the Whole Earth Software Catalog, the last of the series, and also cofounded the WELL—the Whole Earth ’Lectronic Link, a pioneering online community famous for, among other things, facilitating the trade of Grateful Dead bootlegs. He also wrote a hagiographic book about the MIT Media Lab, known for its corporate-sponsored research into new communications tech. “The Lab would cure the pathologies of technology not with economics or politics but with technology,” Brand wrote. Again, not collective action, not policymaking: tools. And Brand then cofounded the Global Business Network, a group of pricey consulting futurists that further connected him to MIT, Stanford, and the Valley. Brand had literally helped bring about the modern digital revolution. His attention then turned toward its upkeep. Brand’s 1994 book, How Buildings Learn: What Happens After They’re Built, argued against high-modernist architectural ideas. Nearly all buildings eventually get remade, he argued, but he especially favored cheap, simple structures that inhabitants could easily retool to suit changing needs. In some ways, Brand was recapitulating the liberated—or libertarian—philosophy of the Whole Earth Catalog: People can remake their world, if they have access to tools. In a chapter titled “The Romance of Maintenance,” he asked readers to see the beauty, value, and occasional pleasures of fixer-uppers of all kinds. This chapter was a touchstone for many of us in the academic subfield of maintenance studies. Researchers in disciplines like history, sociology, and anthropology, as well as artists and practitioners in fields like libraries, IT, and engineering, all started trying to understand the realities and, yes, romance of maintenance and repair. Brand joined and contributed to Listservs, attended conferences, chatted with intellectual leaders. So it’s a bit uncharitable when he writes that his new book is “the first to look at maintenance in general.” He knows better. The real question,

Roboticists used to dream big but build small. They’d hope to match or exceed the extraordinary complexity of the human body, and then they’d spend their career refining robotic arms for auto plants. Aim for C-3P0; end up with the Roomba.  The real ambition for many of these researchers was the robot of science fiction—one that could move through the world, adapt to different environments, and interact safely and helpfully with people. For the socially minded, such a machine could help those with mobility issues, ease loneliness, or do work too dangerous for humans. For the more financially inclined, it would mean a bottomless source of wage-free labor. Either way, a long history of failure left most of Silicon Valley hesitant to bet on helpful robots. That has changed. The machines are yet unbuilt, but the money is flowing: Companies and investors put $6.1 billion into humanoid robots in 2025 alone, four times what was invested in 2024.  What happened? A revolution in how machines have learned to interact with the world.  Imagine you’d like a pair of robot arms installed in your home purely to do one thing: fold clothes. How would it learn to do that? You could start by writing rules. Check the fabric to figure out how much deformation it can tolerate before tearing. Identify a shirt’s collar. Move the gripper to the left sleeve, lift it, and fold it inward by exactly this distance. Repeat for the right sleeve. If the shirt is rotated, turn the plan accordingly. If the sleeve is twisted, correct it. Very quickly the number of rules explodes, but a complete accounting of them could produce reliable results. This was the original craft of robotics: anticipating every possibility and encoding it in advance. Around 2015, the cutting edge started to do things differently: Build a digital simulation of the robotic arms and the clothes, and give the program a reward signal every time it folds successfully and a ding every time it fails. This way, it gets better by trying all sorts of techniques through trial and error, with millions of iterations—the same way AI got good at playing games. The arrival of ChatGPT in 2022 catalyzed the current boom. Trained on vast amounts of text, large language models work not through trial and error but by learning to predict what word should come next in a sentence. Similar models adapted to robotics were soon able to absorb pictures, sensor readings, and the position of a robot’s joints and predict the next action the machine should take, issuing dozens of motor commands every second. This conceptual shift—to reliance on AI models that ingest large amounts of data—seems to work whether that helpful robot is supposed to talk to people, move through an environment, or even do complicated tasks. And it was paired with other ideas about how to accomplish this new way of learning, like deploying robots even if they aren’t yet perfect so they can learn from the environment they’re meant to work in. Today, Silicon Valley roboticists are dreaming big again. Here’s how that happened.  Jibo Jibo A movable social robot carried out conversations long before the age of LLMs. An MIT robotics researcher named Cynthia Breazeal introduced an armless, legless, faceless robot called Jibo to the world in 2014. It looked, in fact, like a lamp. Breazeal’s aim was to create a social robot for families, and the idea pulled in $3.7 million in a crowdsourced funding campaign. Early preorders cost $749. The early Jibo could introduce itself and dance to entertain kids, but that was about it. The vision was always for it to become a sort of embodied assistant that could handle everything from scheduling and emails to telling stories. It earned a number of devoted users, but ultimately the company shut down in 2019. A crowdfunding campaign started in 2014 and drew 4,800 Jibo preorders.COURTESY OF MIT MEDIA LAB In retrospect, one thing that Jibo really needed was better language capabilities. It was competing against Apple’s Siri and Amazon’s Alexa, and all those technologies at the time relied on heavy scripting. In broad terms, when you spoke to them, software would translate your speech into text, analyze what you wanted, and create a response pulled from preapproved snippets. Those snippets could be charming, but they were also repetitive and simply boring—downright robotic. That was especially a challenge for a robot that was supposed to be social and family oriented.  What has happened since, of course, is a revolution in how machines can generate language. Voice mode from any leading AI provider is now engaging and impressive, and multiple hardware startups are trying (and failing) to build products that take advantage of it.  But that comes with a new risk: While scripted conversations can’t really go off the rails, ones generated by AI certainly can. Some popular AI toys have, for example, talked to kids about how to find matches and knives.  OpenAI Dactyl A robot hand trained with simulations tries to model the unpredictability and variation of the real world. By 2018, every leading robotics lab was trying to scrap the old scripted rules and train robots through trial and error. OpenAI tried to train its robotic hand, Dactyl, virtually—with digital models of the hand and of the palm-size cubes Dactyl was supposed to manipulate. The cubes had letters and numbers on their faces; the model might set a task like “Rotate the cube so the red side with the letter O faces upward.” Here’s the problem: A robotic hand might get really good at doing this in its simulated world, but when you take that program and ask it to work on a real version in the real world, the slight differences between the two can cause things to go awry. Colors might be slightly different, or the deformable rubber in the robot’s fingertips could turn out to be stretchier than it was in simulation. Dactyl, part of OpenAI’s first attempt at robotics,

Calling a large language model API at scale is expensive and slow.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. The problem with thinking you’re part Neanderthal There’s a theory that many of us have an “inner Neanderthal.” The idea is that Homo sapiens and a cousin species once bred, leaving some people today with a trace of Neanderthal DNA.  This DNA is arguably the 21st century’s most celebrated discovery in human evolution. But in 2024, a pair of French geneticists called into question the theory’s very foundations.  They proposed that what scientists interpret as interbreeding could instead be explained by population structure—the way genes concentrate in smaller, isolated groups. Find out what it all means for human evolution. —Ben Crair This story is from the next issue of our print magazine, which is all about nature. Subscribe now to read it when it lands on Wednesday, April 22. Why having “humans in the loop” in an AI war is an illusion —Uri Maoz AI is starting to shape real wars. It’s at the center of a legal battle between Anthropic and the Pentagon, playing a growing role in the conflict with Iran, and raising questions about how much humans should remain “in the loop.” Under Pentagon guidelines, human oversight is meant to provide accountability, context, and security. But the idea of “humans in the loop” is a comforting distraction. The real danger isn’t that machines will act without oversight; it’s that human overseers have no idea what the machines are actually “thinking.” Thankfully, science may offer a way forward. Read the full op-ed on the urgent need for new safeguards around AI warfare. The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 Despite blacklisting Anthropic, the White House wants its new modelTrump officials are negotiating access to Mythos. (Axios)+ Anthropic said it was too dangerous for a public release. (Bloomberg $)+ Finance ministers are alarmed about the security risks. (BBC)+ Anthropic just rolled out a model that’s less risky than Mythos. (CNBC)+ The Pentagon has pursued a culture war against the company. (MIT Technology Review) 2 Sam Altman’s side hustles have raised conflict-of-interest concernsHis opaque investments could influence decisions at OpenAI. (WSJ $)+ A jury will soon decide if OpenAI abandoned its founding mission. (Wired $)+ The company is making a big play for science. (MIT Technology Review) 3 A Starlink outage during drone tests exposed the Pentagon’s SpaceX relianceIt was one of several Navy test disruptions linked to Starlink. (Reuters $)+ The DoD is also tapping Ford and GM for military innovations.(NYT $) 4 Data center delays threaten to choke AI expansion40% of this year’s projects are at risk of falling behind schedule. (FT $)+ Partly because no one wants a data center in their backyard. (MIT Technology Review) 5 Alibaba just released its own version of a world modelHappy Oyster is the latest attempt to extend AI’s ability to comprehend physical reality. (SCMP)+ But they still need to understand cause and effect. (FT $) 6 Google’s Gemini is now generating AI images tailored to personal dataBy analyzing users’ Google services and data. (Quartz)+ Google says it will cut the need for detailed prompts. (TechCrunch) 7 OpenAI is beefing up its agentic coding and development systemIts Codex update is a direct shot at Claude Code. (The Verge)+ But not everyone is convinced about AI coding. (MIT Technology Review) 8 Europe’s online age verification app is hereIt’s available for free to any company that wants it. (Wired $)  9 Smartglasses are giving Korean theaters hope of a K-Pop momentTheir AI-powered translations are taking the shows to the world. (NYT $) 10 Global voice actors are fighting Hollywood’s AI pushTheir voices are training the models that are replacing them. (Rest of World) Quote of the day “There’s this dark period between now and some time in the future where the advantage is very much offensive AI.”  —Rob Joyce, former director of cybersecurity at the National Security Agency, tells Bloomberg how AI is creating new hacking threats. One More Thing COURTESY OF NOVEON MAGNETICS The race to produce rare earth elements Access to rare earth elements will determine which countries meet their goals for lowering emissions or generating energy from non-fossil-fuel sources. But some nations, including the US, are worried about the supply of these elements.  China dominates the market, while extraction in the US is limited. As a result, scientists and companies are exploring unconventional sources. Read the full story on their search for critical minerals. —Mureji Fatunde We can still have nice things A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.)+ This ska cover of Rage Against the Machine is an upbeat way to start a revolution.+ We finally know how far Stretch Armstrong can really stretch.+ Customize these ambient sounds to wash away disruptive thoughts.+ Here’s proof childhood dreams can come true: a girl guiding a seal to perform tricks. 

You’ve probably written a decorator or two in your Python career.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Cyberscammers are bypassing banks’ security with illicit tools sold on Telegram  Inside a money-laundering center in Cambodia, an employee opens a banking app on his phone. It asks for a photo linked to the account, so he uploads a picture of a 30-something Asian man.  The app then requests a video “liveness” check. The scammer holds up a static image of a woman who doesn’t match the account. After 90 seconds, he’s in.  The exploit relies on illicit hacking services sold on Telegram that break “Know Your Customer” (KYC) facial scans. MIT Technology Review found 22 channels and groups advertising these services. This is what we discovered.  —Fiona Kelliher  Is carbon removal in trouble?  —Casey Crownhart  Last week, news emerged that Microsoft was pausing carbon removal purchases. It was a bombshell—Microsoft effectively is the carbon removal market, single-handedly purchasing around 80% of all contracted carbon removal.  The report sparked fear across the industry, raising questions about the future of carbon removal and the role of Big Tech. Read the full story.  This story is from The Spark, our weekly newsletter exploring the technology that could combat the climate crisis. Sign up to receive it in your inbox every Wednesday.  The quest to measure our relationship with nature  —Emma Marris  Humans have done some destructive things to the ecosystems around us. But conservationists are learning that we can also be a force for good.  To understand how we work best with nature, a group of scientists, authors, and philosophers have developed new measurements of human-nonhuman relationships. Now, a team in the United Nations is continuing the work. Find out why—and what they hope to achieve.  This story is from the next issue of our print magazine, which is all about nature. Subscribe now to read it when it lands on Wednesday, April 22.   The must-reads  I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.  1 Ukraine says Russian troops have surrendered to robots  They claim a fully automated attack captured army positions for the first time in history. (404 Media) + Europe’s vision for future wars is full of drones. (MIT Technology Review)  2 Monkeys with BCIs are navigating virtual worlds using only their thoughts The research could help people with paralysis. (New Scientist)  + But these implants still face a critical test. (MIT Technology Review)  3 NASA wants to put nuclear reactors on the Moon They could power lunar bases and extend spaceflight. (Wired $) + NASA is also building a nuclear-powered spacecraft. (MIT Technology Review)  4 Plans for online age verification in the US are raising red flags Experts warn of compliance issues and potential data breaches. (NBC News) + In the EU, an age verification app is about to launch. (Reuters $)  5 An AI chip boom just pushed Taiwan’s stock market past the UK’s It’s risen past $4 trillion to become the world’s seventh largest. (FT $) + Future AI chips could be built on glass. (MIT Technology Review)  6 The public backlash against data centers is intensifying in the US Protests and litigation are blocking projects. (CNBC) + One potential solution? Putting them in space. (MIT Technology Review)  7 Five-minute EV charging is becoming a reality China’s BYD has started rolling it out. (Gizmodo)  + “Extended-range electric vehicles” are about to hit US streets. (Atlantic $)  8 Stealth signals are bypassing Iran’s internet blackout  Files hidden in satellite TV broadcasts keep information flowing. (IEEE)  9 Shoe brand Allbirds made a shock pivot to AI, sending stock up 700%  No bubble to see here, folks. (CNBC)  + What even is the AI bubble? (MIT Technology Review)  10 The largest ever map of the universe is complete  It captures 47 million galaxies and quasars. (Space.com)  Quote of the day  “I like the internet as much as anybody, but we’ve got to go on an internet diet. We don’t need to pay for corporations to do their internet stuff.”   —Sylvia Whitt, a 78-year-old retiree based in Virginia, tells the Washington Post why they’re protesting against data centers.   One More Thing  ISRAEL VARGAS AI and the future of sex  Some Republican lawmakers want to criminalize porn and arrest its creators. But what if porn is wholly created by an algorithm? In that case, whether it’s obscene, ethical, or safe becomes a secondary issue. The primary concern will be what it means for porn to be “real”—and what the answer demands from all of us.  Technological advances could even remove the “messy humanity” from sex itself. The rise of AI-generated porn may be a symptom of a new synthetic sexuality, not the cause. Read the full story.  —Leo Herrera  We can still have nice things  A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.)  + An animator turned his son’s drawings into epic anime characters. + Hundreds of baby green sea turtles made a spectacular first journey to the ocean. + You can now track rocket launches from take-off to orbit in real time. + These musical mistakes prove that even the classics aren’t perfect.