Committee

arXiv:2602.22723v1 Announce Type: new Abstract: There is growing recognition that many NLP tasks lack a single ground truth, as human judgments reflect diverse perspectives. To capture this variation, models have been developed to predict full annotation distributions rather than majority labels, while perspectivist models aim to reproduce the interpretations of individual annotators. In this work, we compare these approaches on Implicit Discourse Relation Recognition (IDRR), a highly ambiguous task where disagreement often arises from cognitive complexity rather than ideological bias. Our experiments show that existing annotator-specific models perform poorly in IDRR unless ambiguity is reduced, whereas models trained on label distributions yield more stable predictions. Further analysis indicates that frequent cognitively demanding cases drive inconsistency in human interpretation, posing challenges for perspectivist modeling in IDRR.

arXiv:2602.22647v1 Announce Type: cross Abstract: Generative retrieval has emerged as a powerful paradigm for LLM-based recommendation. However, industrial recommender systems often benefit from restricting the output space to a constrained subset of items based on business logic (e.g. enforcing content freshness or product category), which standard autoregressive decoding cannot natively support. Moreover, existing constrained decoding methods that make use of prefix trees (Tries) incur severe latency penalties on hardware accelerators (TPUs/GPUs). In this work, we introduce STATIC (Sparse Transition Matrix-Accelerated Trie Index for Constrained Decoding), an efficient and scalable constrained decoding technique designed specifically for high-throughput LLM-based generative retrieval on TPUs/GPUs. By flattening the prefix tree into a static Compressed Sparse Row (CSR) matrix, we transform irregular tree traversals into fully vectorized sparse matrix operations, unlocking massive efficiency gains on hardware accelerators. We deploy STATIC on a large-scale industrial video recommendation platform serving billions of users. STATIC produces significant product metric impact with minimal latency overhead (0.033 ms per step and 0.25% of inference time), achieving a 948x speedup over a CPU trie implementation and a 47-1033x speedup over a hardware-accelerated binary-search baseline. Furthermore, the runtime overhead of STATIC remains extremely low across a wide range of practical configurations. To the best of our knowledge, STATIC enables the first production-scale deployment of strictly constrained generative retrieval. In addition, evaluation on academic benchmarks demonstrates that STATIC can considerably improve cold-start performance for generative retrieval. Our code is available at https://github.com/youtube/static-constraint-decoding.

Burrowed in the alleys of Hongik-dong, a hushed residential neighborhood in eastern Seoul, is a faded stone-tiled building stamped “Korea Baduk Association,” the governing body for professional Go. The game is an ancient one, with sacred stature in South Korea.  But inside the building, rooms once filled with the soft clatter of hands dipping into wooden bowls of stones now echo with mouse clicks. Players hunch over their monitors and replay their matches in an AI program. Others huddle around a Go board and debate the best next move, while coaches report how their choices stack up against the AI’s. Some sit in silence, watching AI programs play against each other.  Ten years ago AlphaGo, Google DeepMind’s AI program, stunned the world by defeating the South Korean Go player Lee Sedol. And in the years since, AI has upended the game. It’s overturned centuries-old principles about the best moves and introduced entirely new ones. Players now train to replicate AI’s moves as closely as they can rather than inventing their own, even when the machine’s thinking remains mysterious to them. Today, it is essentially impossible to compete professionally without using AI. Some say the technology has drained the game of its creativity, while others think there is still room for human invention. Meanwhile, AI is democratizing access to training, and more female players are climbing the ranks as a result.  For Shin Jin-seo, the top-ranked Go player in the world, AI is an invaluable training partner. Every morning, he sits at his computer and opens a program called KataGo. Nicknamed “Shintelligence” for how closely his moves mimic AI’s, he traces the glowing “blue spot” that represents the program’s suggestion for the best next move, rearranging the stones on the digital grid to try to understand the machine’s thinking. “I constantly think about why AI chose a move,” he says. When training for a match, Shin spends most of his waking hours poring over KataGo. “It’s almost like an ascetic practice,” he says. According to a study in 2022 by the Korean Baduk League, Shin’s moves match AI’s 37.5% of the time, well above the 28.5% average the study found among all players. “My game has changed a lot,” says Shin, “because I have to follow the directions suggested by AI to some extent.” The Korea Baduk Association says it has reached out to Google DeepMind in the hopes of arranging a match between Shin and AlphaGo, to commemorate the 10th anniversary of its victory over Lee. A spokesperson for Google DeepMind said the company could not provide information at this time. But if a new match does happen, Shin, who has trained on more advanced AI programs, is optimistic that he’d win. “AlphaGo still had some flaws then, so I think I could beat it if I target those weaknesses,” he says. AI rewrites the Go playbook Go is an abstract strategy board game invented in China more than 2,500 years ago. Two players take turns placing black and white stones on a 19×19 grid, aiming to conquer territory by surrounding their opponent’s stones. It’s a game of striking mathematical complexity. The number of possible board configurations—roughly 10170—dwarfs the number of atoms in the universe. If chess is a battle, Go is a war. You suffocate your enemy in one corner while fending off an invasion in another. To train AI to play Go, a vast trove of human Go moves are fed into a neural network, a computing system that mimics the web of neurons in the human brain. AlphaGo, which was later christened AlphaGo Lee after its victory over Lee Sedol, was trained on 30 million Go moves and refined by playing millions of games against itself. In 2017, its successor, AlphaGo Zero, picked up Go from scratch. Without studying any human games, it learned by playing against itself, with moves based only on the rules of the game. The blank-slate approach proved more powerful, unconstrained by the limits of human knowledge. After three days of training, it beat AlphaGo Lee 100 games to zero.  Google DeepMind retired AlphaGo that same year. But then a wave of open-source models inspired by AlphaGo Zero emerged. Today, KataGo is the program most widely used by professional Go players in South Korea. It’s faster and sharper than AlphaGo. It’s learned to predict not just who will win, but also who owns each point on the board at any given moment. While AlphaGo Zero pieced together its understanding of the board by looking at small sections, KataGo learned to read the whole board, developing better judgment for long-term strategies. Instead of just learning how to win, it learned to maximize its score. The software has reshaped how people play. For hundreds of years, professional Go players have navigated the game’s astronomical complexity by developing heuristics that replaced brute calculation. Elegant opening strategies imposed abstract order on the empty grid. Invading corners early was a bad bargain. Each generation of Go players added new principles to the canon.  But “AI has changed everything,” says Park Jeong-sang, a South Korean Go commentator. “Fundamental moves that were once considered common sense aren’t played at all today, and techniques that didn’t exist before have become popular.”  The starkest shift has been in opening moves. Go starts on a blank grid, and the first 50 moves were canvases for abstract thinking and creativity, where players etched their personalities and philosophies. Lee Sedol fashioned provocative moves that invited chaos. Ke Jie, a Chinese player who was defeated by AlphaGo Master in 2017, dazzled with agile, imaginative moves. Now, players memorize the same strain of efficient, calculated opening moves suggested by AI. The crux of the game has shifted to the middle moves, where raw calculation matters more than creativity. Training with AI has led to a homogenization of playing styles. Ke Jie has lamented the strain of watching the same opening moves recycled endlessly. “I feel the exact same way as the fans watching. It’s very tiring

In the current AI landscape, we’ve become accustomed to the ‘ephemeral agent’—a brilliant but forgetful assistant that restarts its cognitive clock with every new chat session. While LLMs have become master coders, they lack the persistent state required to function as true teammates. Nous Research team released Hermes Agent, an open-source autonomous system designed to solve the two biggest bottlenecks in agentic workflows: memory decay and environmental isolation. Built on the high-steerability Hermes-3 model family, Hermes Agent is billed as the assistant that ‘grows with you.’ The Memory Hierarchy: Learning via Skill Documents For an agent to ‘grow,’ it needs more than just a large context window. Hermes Agent utilizes a multi-level memory system that mimics procedural learning. While it handles short-term tasks through standard inference, its long-term utility is driven by Skill Documents. When Hermes Agent completes a complex task—such as debugging a specific microservice or optimizing a data pipeline—it can synthesize that experience into a permanent record. These records are stored as searchable markdown files following the agentskills.io open standard. Procedural Memory: The next time you ask the agent to perform a similar task, it doesn’t start from scratch. It queries its own library of Skill Documents to ‘remember’ the successful steps it took previously. Contextual Persistence: Unlike standard RAG (Retrieval-Augmented Generation), which often pulls disjointed snippets, this system allows the agent to maintain a cohesive understanding of your specific codebase and preferences over weeks or months. Persistent Machine Access: Beyond the Sandbox A major friction point for AI devs is the ‘execution gap.’ Most agents write code but cannot interact with the real world without heavy manual intervention. Hermes Agent closes this gap by providing persistent dedicated machine access. The agent is designed to live inside a functional environment, supporting five distinct backends: Local: Direct interaction with the host machine. Docker: Isolated, reproducible containers for safe code execution. SSH: The ability to log into remote servers or cloud instances. Singularity: High-performance computing (HPC) container support. Modal: Serverless execution for scaling heavy workloads. This persistence is critical for AI devs. You can initialize a long-running EDA (Exploratory Data Analysis) on a remote server via SSH, log-off, and return later. The agent maintains the terminal state, handles background processes, and tracks file system changes independently. It isn’t just simulating a conversation; it is managing a workspace. The Gateway: An Agent in Your Pocket While most technical agents are confined to a CLI or a proprietary web dashboard, Nous Research has prioritized accessibility through the Hermes Gateway. The system integrates directly with existing communication stacks, including Telegram, Discord, Slack, and WhatsApp. This allows for a continuous feedback loop: an engineer can start a task at their workstation and receive a ‘task completed’ notification via Telegram. Through the gateway, you can send follow-up instructions or even voice memos that the agent processes and executes within its persistent environment. Under the Hood: The ReAct Loop and Steerability For the AI devs building on this, the architecture is a refined implementation of the ReAct (Reasoning and Acting) loop. The agent follows a structured cycle: Observation: Reading terminal output or file contents. Reasoning: Analyzing the current state against the goal. Action: Executing a command or calling a tool. This is powered by Hermes-3 (based on Llama 3.1), which was trained using a specialized reinforcement learning framework called Atropos. This training specifically targets tool-calling accuracy and long-range planning, ensuring the agent doesn’t get ‘lost’ during multi-step deployments. Key Takeaways Persistent Machine Access: Unlike stateless chatbots, it operates in real terminal environments (Docker, SSH, Local, etc.), allowing it to run long-term tasks and maintain file states across sessions. Self-Evolving ‘Skill Documents’: It uses a multi-level memory system to record successful workflows as searchable markdown files (via agentskills.io), meaning it literally gets smarter the more you use it. Precision ‘Hermes-3’ Thinking: Powered by the Llama 3.1-based Hermes-3 model, it is fine-tuned with Atropos RL for high steerability and reliable tool-calling within complex reasoning loops. Omnipresent Gateway: You can interact with your agent via Telegram, Discord, or Slack, enabling you to manage heavy engineering tasks or receive status updates from your phone. Check out the Technical details and GitHub Repo. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Nous Research Releases ‘Hermes Agent’ to Fix AI Forgetfulness with Multi-Level Memory and Dedicated Remote Terminal Access Support appeared first on MarkTechPost.

arXiv:2506.09886v2 Announce Type: replace Abstract: Detecting hallucinations in large language models (LLMs) is critical for their safety in many applications. Without proper detection, these systems often provide harmful, unreliable answers. In recent years, LLMs have been actively used in retrieval-augmented generation (RAG) settings. However, hallucinations remain even in this setting, and while numerous hallucination detection methods have been proposed, most approaches are not specifically designed for RAG systems. To overcome this limitation, we introduce a hallucination detection method based on estimating the distances between the distributions of prompt token embeddings and language model response token embeddings. The method examines the geometric structure of token hidden states to reliably extract a signal of factuality in text, while remaining friendly to long sequences. Extensive experiments demonstrate that our method achieves state-of-the-art or competitive performance. It also has transferability from solving the NLI task to the hallucination detection task, making it a fully unsupervised and efficient method with a competitive performance on the final task.

arXiv:2602.21368v1 Announce Type: cross Abstract: Given a black-box AI system and a task, at what confidence level can a practitioner trust the system’s output? We answer with a reliability level — a single number per system-task pair, derived from self-consistency sampling and conformal calibration, that serves as a black-box deployment gate with exact, finite-sample, distribution-free guarantees. Self-consistency sampling reduces uncertainty exponentially; conformal calibration guarantees correctness within 1/(n+1) of the target level, regardless of the system’s errors — made transparently visible through larger answer sets for harder questions. Weaker models earn lower reliability levels (not accuracy — see Definition 2.4): GPT-4.1 earns 94.6% on GSM8K and 96.8% on TruthfulQA, while GPT-4.1-nano earns 89.8% on GSM8K and 66.5% on MMLU. We validate across five benchmarks, five models from three families, and both synthetic and real data. Conditional coverage on solvable items exceeds 0.93 across all configurations; sequential stopping reduces API costs by around 50%.

When a company claims to have created what’s essentially the holy grail of batteries, there are bound to be some questions. Interest has been swirling since Donut Lab, a Finnish company, announced last month that it had a new solid-state battery technology, one that was ready for large-scale production. The company said its batteries can charge super-fast and have a high energy density that would translate to ultra-long-range EVs. What’s more, it claimed the cells can operate safely in the extreme heat and cold, contain “green and abundant materials,” and would cost less than lithium-ion batteries do today. It sounded amazing—this sort of technology could transform the EV industry. But many quickly wondered if it was all too good to be true. Now, Donut Lab is releasing a series of videos that it says will prove its technology has the secret sauce. Let’s dig into why this company is making news, why many experts are skeptical, and what it all means for the battery industry right now. Solid-state batteries could deliver the next generation of EVs. In place of a liquid electrolyte (the material that ions move through inside a battery), the cells use a solid material, so they can be more compact. That means a significantly longer range, which could get more people excited to drive EVs. The problem is, getting these batteries to work and making them at the large scale required for the EV industry hasn’t been a simple task. Some of the world’s most powerful automakers and battery companies have been trying for years to get the technology off the ground. (Toyota at one point said it would have solid-state batteries in cars by 2020. Now it’s shooting for 2027 or 2028.) While it’s been a long time coming, it does feel as if solid-state batteries are closer than ever. Much of the progress so far has been on semi-solid-state batteries, which use materials like gels for electrolytes. But some companies, including several in China, are getting closer to true solid state. The world’s largest battery company, CATL, plans to manufacture small quantities in 2027. Another major Chinese automaker, Changan, plans to start testing installation of all-solid-state batteries in vehicles this year, with mass production expected to begin next year. Still, Donut Lab surprised the battery industry when, in a video released in early January ahead of the Consumer Electronics Show in Las Vegas, the company claimed it would put the world’s first all-solid-state battery into production vehicles. One of the splashiest claims in the announcement was that cells would have an energy density of 400 watt-hours per kilogram (the top commercial lithium-ion batteries today sit at about 250 to 300 Wh/kg). It was also claimed that the cells could charge in as little as five minutes, last 100,000 cycles, and retain 99% of capacity at high and low temperatures—while costing less than lithium-ion cells and being made from “100% green and abundant materials with global availability.” Many experts were immediately skeptical. “In the solid-state field, the technical barriers are very high,” said Shirley Meng, a professor of molecular engineering at the University of Chicago, when I spoke with her last month. She’d recently attended CES and visited Donut Lab’s booth. “They had zero demo, so I don’t believe it,” she says. “Call me conservative, but I would rather be careful than be sorry later.” “It’s one of those things where nobody knows—they’ve never heard of it,” said Eric Wachsman, a professor at the University of Maryland and cofounder of the solid-state battery company Ion Storage Systems, in a January interview. “They came out of nowhere.” Donut Lab has shared very little about what, exactly, this technology might be. It’s not uncommon for battery companies (or any startup, for that matter) to be quiet about technical details before they can get patents filed to protect their technology. But the combination of claims didn’t seem to line up with any known chemistries, leaving experts speculating and, in many cases, doubting Donut Lab’s claims. “All the parameters are contradictory,” said Yang Hongxin, chairman and CEO of the Chinese battery giant Svolt Energy, in remarks to news outlets in January. For example, there’s often a trade-off between high energy density, which requires thicker electrodes that can store more energy, and fast charging, which requires ions to move quickly through cells. High-performance batteries are also expected to be costly, but Donut Lab claims its technology will be cheaper than lithium-ion technology.  In a new video released last week, Donut Lab cofounder and CEO Marko Lehtimäki announced the company would be releasing a video series, called “I Donut Believe,” that would provide evidence for their claims. As a header on the accompanying website reads: “Fair enough. Here you go.” When the website went up last week, it included a countdown timer to Monday February 23, when the company released results from its first third-party testing: a fast charging test. The test showed that a single cell could charge from 0% to 80% capacity in about four and a half minutes—incredibly quick and quite impressive results. (One potential caveat to note is that the cells heated up quite a bit, so thermal management could be important in designing vehicles that use these batteries.) Even as we see the first technical test results, I’m still left with a lot of questions. How many cycles could this battery do at this charging speed? Can this same cell meet the company’s other performance claims? (I’ve reached out to Donut Lab several times over the past month, both to the company’s press email and to leadership on LinkedIn, but I haven’t gotten a response yet.) The company has certainly drummed up a lot of interest and attention with its rollout, and the theatrics aren’t over yet. There’s another countdown timer on Donut Lab’s site, which ends on Monday, March 2. I’m the first one to get excited about a new battery technology. But there’s a sentiment I’ve seen pop up a lot recently online, and

Eons ago, in 2012, I had a weird experience. My iPhone suddenly shut down. When I restarted it, I found it was totally reset—clean, like a new device. This was the early days of iOS, so I wasn’t too concerned until I went to connect it to my computer to restore it from a backup. But when I flipped open the lid of my laptop, it too was mid-restart. And then, suddenly, the screen went gray. It was being remotely wiped. I turned on my iPad. It, too, had been wiped. I was being hacked.  Frantically, I shut down all my devices, unplugged everything connected to the internet in my house, turned off my router, and went next door to use my neighbors’ computer and find out what was going on. Deepening my panic, I realized hackers had also gained control of, and nuked, my Google account. Worse, they were in control of my Twitter, which they were gleefully using to spew all sorts of vile comments. It was nasty.  You have to remember, this was before all of us lived with a constant rain of text messages and emails designed to elicit the information necessary to pull something like this off. These crooks hadn’t brute-forced their way in, or used any sort of sophisticated techniques to gain access to my accounts. Instead, they had relied on publicly available information, and a fake credit card number, to socially engineer their way into my Amazon account, where they looked up the last four digits of my real credit card number. Then they used that information to get into Apple. And because that account was linked to my Gmail, and that to my Twitter, it gave them the keys to everything. But what really troubled me was what I learned as I followed up on my hack over the ensuing weeks and months: This kind of thing was, while still novel, becoming more common. Some version of what happened to me had happened to lots of other people. The kids who were responsible—it was a couple of kids—weren’t criminal masterminds. They had just found a gap, a place where a technology was now commonplace but its risks and exploitable surface areas weren’t yet fully understood. I just happened to have all my stuff in the gap. Today that gap might feature a crypto wallet or a deepfake of a loved one’s voice. (Or both.) Crime changes. The goals stay the same—pursuit of value, pursuit of power—but new technologies create new vulnerabilities, new tactics, and new ways for perpetrators to evade discovery or capture. And the law necessarily lags behind. Relying not on innovation but on precedent, it is intentionally backward-looking and slow. That plodding consideration used to be how we protected our shared democratic society, how we protected each other from each other. But those same new technologies that have allowed crime to outpace law have also reenergized law enforcement and government—offering new ways to root out crime, to gather evidence, to surveil people. Think, for example, of how cold-case investigators tracked down the Golden State Killer years after his murders, using DNA samples and genealogy databases—launching a new era of DNA-powered investigations.  Technology has long made crime and its prosecution a game of cat and mouse. It sometimes calls into question the nature of crime itself. Unregulated behaviors, facilitated by technology, can exist in murky zones of dubious legality. (Until TikTok announced its new ownership structure, Apple and Google were both technically breaking the law by allowing the app to stay on their platforms, under the provisions of the Protecting Americans from Foreign Adversary Controlled Applications Act. Ah! Well. Nevertheless.) That tension is the key to our March/April issue. Thanks to technologies like cryptocurrency and off-the-shelf autonomous autopilots, there’s never been a better time to do crime. Thanks to pervasive surveillance and digital infrastructure, there’s never been a better time to fight it—sometimes at the expense of what we used to think of as fundamental civil rights.  I never pressed charges against the kids who hacked me. The biggest consequence of the hack was that Apple set up two-factor authentication in the following months, which felt like a win. Now I’m not sure anyone expects their personal data to be secure in any meaningful way. I’m certain, though, that somewhere on the net, a new generation of kids is coming up with another novel crime.