Uncategorized

arXiv:2603.10697v1 Announce Type: cross Abstract: Neural text-to-SQL models, which translate natural language questions (NLQs) into SQL queries given a database schema, have achieved remarkable performance. However, database schemas frequently evolve to meet new requirements. Such schema evolution often leads to performance degradation for models trained on static schemas. Existing work either mainly focuses on simply paraphrasing some syntactic or semantic mappings among NLQ, DB and SQL, or lacks a comprehensive and controllable way to investigate the model robustness issue under the schema evolution, which is insufficient when facing the increasingly complex and rich database schema changes in reality, especially in the LLM era. To address the challenges posed by schema evolution, we present EvoSchema, a comprehensive benchmark designed to assess and enhance the robustness of text-to-SQL systems under real-world schema changes. EvoSchema introduces a novel schema evolution taxonomy, encompassing ten perturbation types across columnlevel and table-level modifications, systematically simulating the dynamic nature of database schemas. Through EvoSchema, we conduct an in-depth evaluation spanning different open source and closed-source LLMs, revealing that table-level perturbations have a significantly greater impact on model performance compared to column-level changes. Furthermore, EvoSchema inspires the development of more resilient text-to-SQL systems, in terms of both model training and database design. The models trained on EvoSchema’s diverse schema designs can force the model to distinguish the schema difference for the same questions to avoid learning spurious patterns, which demonstrate remarkable robustness compared to those trained on unperturbed data on average. This benchmark offers valuable insights into model behavior and a path forward for designing systems capable of thriving in dynamic, real-world environments.

arXiv:2603.09688v2 Announce Type: replace Abstract: This research focuses on developing advanced methods for assessing similarity between recipes by combining different sources of information and analytical approaches. We explore the semantic, lexical, and domain similarity of food recipes, evaluated through the analysis of ingredients, preparation methods, and nutritional attributes. A web-based interface was developed to allow domain experts to validate the combined similarity results. After evaluating 318 recipe pairs, experts agreed on 255 (80%). The evaluation of expert assessments enables the estimation of which similarity aspects–lexical, semantic, or nutritional–are most influential in expert decision-making. The application of these methods has broad implications in the food industry and supports the development of personalized diets, nutrition recommendations, and automated recipe generation systems.

arXiv:2603.11008v1 Announce Type: cross Abstract: Pseudo-relevance feedback (PRF) methods built on large language models (LLMs) can be organized along two key design dimensions: the feedback source, which is where the feedback text is derived from and the feedback model, which is how the given feedback text is used to refine the query representation. However, the independent role that each dimension plays is unclear, as both are often entangled in empirical evaluations. In this paper, we address this gap by systematically studying how the choice of feedback source and feedback model impact PRF effectiveness through controlled experimentation. Across 13 low-resource BEIR tasks with five LLM PRF methods, our results show: (1) the choice of feedback model can play a critical role in PRF effectiveness; (2) feedback derived solely from LLM-generated text provides the most cost-effective solution; and (3) feedback derived from the corpus is most beneficial when utilizing candidate documents from a strong first-stage retriever. Together, our findings provide a better understanding of which elements in the PRF design space are most important.

Just a few years ago, the battery industry was hot, hot, hot. There was a seemingly infinite number of companies popping up, with shiny new chemistries and massive fundraising rounds. My biggest problem was sifting through the pile to pick the most exciting news to cover. That tide has turned, and in 2026, what seems to be in unlimited supply isn’t battery success stories but stumbles or straight-up implosions. Companies are failing, investors are pulling back, and batteries, especially for EVs, aren’t looking so hot anymore. On Monday, Steve Levine at The Information (paywalled link) reported that 24M Technologies, a battery company founded in 2010, was shutting down and would auction off its property. The company itself has been silent, but this is the latest in a string of bad signs, and it’s a big one—at one point 24M was worth over $1 billion, and the company’s innovations could have worked with existing technology. So where does that leave the battery industry? Many buzzy battery startups in recent years have been trying to sell some new, innovative chemistry to compete with lithium-ion batteries, the status quo that powers phones, laptops, electric vehicles, and even grid storage arrays today. Think sodium-ion batteries and solid-state cells. 24M wasn’t trying to sell a departure from lithium-ion but improvements that could work with the tech. One of the company’s major innovations was its manufacturing process, which involved essentially smearing materials onto sheets of metal to form the electrodes, a simpler and potentially cheaper technique than the standard one.  The layers in the company’s batteries were thicker, which cut down on some of the inactive materials in cells and improved the energy density. That allows more energy to be stored in a smaller package, boosting the range of EVs—the company famously had a goal of a 1,000-mile battery (about 1,600 kilometers). We’re still thin on details of what exactly went down at 24M and what comes next for its tech. The company didn’t get back to my questions sent to the official press email, and nobody picked up the phone when I called. 24M cofounder and MIT professor Yet-Ming Chiang declined to speak on the record. For those who have been closely following the battery industry, more bad news isn’t too surprising. It feels as if everyone is short on money these days, and as purse strings tighten, there’s less interest in novel ideas. “It just feels like there’s not a lot of appetite for innovation,” says Kara Rodby, a technical principal at Volta Energy Technologies, a venture capital firm that focuses on the energy storage industry. Natron Energy, one of the leading sodium-ion startups in the US, shut down operations in September last year. Ample, an EV battery-swapping company, filed for bankruptcy in December 2025.   There were always going to be failures from the recent battery boom. Money was flowing to all sorts of companies, some pitching truly wild ideas. But what recent months have made clear is that the battery market is turning brutal, even for the relatively safe bets. Because 24M’s technology was designed to work into existing lithium-ion chemistry, it could have been an attractive candidate for existing battery companies to license or even acquire. “It’s a great example of something that should have been easier,” Rodby says.   The gutting of major components of the Inflation Reduction Act, key legislation in the US that provided funding and incentives for batteries and EVs, certainly hasn’t helped. The EV market in the US is cooling off, with automakers canceling EV models and slashing factory plans. There are bright spots. China’s battery industry is thriving, and its battery and EV giants are looking ever more dominant. The market for stationary energy storage is also still seeing positive signs of growth, even in the US.  But overall, it’s not looking great.  This article is from The Spark, MIT Technology Review’s weekly climate newsletter. To receive it in your inbox every Wednesday, sign up here. 

arXiv:2603.09416v1 Announce Type: new Abstract: Large Language Models (LLMs) excel in Natural Language Processing (NLP) tasks, but they often propagate biases embedded in their training data, which is potentially impactful in sensitive domains like healthcare. While existing benchmarks evaluate biases related to individual social determinants of health (SDoH) such as gender or ethnicity, they often overlook interactions between these factors and lack context-specific assessments. This study investigates bias in LLMs by probing the relationships between gender and other SDoH in French patient records. Through a series of experiments, we found that embedded stereotypes can be probed using SDoH input and that LLMs rely on embedded stereotypes to make gendered decisions, suggesting that evaluating interactions among SDoH factors could usefully complement existing approaches to assessing LLM performance and bias.

arXiv:2603.09403v1 Announce Type: new Abstract: Validating evaluation metrics for NLG typically relies on expensive and time-consuming human annotations, which predominantly exist only for English datasets. We propose textit{LLM as a Meta-Judge}, a scalable framework that utilizes LLMs to generate synthetic evaluation datasets via controlled semantic degradation of real data, replacing human judgment. We validate our approach using textit{meta-correlation}, measuring the alignment between metric rankings derived from synthetic data and those from standard human benchmarks. Experiments across Machine Translation, Question Answering, and Summarization demonstrate that synthetic validation serves as a reliable proxy for human judgment, achieving meta-correlations exceeding 0.9 in multilingual QA and proves to be a viable alternative where human judgments are unavailable or too expensive to obtain. Our code and data will become publicly available upon paper acceptance.

arXiv:2409.09245v3 Announce Type: replace-cross Abstract: The discontinuous operations inherent in quantization and sparsification introduce a long-standing obstacle to backpropagation, particularly in ultra-low precision and sparse regimes. While the community has long viewed quantization as unfriendly to gradient descent due to its lack of smoothness, we pinpoint-for the first time-that the key issue is the absence of a proper gradient path that allows training to learn robustness to quantization noise. The standard Straight-Through Estimator (STE) exacerbates this with its well-understood mismatch: a quantization-aware forward pass but oblivious backward pass, leading to unmanaged error and instability. We solve this by explicitly modeling quantization as additive noise, making the full forward-backward path well-defined without heuristic gradient estimation. As one natural solution, we introduce a denoising dequantization transform derived from a principled ridge regression objective, creating an explicit, corrective gradient path that makes learning robust to the noise STE bypasses. We extend this to sparsification by treating it as a special form of quantization that zeros out small values. Our unified framework trains models at arbitrary precisions and sparsity levels with off-the-shelf recipes, enabling stable A1W1 and sub-1-bit networks where others falter. It yields state-of-the-art results, mapping efficiency frontiers for modern LLMs and providing a theoretically grounded path to hyper-efficient neural networks.