Committee

arXiv:2511.12784v1 Announce Type: new Abstract: Large Language Models (LLMs) have recently emerged as powerful tools for autoformalization. Despite their impressive performance, these models can still struggle to produce grounded and verifiable formalizations. Recent work in text-to-SQL, has revealed that LLMs can be sensitive to paraphrased natural language (NL) inputs, even when high degrees of semantic fidelity are preserved (Safarzadeh, Oroojlooyjadid, and Roth 2025). In this paper, we investigate this claim in the autoformalization domain. Specifically, we evaluate the robustness of LLMs generating formal proofs with semantically similar paraphrased NL statements by measuring semantic and compilation validity. Using the formal benchmarks MiniF2F (Zheng, Han, and Polu 2021) and Lean 4 version of ProofNet (Xin et al. 2024), and two modern LLMs, we generate paraphrased natural language statements and cross-evaluate these statements across both models. The results of this paper reveal performance variability across paraphrased inputs, demonstrating that minor shifts in NL statements can significantly impact model outputs.

arXiv:2511.13646v1 Announce Type: cross Abstract: Large Language Models (LLMs) are reshaping almost all industries, including software engineering. In recent years, a number of LLM agents have been proposed to solve real-world software problems. Such software agents are typically equipped with a suite of coding tools and can autonomously decide the next actions to form complete trajectories to solve end-to-end software tasks. While promising, they typically require dedicated design and may still be suboptimal, since it can be extremely challenging and costly to exhaust the entire agent scaffold design space. Recognizing that software agents are inherently software themselves that can be further refined/modified, researchers have proposed a number of self-improving software agents recently, including the Darwin-G”odel Machine (DGM). Meanwhile, such self-improving agents require costly offline training on specific benchmarks and may not generalize well across different LLMs or benchmarks. In this paper, we propose Live-SWE-agent, the first live software agent that can autonomously and continuously evolve itself on-the-fly during runtime when solving real-world software problems. More specifically, Live-SWE-agent starts with the most basic agent scaffold with only access to bash tools (e.g., mini-SWE-agent), and autonomously evolves its own scaffold implementation while solving real-world software problems. Our evaluation on the widely studied SWE-bench Verified benchmark shows that Live-SWE-agent can achieve an impressive solve rate of 75.4% without test-time scaling, outperforming all existing open-source software agents and approaching the performance of the best proprietary solution. Moreover, Live-SWE-agent outperforms state-of-the-art manually crafted software agents on the recent SWE-Bench Pro benchmark, achieving the best-known solve rate of 45.8%.

arXiv:2511.11324v1 Announce Type: new Abstract: Digitized histopathology analysis involves complex, time-intensive workflows and specialized expertise, limiting its accessibility. We introduce NOVA, an agentic framework that translates scientific queries into executable analysis pipelines by iteratively generating and running Python code. NOVA integrates 49 domain-specific tools (e.g., nuclei segmentation, whole-slide encoding) built on open-source software, and can also create new tools ad hoc. To evaluate such systems, we present SlideQuest, a 90-question benchmark — verified by pathologists and biomedical scientists — spanning data processing, quantitative analysis, and hypothesis testing. Unlike prior biomedical benchmarks focused on knowledge recall or diagnostic QA, SlideQuest demands multi-step reasoning, iterative coding, and computational problem solving. Quantitative evaluation shows NOVA outperforms coding-agent baselines, and a pathologist-verified case study links morphology to prognostically relevant PAM50 subtypes, demonstrating its scalable discovery potential.

arXiv:2411.03895v2 Announce Type: replace Abstract: In theatre, playwrights use the portrayal of characters to explore culturally based gender norms. In this paper, we develop quantitative methods to study gender depiction in the non-religious works (comedias) of Pedro Calder’on de la Barca, a prolific Spanish 17th century author. We gather insights from a corpus of more than 100 plays by using a gender classifier and applying model explainability (attribution) methods to determine which text features are most influential in the model’s decision to classify speech as ‘male’ or ‘female’, indicating the most gendered elements of dialogue in Calder’on’s comedias in a human accessible manner. We find that female and male characters are portrayed differently and can be identified by the gender prediction model at practically useful accuracies (up to f=0.83). Analysis reveals semantic aspects of gender portrayal, and demonstrates that the model is even useful in providing a relatively accurate scene-by-scene prediction of cross-dressing characters.

arXiv:2511.10846v1 Announce Type: new Abstract: Automated emotion detection is widely used in applications ranging from well-being monitoring to high-stakes domains like mental health and hiring. However, models often rely on annotations that reflect dominant cultural norms, limiting model ability to recognize emotional expression in dialects often excluded from training data distributions, such as African American Vernacular English (AAVE). This study examines emotion recognition model performance on AAVE compared to General American English (GAE). We analyze 2.7 million tweets geo-tagged within Los Angeles. Texts are scored for strength of AAVE using computational approximations of dialect features. Annotations of emotion presence and intensity are collected on a dataset of 875 tweets with both high and low AAVE densities. To assess model accuracy on a task as subjective as emotion perception, we calculate community-informed “silver” labels where AAVE-dense tweets are labeled by African American, AAVE-fluent (ingroup) annotators. On our labeled sample, GPT and BERT-based models exhibit false positive prediction rates of anger on AAVE more than double than on GAE. SpanEmo, a popular text-based emotion model, increases false positive rates of anger from 25 percent on GAE to 60 percent on AAVE. Additionally, a series of linear regressions reveals that models and non-ingroup annotations are significantly more correlated with profanity-based AAVE features than ingroup annotations. Linking Census tract demographics, we observe that neighborhoods with higher proportions of African American residents are associated with higher predictions of anger (Pearson’s correlation r = 0.27) and lower joy (r = -0.10). These results find an emergent safety issue of emotion AI reinforcing racial stereotypes through biased emotion classification. We emphasize the need for culturally and dialect-informed affective computing systems.

arXiv:2511.10686v1 Announce Type: new Abstract: This work aims to investigate how different Large Language Models (LLMs) alignment methods affect the models’ responses to prompt attacks. We selected open source models based on the most common alignment methods, namely, Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Reinforcement Learning with Human Feedback (RLHF). We conducted a systematic analysis using statistical methods to verify how sensitive the Attack Success Rate (ASR) is when we apply variations to prompts designed to elicit inappropriate content from LLMs. Our results show that even small prompt modifications can significantly change the Attack Success Rate (ASR) according to the statistical tests we run, making the models more or less susceptible to types of attack. Critically, our results demonstrate that running existing ‘attack benchmarks’ alone may not be sufficient to elicit all possible vulnerabilities of both models and alignment methods. This paper thus contributes to ongoing efforts on model attack evaluation by means of systematic and statistically-based analyses of the different alignment methods and how sensitive their ASR is to prompt variation.

arXiv:2511.10688v1 Announce Type: new Abstract: As large language models (LLMs) are adopted in an increasingly wide range of applications, user-model interactions have grown in both frequency and scale. Consequently, research has focused on evaluating the robustness of LLMs, an essential quality for real-world tasks. In this paper, we employ simple multi-turn follow-up prompts to evaluate models’ answer changes, model accuracy dynamics across turns with Markov chains, and examine whether linear probes can predict these changes. Our results show significant vulnerabilities in LLM robustness: a simple “Think again” prompt led to an approximate 10% accuracy drop for Gemini 1.5 Flash over nine turns, while combining this prompt with a semantically equivalent reworded question caused a 7.5% drop for Claude 3.5 Haiku. Additionally, we find that model accuracy across turns can be effectively modeled using Markov chains, enabling the prediction of accuracy probabilities over time. This allows for estimation of the model’s stationary (long-run) accuracy, which we find to be on average approximately 8% lower than its first-turn accuracy for Gemini 1.5 Flash. Our results from a model’s hidden states also reveal evidence that linear probes can help predict future answer changes. Together, these results establish stationary accuracy as a principled robustness metric for interactive settings and expose the fragility of models under repeated questioning. Addressing this instability will be essential for deploying LLMs in high-stakes and interactive settings where consistent reasoning is as important as initial accuracy.

In this tutorial, we build an advanced interactive dashboard using Textual, and we explore how terminal-first UI frameworks can feel as expressive and dynamic as modern web dashboards. As we write and run each snippet, we actively construct the interface piece by piece, widgets, layouts, reactive state, and event flows, so we can see how Textual behaves like a live UI engine right inside Google Colab. By the end, we notice how naturally we can blend tables, trees, forms, and progress indicators into a cohesive application that feels fast, clean, and responsive. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser !pip install textual textual-web nest-asyncio from textual.app import App, ComposeResult from textual.containers import Container, Horizontal, Vertical from textual.widgets import ( Header, Footer, Button, DataTable, Static, Input, Label, ProgressBar, Tree, Select ) from textual.reactive import reactive from textual import on from datetime import datetime import random class StatsCard(Static): value = reactive(0) def __init__(self, title: str, *args, **kwargs): super().__init__(*args, **kwargs) self.title = title def compose(self) -> ComposeResult: yield Label(self.title) yield Label(str(self.value), id=”stat-value”) def watch_value(self, new_value: int) -> None: if self.is_mounted: try: self.query_one(“#stat-value”, Label).update(str(new_value)) except Exception: pass We set up the environment and import all the necessary components to build our Textual application. As we define the StatsCard widget, we establish a reusable component that reacts to changes in value and updates itself automatically. We begin to see how Textual’s reactive system lets us create dynamic UI elements with minimal effort. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser class DataDashboard(App): CSS = “”” Screen { background: $surface; } #main-container { height: 100%; padding: 1; } #stats-row { height: auto; margin-bottom: 1; } StatsCard { border: solid $primary; height: 5; padding: 1; margin-right: 1; width: 1fr; } #stat-value { text-style: bold; color: $accent; content-align: center middle; } #control-panel { height: 12; border: solid $secondary; padding: 1; margin-bottom: 1; } #data-section { height: 1fr; } #left-panel { width: 30; border: solid $secondary; padding: 1; margin-right: 1; } DataTable { height: 100%; border: solid $primary; } Input { margin: 1 0; } Button { margin: 1 1 1 0; } ProgressBar { margin: 1 0; } “”” BINDINGS = [ (“d”, “toggle_dark”, “Toggle Dark Mode”), (“q”, “quit”, “Quit”), (“a”, “add_row”, “Add Row”), (“c”, “clear_table”, “Clear Table”), ] total_rows = reactive(0) total_sales = reactive(0) avg_rating = reactive(0.0) We define the DataDashboard class and configure global styles, key bindings, and reactive attributes. We decide how the app should look and behave right from the top, giving us full control over themes and interactivity. This structure helps us create a polished dashboard without writing any HTML or JS. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser def compose(self) -> ComposeResult: yield Header(show_clock=True) with Container(id=”main-container”): with Horizontal(id=”stats-row”): yield StatsCard(“Total Rows”, id=”card-rows”) yield StatsCard(“Total Sales”, id=”card-sales”) yield StatsCard(“Avg Rating”, id=”card-rating”) with Vertical(id=”control-panel”): yield Input(placeholder=”Product Name”, id=”input-name”) yield Select( [(“Electronics”, “electronics”), (“Books”, “books”), (“Clothing”, “clothing”)], prompt=”Select Category”, id=”select-category” ) with Horizontal(): yield Button(“Add Row”, variant=”primary”, id=”btn-add”) yield Button(“Clear Table”, variant=”warning”, id=”btn-clear”) yield Button(“Generate Data”, variant=”success”, id=”btn-generate”) yield ProgressBar(total=100, id=”progress”) with Horizontal(id=”data-section”): with Container(id=”left-panel”): yield Label(“Navigation”) tree = Tree(“Dashboard”) tree.root.expand() products = tree.root.add(“Products”, expand=True) products.add_leaf(“Electronics”) products.add_leaf(“Books”) products.add_leaf(“Clothing”) tree.root.add_leaf(“Reports”) tree.root.add_leaf(“Settings”) yield tree yield DataTable(id=”data-table”) yield Footer() We compose the entire UI layout, arranging containers, cards, form inputs, buttons, a navigation tree, and a data table. As we structure these components, we watch the interface take shape exactly the way we envision it. This snippet lets us design the visual skeleton of the dashboard in a clean, declarative manner. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser def on_mount(self) -> None: table = self.query_one(DataTable) table.add_columns(“ID”, “Product”, “Category”, “Price”, “Sales”, “Rating”) table.cursor_type = “row” self.generate_sample_data(5) self.set_interval(0.1, self.update_progress) def generate_sample_data(self, count: int = 5) -> None: table = self.query_one(DataTable) categories = [“Electronics”, “Books”, “Clothing”] products = { “Electronics”: [“Laptop”, “Phone”, “Tablet”, “Headphones”], “Books”: [“Novel”, “Textbook”, “Magazine”, “Comic”], “Clothing”: [“Shirt”, “Pants”, “Jacket”, “Shoes”] } for _ in range(count): category = random.choice(categories) product = random.choice(products[category]) row_id = self.total_rows + 1 price = round(random.uniform(10, 500), 2) sales = random.randint(1, 100) rating = round(random.uniform(1, 5), 1) table.add_row( str(row_id), product, category, f”${price}”, str(sales), str(rating) ) self.total_rows += 1 self.total_sales += sales self.update_stats() def update_stats(self) -> None: self.query_one(“#card-rows”, StatsCard).value = self.total_rows self.query_one(“#card-sales”, StatsCard).value = self.total_sales if self.total_rows > 0: table = self.query_one(DataTable) total_rating = sum(float(row[5]) for row in table.rows) self.avg_rating = round(total_rating / self.total_rows, 2) self.query_one(“#card-rating”, StatsCard).value = self.avg_rating def update_progress(self) -> None: progress = self.query_one(ProgressBar) progress.advance(1) if progress.progress >= 100: progress.progress = 0 We implement all the logic for generating data, computing statistics, animating progress, and updating cards. We see how quickly we can bind backend logic to frontend components using Textual’s reactive model. This step makes the dashboard feel alive as numbers update instantly and progress bars animate smoothly. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser @on(Button.Pressed, “#btn-add”) def handle_add_button(self) -> None: name_input = self.query_one(“#input-name”, Input) category = self.query_one(“#select-category”, Select).value if name_input.value and category: table = self.query_one(DataTable) row_id = self.total_rows + 1 price = round(random.uniform(10, 500), 2) sales = random.randint(1, 100) rating = round(random.uniform(1, 5), 1) table.add_row( str(row_id), name_input.value, str(category), f”${price}”, str(sales), str(rating) ) self.total_rows += 1 self.total_sales += sales self.update_stats() name_input.value = “” @on(Button.Pressed, “#btn-clear”) def handle_clear_button(self) -> None: table = self.query_one(DataTable) table.clear() self.total_rows = 0 self.total_sales = 0 self.avg_rating = 0 self.update_stats() @on(Button.Pressed, “#btn-generate”) def handle_generate_button(self) -> None: self.generate_sample_data(10) def action_toggle_dark(self) -> None: self.dark = not self.dark def action_add_row(self) -> None: self.handle_add_button() def action_clear_table(self) -> None: self.handle_clear_button() if __name__ == “__main__”: import nest_asyncio nest_asyncio.apply() app = DataDashboard() app.run() We connect UI events to backend actions using button handlers, keyboard shortcuts, and app-level functions. As we run the app, we interact with a fully functional dashboard that responds instantly to every click and command. This snippet completes the application and demonstrates how easily Textual enables us to build dynamic, state-driven UIs. In conclusion, we see the whole dashboard come together in a fully functional, interactive form that runs directly from a notebook environment. We experience firsthand how Textual lets us design terminal UIs with the structure and