Committee

In this tutorial, we build an Agentic Data and Infrastructure Strategy system using the lightweight Qwen2.5-0.5B-Instruct model for efficient execution. We begin by creating a flexible LLM agent framework and then develop specialized agents that handle different layers of data management, from ingestion and quality analysis to infrastructure optimization. We integrate these agents into an orchestrator that coordinates their interactions, ensuring smooth multi-agent collaboration across the data pipeline. Through hands-on examples like e-commerce and IoT pipelines, we explore how autonomous decision-making can streamline complex data operations. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser !pip install -q transformers torch accelerate datasets huggingface_hub import torch from transformers import AutoModelForCausalLM, AutoTokenizer import json, time from typing import List, Dict, Any from dataclasses import dataclass from datetime import datetime import pandas as pd class LightweightLLMAgent: def __init__(self, role: str, model_name: str = “Qwen/Qwen2.5-0.5B-Instruct”): self.role = role self.model_name = model_name self.device = “cuda” if torch.cuda.is_available() else “cpu” print(f”Loading {model_name} for {role} agent on {self.device}…”) self.tokenizer = AutoTokenizer.from_pretrained(model_name) self.model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.float16 if self.device == “cuda” else torch.float32, device_map=”auto” ) self.conversation_history = [] def generate_response(self, prompt: str, max_tokens: int = 150) -> str: messages = [ {“role”: “system”, “content”: f”You are a {self.role} agent in a data infrastructure system.”}, {“role”: “user”, “content”: prompt} ] text = self.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) model_inputs = self.tokenizer([text], return_tensors=”pt”).to(self.device) with torch.no_grad(): generated_ids = self.model.generate( model_inputs.input_ids, max_new_tokens=max_tokens, temperature=0.7, do_sample=True, top_p=0.95 ) generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)] response = self.tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0] self.conversation_history.append({“prompt”: prompt, “response”: response}) return response We start by setting up the lightweight LLM agent infrastructure using the Qwen2.5-0.5B-Instruct model. We load the model and tokenizer, and define a base agent class capable of handling contextual conversations and generating intelligent responses. This forms the core foundation upon which our specialized agents operate efficiently within Colab. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser class DataIngestionAgent(LightweightLLMAgent): def __init__(self): super().__init__(role=”Data Ingestion Specialist”) def analyze_data_source(self, source_info: Dict) -> Dict: prompt = f”””Analyze this data source and provide ingestion strategy: Source Type: {source_info.get(‘type’, ‘unknown’)} Volume: {source_info.get(‘volume’, ‘unknown’)} Frequency: {source_info.get(‘frequency’, ‘unknown’)} Provide a brief strategy focusing on: 1) Ingestion method, 2) Key considerations.””” strategy = self.generate_response(prompt, max_tokens=100) return {“source”: source_info, “strategy”: strategy, “timestamp”: datetime.now().isoformat()} class DataQualityAgent(LightweightLLMAgent): def __init__(self): super().__init__(role=”Data Quality Analyst”) def assess_data_quality(self, data_sample: Dict) -> Dict: prompt = f”””Assess data quality for this sample: Completeness: {data_sample.get(‘completeness’, ‘N/A’)}% Consistency: {data_sample.get(‘consistency’, ‘N/A’)}% Issues Found: {data_sample.get(‘issues’, 0)} Provide brief quality assessment and top 2 recommendations.””” assessment = self.generate_response(prompt, max_tokens=100) return {“assessment”: assessment, “severity”: self._calculate_severity(data_sample), “timestamp”: datetime.now().isoformat()} def _calculate_severity(self, data_sample: Dict) -> str: completeness = data_sample.get(‘completeness’, 100) consistency = data_sample.get(‘consistency’, 100) avg_score = (completeness + consistency) / 2 if avg_score >= 90: return “LOW” elif avg_score >= 70: return “MEDIUM” else: return “HIGH” We design the Data Ingestion and Data Quality agents to focus on structured analysis of data pipelines. We let the ingestion agent determine the best approach to data flow, while the quality agent evaluates data completeness, consistency, and issues to provide actionable insights. Together, they establish the first two layers of autonomous data management. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser class InfrastructureOptimizationAgent(LightweightLLMAgent): def __init__(self): super().__init__(role=”Infrastructure Optimization Specialist”) def optimize_resources(self, metrics: Dict) -> Dict: prompt = f”””Analyze infrastructure metrics and suggest optimizations: CPU Usage: {metrics.get(‘cpu_usage’, 0)}% Memory Usage: {metrics.get(‘memory_usage’, 0)}% Storage: {metrics.get(‘storage_used’, 0)}GB / {metrics.get(‘storage_total’, 0)}GB Query Latency: {metrics.get(‘query_latency’, 0)}ms Provide 2 optimization recommendations.””” recommendations = self.generate_response(prompt, max_tokens=100) return {“current_metrics”: metrics, “recommendations”: recommendations, “priority”: self._calculate_priority(metrics), “timestamp”: datetime.now().isoformat()} def _calculate_priority(self, metrics: Dict) -> str: cpu = metrics.get(‘cpu_usage’, 0) memory = metrics.get(‘memory_usage’, 0) if cpu > 85 or memory > 85: return “CRITICAL” elif cpu > 70 or memory > 70: return “HIGH” else: return “NORMAL” We develop the Infrastructure Optimization Agent to continuously analyze key metrics like CPU, memory, and storage utilization. We use it to generate intelligent optimization suggestions, helping us maintain high performance and resource efficiency. This agent ensures that our infrastructure remains responsive and scalable during data operations. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser class AgenticDataOrchestrator: def __init__(self): print(“n” + “=”*70) print(“Initializing Agentic Data Infrastructure System”) print(“=”*70 + “n”) self.ingestion_agent = DataIngestionAgent() self.quality_agent = DataQualityAgent() self.optimization_agent = InfrastructureOptimizationAgent() self.execution_log = [] def process_data_pipeline(self, pipeline_config: Dict) -> Dict: results = {“pipeline_id”: pipeline_config.get(“id”, “unknown”), “start_time”: datetime.now().isoformat(), “stages”: []} print(“n[Stage 1] Data Ingestion Analysis”) ingestion_result = self.ingestion_agent.analyze_data_source(pipeline_config.get(“source”, {})) print(f”Strategy: {ingestion_result[‘strategy’][:150]}…”) results[“stages”].append({“stage”: “ingestion”, “result”: ingestion_result}) print(“n[Stage 2] Data Quality Assessment”) quality_result = self.quality_agent.assess_data_quality(pipeline_config.get(“quality_metrics”, {})) print(f”Assessment: {quality_result[‘assessment’][:150]}…”) print(f”Severity: {quality_result[‘severity’]}”) results[“stages”].append({“stage”: “quality”, “result”: quality_result}) print(“n[Stage 3] Infrastructure Optimization”) optimization_result = self.optimization_agent.optimize_resources(pipeline_config.get(“infrastructure_metrics”, {})) print(f”Recommendations: {optimization_result[‘recommendations’][:150]}…”) print(f”Priority: {optimization_result[‘priority’]}”) results[“stages”].append({“stage”: “optimization”, “result”: optimization_result}) results[“end_time”] = datetime.now().isoformat() results[“status”] = “completed” self.execution_log.append(results) return results def generate_summary_report(self) -> pd.DataFrame: if not self.execution_log: return pd.DataFrame() summary_data = [] for log in self.execution_log: summary_data.append({“Pipeline ID”: log[“pipeline_id”], “Start Time”: log[“start_time”], “Status”: log[“status”], “Stages Completed”: len(log[“stages”])}) return pd.DataFrame(summary_data) We built an Agentic Data Orchestrator to coordinate all specialized agents under a unified workflow. We use it to manage end-to-end pipeline execution, triggering ingestion, quality checks, and optimization sequentially. By doing this, we bring structure, collaboration, and automation to the entire multi-agent system. Check out the FULL CODES here. Copy CodeCopiedUse a different Browser def main(): orchestrator = AgenticDataOrchestrator() print(“n” + “=”*70) print(“EXAMPLE 1: E-commerce Data Pipeline”) print(“=”*70) ecommerce_pipeline = { “id”: “ecommerce_pipeline_001”, “source”: {“type”: “REST API”, “volume”: “10GB/day”, “frequency”: “real-time”}, “quality_metrics”: {“completeness”: 87, “consistency”: 92, “issues”: 15}, “infrastructure_metrics”: {“cpu_usage”: 78, “memory_usage”: 82, “storage_used”: 450, “storage_total”: 1000, “query_latency”: 250} } result1 = orchestrator.process_data_pipeline(ecommerce_pipeline) print(“nn” + “=”*70) print(“EXAMPLE 2: IoT Sensor Data Pipeline”) print(“=”*70) iot_pipeline = { “id”: “iot_pipeline_002”, “source”: {“type”: “Message Queue (Kafka)”, “volume”: “50GB/day”, “frequency”: “streaming”}, “quality_metrics”: {“completeness”: 95, “consistency”: 88, “issues”: 8}, “infrastructure_metrics”: {“cpu_usage”: 65, “memory_usage”: 71, “storage_used”: 780, “storage_total”: 2000, “query_latency”: 180} } result2 = orchestrator.process_data_pipeline(iot_pipeline) print(“nn” + “=”*70) print(“EXECUTION SUMMARY REPORT”) print(“=”*70 + “n”) summary_df = orchestrator.generate_summary_report() print(summary_df.to_string(index=False)) print(“n” + “=”*70) print(“Tutorial Complete!”) print(“=”*70) print(“nKey Concepts Demonstrated:”) print(“✓ Lightweight LLM agent architecture”) print(“✓ Specialized agents for different data tasks”) print(“✓ Multi-agent orchestration”) print(“✓ Infrastructure monitoring and optimization”) print(“✓ Autonomous decision-making in data pipelines”) if __name__ == “__main__”: main() We demonstrate our complete

arXiv:2509.12760v2 Announce Type: replace-cross Abstract: We introduce the Similarity-Distance-Magnitude (SDM) activation function, a more robust and interpretable formulation of the standard softmax activation function, adding Similarity (i.e., correctly predicted depth-matches into training) awareness and Distance-to-training-distribution awareness to the existing output Magnitude (i.e., decision-boundary) awareness, and enabling interpretability-by-exemplar via dense matching. We further introduce the SDM estimator, based on a data-driven partitioning of the class-wise empirical CDFs via the SDM activation, to control the class- and prediction-conditional accuracy among selective classifications. When used as the final-layer activation over pre-trained language models for selective classification, the SDM estimator is more robust to co-variate shifts and out-of-distribution inputs than existing calibration methods using softmax activations, while remaining informative over in-distribution data.

arXiv:2510.04226v4 Announce Type: replace Abstract: Large language models (LLMs) tend to generate lexically, semantically, and stylistically homogenous texts. This poses a risk of knowledge collapse, where homogenous LLMs mediate a shrinking in the range of accessible information over time. Existing works on homogenization are limited by a focus on closed-ended multiple-choice setups or fuzzy semantic features, and do not look at trends across time and cultural contexts. To overcome this, we present a new methodology to measure epistemic diversity, i.e., variation in real-world claims in LLM outputs, which we use to perform a broad empirical study of LLM knowledge collapse. We test 27 LLMs, 155 topics covering 12 countries, and 200 prompt variations sourced from real user chats. For the topics in our study, we show that while newer models tend to generate more diverse claims, nearly all models are less epistemically diverse than a basic web search. We find that model size has a negative impact on epistemic diversity, while retrieval-augmented generation (RAG) has a positive impact, though the improvement from RAG varies by the cultural context. Finally, compared to a traditional knowledge source (Wikipedia), we find that country-specific claims reflect the English language more than the local one, highlighting a gap in epistemic representation

arXiv:2510.24789v1 Announce Type: new Abstract: Watermarking has been proposed as a lightweight mechanism to identify AI-generated text, with schemes typically relying on perturbations to token distributions. While prior work shows that paraphrasing can weaken such signals, these attacks remain partially detectable or degrade text quality. We demonstrate that cross-lingual summarization attacks (CLSA) — translation to a pivot language followed by summarization and optional back-translation — constitute a qualitatively stronger attack vector. By forcing a semantic bottleneck across languages, CLSA systematically destroys token-level statistical biases while preserving semantic fidelity. In experiments across multiple watermarking schemes (KGW, SIR, XSIR, Unigram) and five languages (Amharic, Chinese, Hindi, Spanish, Swahili), we show that CLSA reduces watermark detection accuracy more effectively than monolingual paraphrase at similar quality levels. Our results highlight an underexplored vulnerability that challenges the practicality of watermarking for provenance or regulation. We argue that robust provenance solutions must move beyond distributional watermarking and incorporate cryptographic or model-attestation approaches. On 300 held-out samples per language, CLSA consistently drives detection toward chance while preserving task utility. Concretely, for XSIR (explicitly designed for cross-lingual robustness), AUROC with paraphrasing is $0.827$, with Cross-Lingual Watermark Removal Attacks (CWRA) [He et al., 2024] using Chinese as the pivot, it is $0.823$, whereas CLSA drives it down to $0.53$ (near chance). Results highlight a practical, low-cost removal pathway that crosses languages and compresses content without visible artifacts.

arXiv:2510.25536v1 Announce Type: new Abstract: Large Language Models (LLMs) are exhibiting emergent human-like abilities and are increasingly envisioned as the foundation for simulating an individual’s communication style, behavioral tendencies, and personality traits. However, current evaluations of LLM-based persona simulation remain limited: most rely on synthetic dialogues, lack systematic frameworks, and lack analysis of the capability requirement. To address these limitations, we introduce TwinVoice, a comprehensive benchmark for assessing persona simulation across diverse real-world contexts. TwinVoice encompasses three dimensions: Social Persona (public social interactions), Interpersonal Persona (private dialogues), and Narrative Persona (role-based expression). It further decomposes the evaluation of LLM performance into six fundamental capabilities, including opinion consistency, memory recall, logical reasoning, lexical fidelity, persona tone, and syntactic style. Experimental results reveal that while advanced models achieve moderate accuracy in persona simulation, they still fall short of capabilities such as syntactic style and memory recall. Consequently, the average performance achieved by LLMs remains considerably below the human baseline.

arXiv:2510.24762v1 Announce Type: new Abstract: We introduce Falcon, a cross-domain Chinese text-to-SQL benchmark grounded in an enterprise-compatible dialect (MaxCompute/Hive). It contains 600 Chinese questions over 28 databases; 77% require multi-table reasoning and over half touch more than four tables. Each example is annotated along SQL-computation features and Chinese semantics. For evaluation, we release a robust execution comparator and an automated evaluation pipeline, under which all current state-of-the-art large-scale models (including Deepseek) achieve accuracies of at most 50%. Major errors originate from two sources: (1) schema linking in large enterprise landscapes – hundreds of tables, denormalized fields, ambiguous column names, implicit foreign-key relations and domain-specific synonyms that make correct join/column selection difficult; and (2) mapping concise, colloquial Chinese into the exact operators and predicates required for analytics – e.g., choosing the correct aggregation and group-by keys, expressing time windows and granularities, applying unit conversions, handling NULLs and data-quality rules, and formulating nested or windowed subqueries. Falcon therefore targets Chinese-specific semantics and enterprise dialects (abbreviations, business jargon, fuzzy entity references) and provides a reproducible middle ground before full production deployment by using realistic enterprise schemas, query templates, an execution comparator, and an automated evaluation pipeline for end-to-end validation.

arXiv:2510.25117v1 Announce Type: new Abstract: The advancement of Large Language Models (LLMs) has revolutionized natural language processing, yet their training on massive corpora poses significant risks, including the memorization of sensitive personal data, copyrighted material, and knowledge that could facilitate malicious activities. To mitigate these issues and align with legal and ethical standards such as the “right to be forgotten”, machine unlearning has emerged as a critical technique to selectively erase specific knowledge from LLMs without compromising their overall performance. This survey provides a systematic review of over 180 papers on LLM unlearning published since 2021, focusing exclusively on large-scale generative models. Distinct from prior surveys, we introduce novel taxonomies for both unlearning methods and evaluations. We clearly categorize methods into training-time, post-training, and inference-time based on the training stage at which unlearning is applied. For evaluations, we not only systematically compile existing datasets and metrics but also critically analyze their advantages, disadvantages, and applicability, providing practical guidance to the research community. In addition, we discuss key challenges and promising future research directions. Our comprehensive overview aims to inform and guide the ongoing development of secure and reliable LLMs.

arXiv:2509.24958v2 Announce Type: replace Abstract: An effective physician should possess a combination of empathy, expertise, patience, and clear communication when treating a patient. Recent advances have successfully endowed AI doctors with expert diagnostic skills, particularly the ability to actively seek information through inquiry. However, other essential qualities of a good doctor remain overlooked. To bridge this gap, we present MAQuE(Medical Agent Questioning Evaluation), the largest-ever benchmark for the automatic and comprehensive evaluation of medical multi-turn questioning. It features 3,000 realistically simulated patient agents that exhibit diverse linguistic patterns, cognitive limitations, emotional responses, and tendencies for passive disclosure. We also introduce a multi-faceted evaluation framework, covering task success, inquiry proficiency, dialogue competence, inquiry efficiency, and patient experience. Experiments on different LLMs reveal substantial challenges across the evaluation aspects. Even state-of-the-art models show significant room for improvement in their inquiry capabilities. These models are highly sensitive to variations in realistic patient behavior, which considerably impacts diagnostic accuracy. Furthermore, our fine-grained metrics expose trade-offs between different evaluation perspectives, highlighting the challenge of balancing performance and practicality in real-world clinical settings.

arXiv:2405.07883v2 Announce Type: replace Abstract: Language models (LMs) are bound to their tokenizer, which maps raw text to a sequence of vocabulary items (tokens). This restricts their flexibility: for example, LMs trained primarily on English may still perform well in other natural and programming languages, but have vastly decreased efficiency due to their English-centric tokenizer. To mitigate this, we should be able to swap the original LM tokenizer with an arbitrary one, on the fly, without degrading performance. Hence, in this work we define a new problem: Zero-Shot Tokenizer Transfer (ZeTT). The challenge at the core of ZeTT is finding embeddings for the tokens in the vocabulary of the new tokenizer. Since prior heuristics for initializing embeddings often perform at chance level in a ZeTT setting, we propose a new solution: we train a hypernetwork taking a tokenizer as input and predicting the corresponding embeddings. We empirically demonstrate that the hypernetwork generalizes to new tokenizers both with encoder (e.g., XLM-R) and decoder LLMs (e.g., Mistral-7B). Our method comes close to the original models’ performance in cross-lingual and coding tasks while markedly reducing the length of the tokenized sequence. We also find that the remaining gap can be quickly closed by continued training on less than 1B tokens. Finally, we show that a ZeTT hypernetwork trained for a base (L)LM can also be applied to fine-tuned variants without extra training. Overall, our results make substantial strides toward detaching LMs from their tokenizer.