Uncategorized

arXiv:2506.17781v1 Announce Type: cross Abstract: Dense embeddings are fundamental to modern machine learning systems, powering Retrieval-Augmented Generation (RAG), information retrieval, and representation learning. While instruction-conditioning has become the dominant approach for embedding specialization, its direct application to low-capacity models imposes fundamental representational constraints that limit the performance gains derived from specialization. In this paper, we analyze these limitations and introduce the Mixture of Task Experts (MoTE) transformer block, which leverages task-specialized parameters trained with Task-Aware Contrastive Learning (tacl) to enhance the model ability to generate specialized embeddings. Empirical results show that MoTE achieves $64%$ higher performance gains in retrieval datasets ($+3.27 rightarrow +5.21$) and $43%$ higher performance gains across all datasets ($+1.81 rightarrow +2.60$). Critically, these gains are achieved without altering instructions, training data, inference time, or number of active parameters.

arXiv:2506.17863v1 Announce Type: new Abstract: Offsite marketing is essential in e-commerce, enabling businesses to reach customers through external platforms and drive traffic to retail websites. However, most current offsite marketing content is overly generic, template-based, and poorly aligned with landing pages, limiting its effectiveness. To address these limitations, we propose MarketingFM, a retrieval-augmented system that integrates multiple data sources to generate keyword-specific ad copy with minimal human intervention. We validate MarketingFM via offline human and automated evaluations and large-scale online A/B tests. In one experiment, keyword-focused ad copy outperformed templates, achieving up to 9% higher CTR, 12% more impressions, and 0.38% lower CPC, demonstrating gains in ad ranking and cost efficiency. Despite these gains, human review of generated ads remains costly. To address this, we propose AutoEval-Main, an automated evaluation system that combines rule-based metrics with LLM-as-a-Judge techniques to ensure alignment with marketing principles. In experiments with large-scale human annotations, AutoEval-Main achieved 89.57% agreement with human reviewers. Building on this, we propose AutoEval-Update, a cost-efficient LLM-human collaborative framework to dynamically refine evaluation prompts and adapt to shifting criteria with minimal human input. By selectively sampling representative ads for human review and using a critic LLM to generate alignment reports, AutoEval-Update improves evaluation consistency while reducing manual effort. Experiments show the critic LLM suggests meaningful refinements, improving LLM-human agreement. Nonetheless, human oversight remains essential for setting thresholds and validating refinements before deployment.

arXiv:2409.00097v3 Announce Type: replace Abstract: Automatic disease diagnosis has become increasingly valuable in clinical practice. The advent of large language models (LLMs) has catalyzed a paradigm shift in artificial intelligence, with growing evidence supporting the efficacy of LLMs in diagnostic tasks. Despite the increasing attention in this field, a holistic view is still lacking. Many critical aspects remain unclear, such as the diseases and clinical data to which LLMs have been applied, the LLM techniques employed, and the evaluation methods used. In this article, we perform a comprehensive review of LLM-based methods for disease diagnosis. Our review examines the existing literature across various dimensions, including disease types and associated clinical specialties, clinical data, LLM techniques, and evaluation methods. Additionally, we offer recommendations for applying and evaluating LLMs for diagnostic tasks. Furthermore, we assess the limitations of current research and discuss future directions. To our knowledge, this is the first comprehensive review for LLM-based disease diagnosis.

arXiv:2506.16370v1 Announce Type: new Abstract: Large Language Models (LLMs) such as GPT-4 produce compelling responses to a wide range of prompts. But their representational capacities are uncertain. Many LLMs have no direct contact with extra-linguistic reality: their inputs, outputs and training data consist solely of text, raising the questions (1) can LLMs represent anything and (2) if so, what? In this paper, I explore what it would take to answer these questions according to a structural-correspondence based account of representation, and make an initial survey of this evidence. I argue that the mere existence of structural correspondences between LLMs and worldly entities is insufficient to ground representation of those entities. However, if these structural correspondences play an appropriate role – they are exploited in a way that explains successful task performance – then they could ground real world contents. This requires overcoming a challenge: the text-boundedness of LLMs appears, on the face of it, to prevent them engaging in the right sorts of tasks.

arXiv:2310.17143v4 Announce Type: replace-cross Abstract: Academic writing is an indispensable yet laborious part of the research enterprise. This Perspective maps out principles and methods for using generative artificial intelligence (AI), specifically large language models (LLMs), to elevate the quality and efficiency of academic writing. We introduce a human-AI collaborative framework that delineates the rationale (why), process (how), and nature (what) of AI engagement in writing. The framework pinpoints both short-term and long-term reasons for engagement and their underlying mechanisms (e.g., cognitive offloading and imaginative stimulation). It reveals the role of AI throughout the writing process, conceptualized through a two-stage model for human-AI collaborative writing, and the nature of AI assistance in writing, represented through a model of writing-assistance types and levels. Building on this framework, we describe effective prompting techniques for incorporating AI into the writing routine (outlining, drafting, and editing) as well as strategies for maintaining rigorous scholarship, adhering to varied journal policies, and avoiding overreliance on AI. Ultimately, the prudent integration of AI into academic writing can ease the communication burden, empower authors, accelerate discovery, and promote diversity in science.

arXiv:2506.16381v1 Announce Type: new Abstract: In modern speech synthesis, paralinguistic information–such as a speaker’s vocal timbre, emotional state, and dynamic prosody–plays a critical role in conveying nuance beyond mere semantics. Traditional Text-to-Speech (TTS) systems rely on fixed style labels or inserting a speech prompt to control these cues, which severely limits flexibility. Recent attempts seek to employ natural-language instructions to modulate paralinguistic features, substantially improving the generalization of instruction-driven TTS models. Although many TTS systems now support customized synthesis via textual description, their actual ability to interpret and execute complex instructions remains largely unexplored. In addition, there is still a shortage of high-quality benchmarks and automated evaluation metrics specifically designed for instruction-based TTS, which hinders accurate assessment and iterative optimization of these models. To address these limitations, we introduce InstructTTSEval, a benchmark for measuring the capability of complex natural-language style control. We introduce three tasks, namely Acoustic-Parameter Specification, Descriptive-Style Directive, and Role-Play, including English and Chinese subsets, each with 1k test cases (6k in total) paired with reference audio. We leverage Gemini as an automatic judge to assess their instruction-following abilities. Our evaluation of accessible instruction-following TTS systems highlights substantial room for further improvement. We anticipate that InstructTTSEval will drive progress toward more powerful, flexible, and accurate instruction-following TTS.

arXiv:2506.16633v1 Announce Type: new Abstract: Multimodal reasoning is a process of understanding, integrating and inferring information across different data modalities. It has recently attracted surging academic attention as a benchmark for Artificial Intelligence (AI). Although there are various tasks for evaluating multimodal reasoning ability, they still have limitations. Lack of reasoning on hierarchical visual clues at different levels of granularity, e.g., local details and global context, is of little discussion, despite its frequent involvement in real scenarios. To bridge the gap, we introduce a novel and challenging task for multimodal reasoning, namely GeoGuess. Given a street view image, the task is to identify its location and provide a detailed explanation. A system that succeeds in GeoGuess should be able to detect tiny visual clues, perceive the broader landscape, and associate with vast geographic knowledge. Therefore, GeoGuess would require the ability to reason between hierarchical visual information and geographic knowledge. In this work, we establish a benchmark for GeoGuess by introducing a specially curated dataset GeoExplain which consists of panoramas-geocoordinates-explanation tuples. Additionally, we present a multimodal and multilevel reasoning method, namely SightSense which can make prediction and generate comprehensive explanation based on hierarchy of visual information and external knowledge. Our analysis and experiments demonstrate their outstanding performance in GeoGuess.

Introduction: Understanding Generalization in Deep Generative Models Deep generative models, including diffusion and flow matching, have shown outstanding performance in synthesizing realistic multi-modal content across images, audio, video, and text. However, the generalization capabilities and underlying mechanisms of these models are challenging in deep generative modeling. The core challenge includes understanding whether generative models truly generalize or simply memorize training data. Current research reveals conflicting evidence: some studies show that large diffusion models memorize individual samples from training sets, while others show clear signs of generalization when trained on large datasets. This contradiction points to a sharp phase transition between memorization and generalization. Existing Literature on Flow Matching and Generalization Mechanisms Existing research includes the utilization of closed-form solutions, studying memorization versus generalization, and characterizing different phases of generating dynamics. Methods like closed-form velocity field regression and a smoothed version of optimal velocity generation have been proposed. Studies on memorization relate the transition to generalization with training dataset size through geometric interpretations, while others focus on stochasticity in target objectives. Temporal regime analysis identifies distinct phases in generative dynamics, which show reliance on dimension and sample numbers. But validation methods depend on backward process stochasticity, which doesn’t apply to flow matching models, leaving significant gaps in understanding. New Findings: Early Trajectory Failures Drive Generalization Researchers from Université Jean Monnet Saint-Etienne and Université Claude Bernard Lyon provide an answer to whether training on noisy or stochastic targets improves flow matching generalization and identify the main sources of generalization. The method reveals that generalization emerges when limited-capacity neural networks fail to approximate the exact velocity field during critical time intervals at early and late phases. The researchers identify that generalization arises mainly early along flow matching trajectories, corresponding to the transition from stochastic to deterministic behaviour. Moreover, they propose a learning algorithm that explicitly regresses against the exact velocity field, showing enhanced generalization capabilities on standard image datasets. Investigating the Sources of Generalization in Flow Matching Researchers investigate the key sources of generalization. First, they challenge target stochasticity assumptions by using closed-form optimal velocity field formulations, showing that after small time values, the weighted average of conditional flow matching targets equals single expectation values. Second, they analyze the approximate quality between learned velocity fields and optimal velocity fields through systematic experiments on subsampled CIFAR-10 datasets ranging from 10 to 10,000 samples. Third, they construct hybrid models using piecewise trajectories governed by optimal velocity fields for early time intervals and learned velocity fields for later intervals, with adjustable threshold parameters to determine critical periods. Empirical Flow Matching: A Learning Algorithm for Deterministic Targets Researchers implement a learning algorithm that regresses against more deterministic targets using closed-form formulas. It compares vanilla conditional flow matching, optimal transport flow matching, and empirical flow matching across CIFAR-10 and CelebA datasets using multiple samples to estimate empirical means. Moreover, evaluation metrics include Fréchet Inception Distance with Inception-V3 and DINOv2 embeddings for a less biased assessment. The computational architecture operates with complexity O(M × |B| × d). Training configurations demonstrate that increasing sample numbers M for empirical mean computation creates less stochastic targets, leading to more stable performance improvements with modest computational overhead when M equals the batch size. Conclusion: Velocity Field Approximation as the Core of Generalization In this paper, researchers challenge the assumption that stochasticity in loss functions drives generalization in flow matching models, clarifying the critical role of exact velocity field approximation instead. While research provides empirical insights into practical learned models, precise characterization of learned velocity fields outside optimal trajectories remains an open challenge, suggesting future work to use architectural inductive biases. The broader implications include concerns about potential misuse of improved generative models for creating deepfakes, privacy violations, and synthetic content generation. So, it is necessary to give careful consideration to ethical applications. Why This Research Matters? This research is significant because it challenges a prevailing assumption in generative modeling—that stochasticity in training objectives is a key driver of generalization in flow matching models. By demonstrating that generalization instead arises from the failure of neural networks to precisely approximate the closed-form velocity field, especially during early trajectory phases, the study reframes our understanding of what enables models to produce novel data. This insight has direct implications for designing more efficient and interpretable generative systems, reducing computational overhead while maintaining or even enhancing generalization. It also informs better training protocols that avoid unnecessary stochasticity, improving reliability and reproducibility in real-world applications. Check out the Paper. All credit for this research goes to the researchers of this project. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. The post Why Generalization in Flow Matching Models Comes from Approximation, Not Stochasticity appeared first on MarkTechPost.