AI

arXiv:2508.12265v1 Announce Type: new Abstract: Large Language Models (LLMs) have demonstrated remarkable progress in reasoning across diverse domains. However, effective reasoning in real-world tasks requires adapting the reasoning strategy to the demands of the problem, ranging from fast, intuitive responses to deliberate, step-by-step reasoning and tool-augmented thinking. Drawing inspiration from cognitive psychology, we propose a novel taxonomy of LLM reasoning strategies along two knowledge boundaries: a fast/slow boundary separating intuitive from deliberative processes, and an internal/external boundary distinguishing reasoning grounded in the model’s parameters from reasoning augmented by external tools. We systematically survey recent work on adaptive reasoning in LLMs and categorize methods based on key decision factors. We conclude by highlighting open challenges and future directions toward more adaptive, efficient, and reliable LLMs.

arXiv:2508.11388v1 Announce Type: new Abstract: Concurrent to the rapid progress in the development of neural-network based models in areas like natural language processing and computer vision, the need for creating explanations for the predictions of these black-box models has risen steadily. We propose a new method to generate extractive explanations for predictions made by neural networks, that is based on masking parts of the input which the model does not consider to be indicative of the respective class. The masking is done using gradient-based optimization combined with a new regularization scheme that enforces sufficiency, comprehensiveness and compactness of the generated explanation, three properties that are known to be desirable from the related field of rationale extraction in natural language processing. In this way, we bridge the gap between model interpretability and rationale extraction, thereby proving that the latter of which can be performed without training a specialized model, only on the basis of a trained classifier. We further apply the same method to image inputs and obtain high quality explanations for image classifications, which indicates that the conditions proposed for rationale extraction in natural language processing are more broadly applicable to different input types.

arXiv:2508.11364v1 Announce Type: new Abstract: Automated feedback generation has the potential to enhance students’ learning progress by providing timely and targeted feedback. Moreover, it can assist teachers in optimizing their time, allowing them to focus on more strategic and personalized aspects of teaching. To generate high-quality, information-rich formative feedback, it is essential first to extract relevant indicators, as these serve as the foundation upon which the feedback is constructed. Teachers often employ feedback criteria grids composed of various indicators that they evaluate systematically. This study examines the initial phase of extracting such indicators from students’ submissions of a language learning course using the large language model Llama 3.1. Accordingly, the alignment between indicators generated by the LLM and human ratings across various feedback criteria is investigated. The findings demonstrate statistically significant strong correlations, even in cases involving unanticipated combinations of indicators and criteria. The methodology employed in this paper offers a promising foundation for extracting indicators from students’ submissions using LLMs. Such indicators can potentially be utilized to auto-generate explainable and transparent formative feedback in future research.

arXiv:2508.11386v1 Announce Type: new Abstract: This technical report details a novel approach to combining reasoning and retrieval augmented generation (RAG) within a single, lean language model architecture. While existing RAG systems typically rely on large-scale models and external APIs, our work addresses the increasing demand for performant and privacy-preserving solutions deployable in resource-constrained or secure environments. Building on recent developments in test-time scaling and small-scale reasoning models, we develop a retrieval augmented conversational agent capable of interpreting complex, domain-specific queries using a lightweight backbone model. Our system integrates a dense retriever with fine-tuned Qwen2.5-Instruct models, using synthetic query generation and reasoning traces derived from frontier models (e.g., DeepSeek-R1) over a curated corpus, in this case, the NHS A-to-Z condition pages. We explore the impact of summarisation-based document compression, synthetic data design, and reasoning-aware fine-tuning on model performance. Evaluation against both non-reasoning and general-purpose lean models demonstrates that our domain-specific fine-tuning approach yields substantial gains in answer accuracy and consistency, approaching frontier-level performance while remaining feasible for local deployment. All implementation details and code are publicly released to support reproducibility and adaptation across domains.

arXiv:2508.11281v1 Announce Type: new Abstract: Detecting toxic content using language models is crucial yet challenging. While substantial progress has been made in English, toxicity detection in French remains underdeveloped, primarily due to the lack of culturally relevant, large-scale datasets. In this work, we introduce TOXIFRENCH, a new public benchmark of 53,622 French online comments, constructed via a semi-automated annotation pipeline that reduces manual labeling to only 10% through high-confidence LLM-based pre-annotation and human verification. Then, we benchmark a broad range of models and uncover a counterintuitive insight: Small Language Models (SLMs) outperform many larger models in robustness and generalization under the toxicity detection task. Motivated by this finding, we propose a novel Chain-of-Thought (CoT) fine-tuning strategy using a dynamic weighted loss that progressively emphasizes the model’s final decision, significantly improving faithfulness. Our fine-tuned 4B model achieves state-of-the-art performance, improving its F1 score by 13% over its baseline and outperforming LLMs such as GPT-40 and Gemini-2.5. Further evaluation on a cross-lingual toxicity benchmark demonstrates strong multilingual ability, suggesting that our methodology can be effectively extended to other languages and safety-critical classification tasks.

Artificial Intelligence (AI) has evolved rapidly—especially in how models are deployed and operated in real-world systems. The core function that connects model training to practical applications is “inference”. This article offers a technical deep dive into AI inference as of 2025, covering its distinction from training, latency challenges for modern models, and optimization strategies such as quantization, pruning, and hardware acceleration. Inference vs. Training: The Critical Difference AI model deployment consists of two primary phases: Training is the process where a model learns patterns from massive, labeled datasets, using iterative algorithms (typically, backpropagation on neural networks). This phase is computation-heavy and generally done offline, leveraging accelerators like GPUs. Inference is the model’s “in action” phase—making predictions on new, unseen data. Here, the trained network is fed input, and the output is produced via a forward pass only. Inference happens in production environments, often requiring rapid responses and lower resource use. Aspect Training Inference Purpose Learn patterns, optimize weights Make predictions on new data Computation Heavy, iterative, uses backpropagation Lighter, forward pass only Time Sensitivity Offline, can take hours/days/weeks Real-time or near-real-time Hardware GPUs/TPUs, datacenter-scale CPUs, GPUs, FPGAs, edge devices Inference Latency: Challenges for 2025 Latency—the time from input to output—is one of the top technical challenges in deploying AI, especially large language models (LLMs) and real-time applications (autonomous vehicles, conversational bots, etc.). Key Sources of Latency Computational Complexity: Modern architectures—like transformers—have quadratic computational costs due to self-attention. (e.g., O ( n 2 d ) O(n 2 d) for sequence length n n and embedding dimension d d). Memory Bandwidth: Large models (with billions of parameters) require tremendous data movement, which often bottlenecks on memory speed and system I/O. Network Overhead: For cloud inference, network latency and bandwidth become critical—especially for distributed and edge deployments. Predictable vs. Unpredictable Latency: Some delays can be designed for (e.g., batch inference), while others—hardware contention, network jitter—cause unpredictable delays. Real-World Impact Latency directly affects user experience (voice assistants, fraud detection), system safety (driverless cars), and operational cost (cloud compute resources). As models grow, optimizing latency becomes increasingly complex and essential. Quantization: Lightening the Load Quantization reduces model size and computational requirements by lowering the numerical precision (e.g., converting 32-bit floats to 8-bit integers). How It Works: Quantization replaces high-precision parameters with lower-precision approximations, decreasing memory and compute needs. Types: Uniform/Non-uniform quantization Post-Training Quantization (PTQ) Quantization-Aware Training (QAT) Trade-offs: While quantization can dramatically speed up inference, it might slightly reduce model accuracy—careful application maintains performance within acceptable bounds. LLMs & Edge Devices: Especially valuable for LLMs and battery-powered devices, allowing for fast, low-cost inference. Pruning: Model Simplification Pruning is the process of removing redundant or non-essential model components—such as neural network weights or decision tree branches. Techniques: L1 Regularization: Penalizes large weights, shrinking less useful ones to zero. Magnitude Pruning: Removes lowest-magnitude weights or neurons. Taylor Expansion: Estimates the least impactful weights and prunes them. SVM Pruning: Reduces support vectors to simplify decision boundaries. Benefits: Lower memory. Faster inference. Reduced overfitting. Easier model deployment to resource-constrained environments. Risks: Aggressive pruning may degrade accuracy—balancing efficiency and accuracy is key. Hardware Acceleration: Speeding Up Inference Specialized hardware is transforming AI inference in 2025: GPUs: Offer massive parallelism, ideal for matrix and vector operations. NPUs (Neural Processing Units): Custom processors, optimized for neural network workloads. FPGAs (Field-Programmable Gate Arrays): Configurable chips for targeted, low-latency inference in embedded/edge devices. ASICs (Application-Specific Integrated Circuits): Purpose-built for highest efficiency and speed in large-scale deployments. Trends: Real-time, Energy-efficient Processing: Essential for autonomous systems, mobile devices, and IoT. Versatile Deployment: Hardware accelerators now span cloud servers to edge devices. Reduced Cost and Energy: Emerging accelerator architectures slash operational costs and carbon footprints. Here are the top 9 AI inference providers in 2025: Together AI Specializes in scalable LLM deployments, offering fast inference APIs and unique multi-model routing for hybrid cloud setups. Fireworks AI Renowned for ultra-fast multi-modal inference and privacy-oriented deployments, leveraging optimized hardware and proprietary engines for low latency. Hyperbolic Delivers serverless inference for generative AI, integrating automated scaling and cost optimization for high-volume workloads. Replicate Focuses on model hosting and deployment, allowing developers to run and share AI models rapidly in production with easy integrations. Hugging Face The go-to platform for transformer and LLM inference, providing robust APIs, customization options, and community-backed open-source models. Groq Known for custom Language Processing Unit (LPU) hardware that achieves unprecedented low-latency and high-throughput inference speeds for large models. DeepInfra Offers a dedicated cloud for high-performance inference, catering especially to startups and enterprise teams with customizable infrastructure. OpenRouter Aggregates multiple LLM engines, providing dynamic model routing and cost transparency for enterprise-grade inference orchestration. Lepton (Acquired by NVIDIA) Specializes in compliance-focused, secure AI inference with real-time monitoring and scalable edge/cloud deployment options. Conclusion Inference is where AI meets the real world, turning data-driven learning into actionable predictions. Its technical challenges—latency, resource constraints—are being met by innovations in quantization, pruning, and hardware acceleration. As AI models scale and diversify, mastering inference efficiency is the frontier for competitive, impactful deployment in 2025. Whether deploying conversational LLMs, real-time computer vision systems, or on-device diagnostics, understanding and optimizing inference will be central for technologists and enterprises aiming to lead in the AI era. The post What is AI Inference? A Technical Deep Dive and Top 9 AI Inference Providers (2025 Edition) appeared first on MarkTechPost.

In this tutorial, we implement an advanced data pipeline using Dagster. We set up a custom CSV-based IOManager to persist assets, define partitioned daily data generation, and process synthetic sales data through cleaning, feature engineering, and model training. Along the way, we add a data-quality asset check to validate nulls, ranges, and categorical values, and we ensure that metadata and outputs are stored in a structured way. The focus throughout is on hands-on implementation, showing how to integrate raw data ingestion, transformations, quality checks, and machine learning into a single reproducible workflow. Copy CodeCopiedUse a different Browser import sys, subprocess, json, os subprocess.check_call([sys.executable, “-m”, “pip”, “install”, “-q”, “dagster”, “pandas”, “scikit-learn”]) import numpy as np, pandas as pd from pathlib import Path from dagster import ( asset, AssetCheckResult, asset_check, Definitions, materialize, Output, DailyPartitionsDefinition, IOManager, io_manager ) from sklearn.linear_model import LinearRegression BASE = Path(“/content/dagstore”); BASE.mkdir(parents=True, exist_ok=True) START = “2025-08-01″ We begin by installing the required libraries, Dagster, Pandas, and scikit-learn, so that we have the full toolset available in Colab. We then import essential modules, set up NumPy and Pandas for data handling, and define a base directory along with a start date to organize our pipeline outputs. Copy CodeCopiedUse a different Browser class CSVIOManager(IOManager): def __init__(self, base: Path): self.base = base def _path(self, key, ext): return self.base / f”{‘_’.join(key.path)}.{ext}” def handle_output(self, context, obj): if isinstance(obj, pd.DataFrame): p = self._path(context.asset_key, “csv”); obj.to_csv(p, index=False) context.log.info(f”Saved {context.asset_key} -> {p}”) else: p = self._path(context.asset_key, “json”); p.write_text(json.dumps(obj, indent=2)) context.log.info(f”Saved {context.asset_key} -> {p}”) def load_input(self, context): k = context.upstream_output.asset_key; p = self._path(k, “csv”) df = pd.read_csv(p); context.log.info(f”Loaded {k} <- {p} ({len(df)} rows)”); return df @io_manager def csv_io_manager(_): return CSVIOManager(BASE) daily = DailyPartitionsDefinition(start_date=START) We define a custom CSVIOManager to save asset outputs as CSV or JSON files and reload them when needed. We then register it with Dagster as csv_io_manager and set up a daily partitioning scheme so that our pipeline can process data for each date independently. Copy CodeCopiedUse a different Browser @asset(partitions_def=daily, description=”Synthetic raw sales with noise & occasional nulls.”) def raw_sales(context) -> Output[pd.DataFrame]: rng = np.random.default_rng(42) n = 200; day = context.partition_key x = rng.normal(100, 20, n); promo = rng.integers(0, 2, n); noise = rng.normal(0, 10, n) sales = 2.5 * x + 30 * promo + noise + 50 x[rng.choice(n, size=max(1, n // 50), replace=False)] = np.nan df = pd.DataFrame({“date”: day, “units”: x, “promo”: promo, “sales”: sales}) meta = {“rows”: n, “null_units”: int(df[“units”].isna().sum()), “head”: df.head().to_markdown()} return Output(df, metadata=meta) @asset(description=”Clean nulls, clip outliers for robust downstream modeling.”) def clean_sales(context, raw_sales: pd.DataFrame) -> Output[pd.DataFrame]: df = raw_sales.dropna(subset=[“units”]).copy() lo, hi = df[“units”].quantile([0.01, 0.99]); df[“units”] = df[“units”].clip(lo, hi) meta = {“rows”: len(df), “units_min”: float(df.units.min()), “units_max”: float(df.units.max())} return Output(df, metadata=meta) @asset(description=”Feature engineering: interactions & standardized columns.”) def features(context, clean_sales: pd.DataFrame) -> Output[pd.DataFrame]: df = clean_sales.copy() df[“units_sq”] = df[“units”] ** 2; df[“units_promo”] = df[“units”] * df[“promo”] for c in [“units”, “units_sq”, “units_promo”]: mu, sigma = df[c].mean(), df[c].std(ddof=0) or 1.0 df[f”z_{c}”] = (df[c] – mu) / sigma return Output(df, metadata={“rows”: len(df), “cols”: list(df.columns)}) We create three core assets for the pipeline. First, raw_sales generates synthetic daily sales data with noise and occasional missing values, simulating real-world imperfections. Next, clean_sales removes nulls and clips outliers to stabilize the dataset, while logging metadata about ranges and row counts. Finally, features perform feature engineering by adding interaction and standardized variables, preparing the data for downstream modeling. Copy CodeCopiedUse a different Browser @asset_check(asset=clean_sales, description=”No nulls; promo in {0,1}; units within clipped bounds.”) def clean_sales_quality(clean_sales: pd.DataFrame) -> AssetCheckResult: nulls = int(clean_sales.isna().sum().sum()) promo_ok = bool(set(clean_sales[“promo”].unique()).issubset({0, 1})) units_ok = bool(clean_sales[“units”].between(clean_sales[“units”].min(), clean_sales[“units”].max()).all()) passed = bool((nulls == 0) and promo_ok and units_ok) return AssetCheckResult( passed=passed, metadata={“nulls”: nulls, “promo_ok”: promo_ok, “units_ok”: units_ok}, ) @asset(description=”Train a tiny linear regressor; emit R^2 and coefficients.”) def tiny_model_metrics(context, features: pd.DataFrame) -> dict: X = features[[“z_units”, “z_units_sq”, “z_units_promo”, “promo”]].values y = features[“sales”].values model = LinearRegression().fit(X, y) return {“r2_train”: float(model.score(X, y)), **{n: float(c) for n, c in zip([“z_units”,”z_units_sq”,”z_units_promo”,”promo”], model.coef_)}} We strengthen the pipeline with validation and modeling. The clean_sales_quality asset check enforces data integrity by verifying that there are no nulls, the promo field only has 0/1 values, and the cleaned units remain within valid bounds. After that, tiny_model_metrics trains a simple linear regression on the engineered features and outputs key metrics like training and learned coefficients, giving us a lightweight but complete modeling step within the Dagster workflow. Copy CodeCopiedUse a different Browser defs = Definitions( assets=[raw_sales, clean_sales, features, tiny_model_metrics, clean_sales_quality], resources={“io_manager”: csv_io_manager} ) if __name__ == “__main__”: run_day = os.environ.get(“RUN_DATE”) or START print(“Materializing everything for:”, run_day) result = materialize( [raw_sales, clean_sales, features, tiny_model_metrics, clean_sales_quality], partition_key=run_day, resources={“io_manager”: csv_io_manager}, ) print(“Run success:”, result.success) for fname in [“raw_sales.csv”,”clean_sales.csv”,”features.csv”,”tiny_model_metrics.json”]: f = BASE / fname if f.exists(): print(fname, “->”, f.stat().st_size, “bytes”) if fname.endswith(“.json”): print(“Metrics:”, json.loads(f.read_text())) We register our assets and the IO manager in Definitions, then materialize the entire DAG for a selected partition key in one run. We persist CSV/JSON artifacts to /content/dagstore and print a quick success flag, plus saved file sizes and model metrics for immediate verification. In conclusion, we materialize all assets and checks in a single Dagster run, confirm data quality, and train a regression model whose metrics are stored for inspection. We keep the pipeline modular, with each asset producing and persisting its outputs in CSV or JSON, and ensure compatibility by explicitly converting metadata values to supported types. This tutorial demonstrates how we can combine partitioning, asset definitions, and checks to build a technically robust and reproducible workflow, giving us a practical framework to extend toward more complex real-world pipelines. Check out the FULL CODES here. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Partner with Marktechpost for Promotion The post A Coding Guide to Build and Validate End-to-End Partitioned Data Pipelines in Dagster with Machine Learning Integration appeared first on MarkTechPost.