Nachrichten

In this tutorial, we build a complete, production-grade ML experimentation and deployment workflow using MLflow. We start by launching a dedicated MLflow Tracking Server with a structured backend and artifact store, enabling us to track experiments in a scalable, reproducible manner. We then train multiple machine learning models using a nested hyperparameter sweep while automatically logging parameters, metrics, and model artifacts. We enhance the experiment by logging diagnostic visualizations, evaluating the best model using MLflow’s built-in evaluation framework, and storing detailed evaluation results for future analysis. We also deploy the trained model using MLflow’s native serving capabilities and interact with it via a REST API, demonstrating how MLflow bridges the gap between experimentation and real-world model deployment. Copy CodeCopiedUse a different Browser !pip -q install “mlflow>=3.0.0″ scikit-learn pandas numpy matplotlib requests import os import time import json import shutil import socket import signal import subprocess from pathlib import Path import numpy as np import pandas as pd import matplotlib.pyplot as plt import requests from sklearn.datasets import load_breast_cancer from sklearn.model_selection import train_test_split from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression from sklearn.metrics import ( roc_auc_score, accuracy_score, precision_score, recall_score, f1_score, confusion_matrix, ConfusionMatrixDisplay, ) import mlflow import mlflow.sklearn from mlflow.models.signature import infer_signature def _is_port_open(host: str, port: int, timeout_s: float = 0.2) -> bool: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: s.settimeout(timeout_s) return s.connect_ex((host, port)) == 0 def _wait_for_http(url: str, timeout_s: int = 30) -> None: t0 = time.time() last_err = None while time.time() – t0 < timeout_s: try: r = requests.get(url, timeout=1) if r.status_code < 500: return except Exception as e: last_err = e time.sleep(0.5) raise RuntimeError(f”Server not ready at {url}. Last error: {last_err}”) def _safe_kill(proc: subprocess.Popen): if proc is None: return try: proc.terminate() try: proc.wait(timeout=5) except subprocess.TimeoutExpired: proc.kill() except Exception: pass We install all required dependencies and import the complete MLflow, scikit-learn, and system libraries needed for experiment tracking and deployment. We define utility functions that allow us to check port availability, wait for server readiness, and safely terminate background processes. We establish the foundational infrastructure to ensure our MLflow tracking server and model-serving components operate reliably in the Colab environment. Copy CodeCopiedUse a different Browser BASE_DIR = Path(“/content/mlflow_colab_demo”).resolve() BACKEND_DB = BASE_DIR / “mlflow.db” ARTIFACT_ROOT = BASE_DIR / “mlartifacts” os.makedirs(BASE_DIR, exist_ok=True) os.makedirs(ARTIFACT_ROOT, exist_ok=True) HOST = “127.0.0.1” PORT = 5000 TRACKING_URI = f”http://{HOST}:{PORT}” if _is_port_open(HOST, PORT): for p in range(5001, 5015): if not _is_port_open(HOST, p): PORT = p TRACKING_URI = f”http://{HOST}:{PORT}” break print(“Using TRACKING_URI:”, TRACKING_URI) print(“Backend DB:”, BACKEND_DB) print(“Artifact root:”, ARTIFACT_ROOT) server_cmd = [ “mlflow”, “server”, “–host”, HOST, “–port”, str(PORT), “–backend-store-uri”, f”sqlite:///{BACKEND_DB}”, “–default-artifact-root”, str(ARTIFACT_ROOT), ] mlflow_server = subprocess.Popen( server_cmd, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, ) _wait_for_http(TRACKING_URI, timeout_s=45) mlflow.set_tracking_uri(TRACKING_URI) print(“MLflow server is up.”) EXPERIMENT_NAME = “colab-advanced-mlflow-sklearn” mlflow.set_experiment(EXPERIMENT_NAME) We configure the MLflow backend storage and artifact directories to create a structured, persistent experiment-tracking environment. We launch the MLflow Tracking Server with a SQLite database and a local artifact store, enabling full experiment logging and management. We connect our notebook to the running MLflow server and initialize a dedicated experiment that will organize all training runs and associated metadata. Copy CodeCopiedUse a different Browser data = load_breast_cancer(as_frame=True) df = data.frame.copy() target_col = “target” X = df.drop(columns=[target_col]) y = df[target_col].astype(int) mlflow.sklearn.autolog( log_input_examples=False, log_model_signatures=False, silent=True ) C_VALUES = [0.01, 0.1, 1.0, 3.0] SOLVERS = [“liblinear”, “lbfgs”] best = {“auc”: -1.0, “run_id”: None, “params”: None} We load the dataset and prepare the training and testing splits required for machine learning experimentation. We enable MLflow autologging, allowing automatic tracking of parameters, metrics, and model artifacts without manual intervention. We define the hyperparameter search space and initialize the structure to identify and store the best-performing model configuration. Copy CodeCopiedUse a different Browser with mlflow.start_run(run_name=”parent_sweep_run”) as parent_run: mlflow.log_param(“dataset”, “sklearn_breast_cancer”) mlflow.log_param(“n_features”, X_train.shape[1]) mlflow.log_param(“n_train”, X_train.shape[0]) mlflow.log_param(“n_test”, X_test.shape[0]) for C in C_VALUES: for solver in SOLVERS: with mlflow.start_run(run_name=f”child_C={C}_solver={solver}”, nested=True) as child_run: pipe = Pipeline([ (“scaler”, StandardScaler()), (“clf”, LogisticRegression( C=C, solver=solver, penalty=”l2″, max_iter=2000, random_state=42 )) ]) pipe.fit(X_train, y_train) proba = pipe.predict_proba(X_test)[:, 1] pred = (proba >= 0.5).astype(int) auc = roc_auc_score(y_test, proba) acc = accuracy_score(y_test, pred) prec = precision_score(y_test, pred, zero_division=0) rec = recall_score(y_test, pred, zero_division=0) f1 = f1_score(y_test, pred, zero_division=0) mlflow.log_metrics({ “test_auc”: float(auc), “test_accuracy”: float(acc), “test_precision”: float(prec), “test_recall”: float(rec), “test_f1″: float(f1), }) cm = confusion_matrix(y_test, pred) disp = ConfusionMatrixDisplay(cm, display_labels=data.target_names) fig, ax = plt.subplots(figsize=(5, 4)) disp.plot(ax=ax, values_format=”d”) ax.set_title(f”Confusion Matrix (C={C}, solver={solver})”) cm_path = BASE_DIR / “confusion_matrix.png” fig.tight_layout() fig.savefig(cm_path, dpi=140) plt.close(fig) mlflow.log_artifact(str(cm_path), artifact_path=”diagnostics”) if auc > best[“auc”]: best.update({ “auc”: float(auc), “run_id”: child_run.info.run_id, “params”: {“C”: C, “solver”: solver} }) mlflow.log_dict(best, “best_run_summary.json”) print(“Best config:”, best) We perform a nested hyperparameter sweep, training multiple models within a structured parent-child run hierarchy. We compute performance metrics and log them alongside diagnostic artifacts, such as confusion matrices, to enable detailed analysis of experiments. We continuously monitor model performance and update our tracking structure to identify the best configuration across all training runs. Copy CodeCopiedUse a different Browser best_C = best[“params”][“C”] best_solver = best[“params”][“solver”] final_pipe = Pipeline([ (“scaler”, StandardScaler()), (“clf”, LogisticRegression( C=best_C, solver=best_solver, penalty=”l2″, max_iter=2000, random_state=42 )) ]) with mlflow.start_run(run_name=”final_model_run”) as final_run: final_pipe.fit(X_train, y_train) proba = final_pipe.predict_proba(X_test)[:, 1] pred = (proba >= 0.5).astype(int) metrics = { “test_auc”: float(roc_auc_score(y_test, proba)), “test_accuracy”: float(accuracy_score(y_test, pred)), “test_precision”: float(precision_score(y_test, pred, zero_division=0)), “test_recall”: float(recall_score(y_test, pred, zero_division=0)), “test_f1”: float(f1_score(y_test, pred, zero_division=0)), } mlflow.log_metrics(metrics) mlflow.log_params({“C”: best_C, “solver”: best_solver, “model”: “LogisticRegression+StandardScaler”}) input_example = X_test.iloc[:5].copy() signature = infer_signature(input_example, final_pipe.predict_proba(input_example)[:, 1]) model_info = mlflow.sklearn.log_model( sk_model=final_pipe, artifact_path=”model”, signature=signature, input_example=input_example, registered_model_name=None, ) print(“Final run_id:”, final_run.info.run_id) print(“Logged model URI:”, model_info.model_uri) eval_df = X_test.copy() eval_df[“label”] = y_test.values eval_result = mlflow.models.evaluate( model=model_info.model_uri, data=eval_df, targets=”label”, model_type=”classifier”, evaluators=”default”, ) eval_summary = { “metrics”: {k: float(v) if isinstance(v, (int, float, np.floating)) else str(v) for k, v in eval_result.metrics.items()}, “artifacts”: {k: str(v) for k, v in eval_result.artifacts.items()}, } mlflow.log_dict(eval_summary, “evaluation/eval_summary.json”) We train the final model using the best hyperparameters identified during the experiment sweep and log it with a proper signature and input example. We evaluate the model using MLflow’s built-in evaluation framework, which generates detailed metrics and evaluation artifacts. We store the evaluation summary within MLflow, ensuring the final model is fully documented, reproducible, and ready for deployment. Copy CodeCopiedUse a different Browser SERVE_PORT = 6000 if _is_port_open(HOST, SERVE_PORT): for p

As the industry moves from simple Large Language Model (LLM) inference toward autonomous agentic systems, the challenge for devs have shifted. It is no longer just about the model; it is about the environment in which that model operates. A team of researchers from Alibaba released CoPaw, an open-source framework designed to address this by providing a standardized workstation for deploying and managing personal AI agents. CoPaw is built on a technical stack comprising AgentScope, AgentScope Runtime, and ReMe. It functions as a bridge between high-level agent logic and the practical requirements of a personal assistant, such as persistent memory, multi-channel connectivity, and task scheduling. The Architecture: AgentScope and ReMe Integration CoPaw is not a standalone bot but a workstation that orchestrates multiple components to create a cohesive ‘Agentic App.’ The system relies on three primary layers: AgentScope: The underlying framework that handles agent communication and logic. AgentScope Runtime: The execution environment that ensures stable operation and resource management. ReMe (Memory Management): A specialized module that handles both local and cloud-based memory. This allows agents to maintain ‘Long-Term Experience,’ solving the statelessness issue inherent in standard LLM APIs. By leveraging ReMe, CoPaw allows users to control their data privacy while ensuring the agent retains context across different sessions and platforms. This persistent memory is what enables the workstation to adapt to a user’s specific workflows over time. Extensibility via the Skills System A core feature of the CoPaw workstation is its Skill Extension capability. In this framework, a ‘Skill’ is a discrete unit of functionality—essentially a tool that the agent can invoke to interact with the external world. Adding capabilities to CoPaw does not require modifying the core engine. Instead, CoPaw supports a custom skill directory where engineers can drop Python-based functions. These skills follow a standardized specification (influenced by anthropics/skills), allowing the agent to: Perform web scraping (e.g., summarizing Reddit threads or YouTube videos). Interact with local files and desktop environments. Query personal knowledge bases stored within the workstation. Manage calendars and email via natural language. This design allows for the creation of Agentic Apps—complex workflows where the agent uses a combination of built-in skills and scheduled tasks to achieve a goal autonomously. Multi-Channel Connectivity (All-Domain Access) One of the primary technical hurdles in personal AI is deployment across fragmented communication platforms. CoPaw addresses this through its All-Domain Access layer, which standardizes how agents interact with different messaging protocols. Currently, CoPaw supports integration with: Enterprise Platforms: DingTalk and Lark (Feishu). Social/Developer Platforms: Discord, QQ, and iMessage. This multi-channel support means that a developer can initialize a single CoPaw instance and interact with it from any of these endpoints. The workstation handles the translation of messages between the agent’s logic and the specific channel’s API, maintaining a consistent state and memory regardless of where the interaction occurs. Key Takeaways Shift from Model to Workstation: CoPaw moves the focus away from just the Large Language Model (LLM) and toward a structured Workstation architecture. It acts as a middleware layer that orchestrates the AgentScope framework, AgentScope Runtime, and external communication channels to turn raw LLM capabilities into a functional, persistent assistant. Long-Term Memory via ReMe: Unlike standard stateless LLM interactions, CoPaw integrates the ReMe (Memory Management) module. This allows agents to maintain ‘Long-Term Experience’ by storing user preferences and past task data either locally or in the cloud, enabling a personalized evolution of the agent’s behavior over time. Extensible Python-Based ‘Skills’: The framework uses a decoupled Skill Extension system based on the anthropics/skills specification. Developers can extend an agent’s utility by simply adding Python functions to a custom skill directory, allowing the agent to perform specific tasks like web scraping, file manipulation, or API integrations without modifying the core codebase. All-Domain Multi-Channel Access: CoPaw provides a unified interface for cross-platform deployment. A single workstation instance can be connected to enterprise tools (Lark, DingTalk) and social/developer platforms (Discord, QQ, iMessage), allowing the same agent and its memory to be accessed across different environments. Automated Agentic Workflows: By combining Scheduled Tasks with the skills system, CoPaw transitions from reactive chat to proactive automation. Devs can program ‘Agentic Apps’ that perform background operations—such as daily research synthesis or automated repository monitoring—and push results to the user’s preferred communication channel. Check out the Repo here and Website. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Alibaba Team Open-Sources CoPaw: A High-Performance Personal Agent Workstation for Developers to Scale Multi-Channel AI Workflows and Memory appeared first on MarkTechPost.

arXiv:2602.22897v1 Announce Type: cross Abstract: Human intelligence naturally intertwines omni-modal perception — spanning vision, audio, and language — with complex reasoning and tool usage to interact with the world. However, current multi-modal LLMs are primarily confined to bi-modal interactions (e.g., vision-language), lacking the unified cognitive capabilities required for general AI assistants. To bridge this gap, we introduce OmniGAIA, a comprehensive benchmark designed to evaluate omni-modal agents on tasks necessitating deep reasoning and multi-turn tool execution across video, audio, and image modalities. Constructed via a novel omni-modal event graph approach, OmniGAIA synthesizes complex, multi-hop queries derived from real-world data that require cross-modal reasoning and external tool integration. Furthermore, we propose OmniAtlas, a native omni-modal foundation agent under tool-integrated reasoning paradigm with active omni-modal perception. Trained on trajectories synthesized via a hindsight-guided tree exploration strategy and OmniDPO for fine-grained error correction, OmniAtlas effectively enhances the tool-use capabilities of existing open-source models. This work marks a step towards next-generation native omni-modal AI assistants for real-world scenarios.

arXiv:2602.21262v2 Announce Type: replace Abstract: With increasing integration of Large Language Models (LLMs) into areas of high-stakes human decision-making, it is important to understand the risks they introduce as advisors. To be useful advisors, LLMs must sift through large amounts of content, written with both benevolent and malicious intent, and then use this information to convince a user to take a specific action. This involves two social capacities: vigilance (the ability to determine which information to use, and which to discard) and persuasion (synthesizing the available evidence to make a convincing argument). While existing work has investigated these capacities in isolation, there has been little prior investigation of how these capacities may be linked. Here, we use a simple multi-turn puzzle-solving game, Sokoban, to study LLMs’ abilities to persuade and be rationally vigilant towards other LLM agents. We find that puzzle-solving performance, persuasive capability, and vigilance are dissociable capacities in LLMs. Performing well on the game does not automatically mean a model can detect when it is being misled, even if the possibility of deception is explicitly mentioned. However, LLMs do consistently modulate their token use, using fewer tokens to reason when advice is benevolent and more when it is malicious, even if they are still persuaded to take actions leading them to failure. To our knowledge, our work presents the first investigation of the relationship between persuasion, vigilance, and task performance in LLMs, and suggests that monitoring all three independently will be critical for future work in AI safety.

arXiv:2602.22698v1 Announce Type: new Abstract: Leveraging Large Language Models (LLMs) for Knowledge Graph Completion (KGC) is promising but hindered by a fundamental granularity mismatch. LLMs operate on fragmented token sequences, whereas entities are the fundamental units in knowledge graphs (KGs) scenarios. Existing approaches typically constrain predictions to limited candidate sets or align entities with the LLM’s vocabulary by pooling multiple tokens or decomposing entities into fixed-length token sequences, which fail to capture both the semantic meaning of the text and the structural integrity of the graph. To address this, we propose KGT, a novel framework that uses dedicated entity tokens to enable efficient, full-space prediction. Specifically, we first introduce specialized tokenization to construct feature representations at the level of dedicated entity tokens. We then fuse pre-trained structural and textual features into these unified embeddings via a relation-guided gating mechanism, avoiding training from scratch. Finally, we implement decoupled prediction by leveraging independent heads to separate and combine semantic and structural reasoning. Experimental results show that KGT consistently outperforms state-of-the-art methods across multiple benchmarks.

arXiv:2602.22658v1 Announce Type: cross Abstract: Deepfake speech utterances can be forged by replacing one or more words in a bona fide utterance with semantically different words synthesized by speech generative models. While a dedicated synthetic word detector could be developed, we investigate a cost-effective method that fine-tunes a pre-trained Whisper model to detect synthetic words while transcribing the input utterance via next-token prediction. We further investigate using partially vocoded utterances as the fine-tuning data, thereby reducing the cost of data collection. Our experiments demonstrate that, on in-domain test data, the fine-tuned Whisper yields low synthetic-word detection error rates and transcription error rates. On out-of-domain test data with synthetic words produced by unseen speech generative models, the fine-tuned Whisper remains on par with a dedicated ResNet-based detection model; however, the overall performance degradation calls for strategies to improve its generalization capability.

arXiv:2602.23070v1 Announce Type: cross Abstract: Although Automatic Speech Recognition (ASR) in Bengali has seen significant progress, processing long-duration audio and performing robust speaker diarization remain critical research gaps. To address the severe scarcity of joint ASR and diarization resources for this language, we introduce Lipi-Ghor-882, a comprehensive 882-hour multi-speaker Bengali dataset. In this paper, detailing our submission to the DL Sprint 4.0 competition, we systematically evaluate various architectures and approaches for long-form Bengali speech. For ASR, we demonstrate that raw data scaling is ineffective; instead, targeted fine-tuning utilizing perfectly aligned annotations paired with synthetic acoustic degradation (noise and reverberation) emerges as the singular most effective approach. Conversely, for speaker diarization, we observed that global open-source state-of-the-art models (such as Diarizen) performed surprisingly poorly on this complex dataset. Extensive model retraining yielded negligible improvements; instead, strategic, heuristic post-processing of baseline model outputs proved to be the primary driver for increasing accuracy. Ultimately, this work outlines a highly optimized dual pipeline achieving a $sim$0.019 Real-Time Factor (RTF), establishing a practical, empirically backed benchmark for low-resource, long-form speech processing.

arXiv:2602.22225v1 Announce Type: cross Abstract: Retrieval-augmented generation (RAG) has strong potential for producing accurate and factual outputs by combining language models (LMs) with evidence retrieved from large text corpora. However, current pipelines are limited by static chunking and flat retrieval: documents are split into short, predetermined, fixed-size chunks, embeddings are retrieved uniformly, and generation relies on whatever chunks are returned. This design brings challenges, as retrieval quality is highly sensitive to chunk size, often introduces noise from irrelevant or misleading chunks, and scales poorly to large corpora. We present SmartChunk retrieval, a query-adaptive framework for efficient and robust long-document question answering (QA). SmartChunk uses (i) a planner that predicts the optimal chunk abstraction level for each query, and (ii) a lightweight compression module that produces high-level chunk embeddings without repeated summarization. By adapting retrieval granularity on the fly, SmartChunk balances accuracy with efficiency and avoids the drawbacks of fixed strategies. Notably, our planner can reason about chunk abstractions through a novel reinforcement learning scheme, STITCH, which boosts accuracy and generalization. To reflect real-world applications, where users face diverse document types and query styles, we evaluate SmartChunk on five QA benchmarks plus one out-of-domain dataset. Across these evaluations, SmartChunk outperforms state-of-the-art RAG baselines, while reducing cost. Further analysis demonstrates strong scalability with larger corpora and consistent gains on out-of-domain datasets, highlighting its effectiveness as a general framework for adaptive retrieval.

arXiv:2602.22723v1 Announce Type: new Abstract: There is growing recognition that many NLP tasks lack a single ground truth, as human judgments reflect diverse perspectives. To capture this variation, models have been developed to predict full annotation distributions rather than majority labels, while perspectivist models aim to reproduce the interpretations of individual annotators. In this work, we compare these approaches on Implicit Discourse Relation Recognition (IDRR), a highly ambiguous task where disagreement often arises from cognitive complexity rather than ideological bias. Our experiments show that existing annotator-specific models perform poorly in IDRR unless ambiguity is reduced, whereas models trained on label distributions yield more stable predictions. Further analysis indicates that frequent cognitively demanding cases drive inconsistency in human interpretation, posing challenges for perspectivist modeling in IDRR.