YouZum

Uncategorized

AI, Committee, Notizie, Uncategorized

LlamaIndex Releases LiteParse: A CLI and TypeScript-Native Library for Spatial PDF Parsing in AI Agent Workflows

In the current landscape of Retrieval-Augmented Generation (RAG), the primary bottleneck for developers is no longer the large language model (LLM) itself, but the data ingestion pipeline. For software developers, converting complex PDFs into a format that an LLM can reason over remains a high-latency, often expensive task. LlamaIndex has recently introduced LiteParse, an open-source, local-first document parsing library designed to address these friction points. Unlike many existing tools that rely on cloud-based APIs or heavy Python-based OCR libraries, LiteParse is a TypeScript-native solution built to run entirely on a user’s local machine. It serves as a ‘fast-mode’ alternative to the company’s managed LlamaParse service, prioritizing speed, privacy, and spatial accuracy for agentic workflows. The Technical Pivot: TypeScript and Spatial Text The most significant technical distinction of LiteParse is its architecture. While the majority of the AI ecosystem is built on Python, LiteParse is written in TypeScript (TS) and runs on Node.js. It utilizes PDF.js (specifically pdf.js-extract) for text extraction and Tesseract.js for local optical character recognition (OCR). By opting for a TypeScript-native stack, LlamaIndex team ensures that LiteParse has zero Python dependencies, making it easier to integrate into modern web-based or edge-computing environments. It is available as both a command-line interface (CLI) and a library, allowing developers to process documents at scale without the overhead of a Python runtime. The library’s core logic stands on Spatial Text Parsing. Most traditional parsers attempt to convert documents into Markdown. However, Markdown conversion often fails when dealing with multi-column layouts or nested tables, leading to a loss of context. LiteParse avoids this by projecting text onto a spatial grid. It preserves the original layout of the page using indentation and white space, allowing the LLM to use its internal spatial reasoning capabilities to ‘read’ the document as it appeared on the page. Solving the Table Problem Through Layout Preservation A recurring challenge for AI devs is extracting tabular data. Conventional methods involve complex heuristics to identify cells and rows, which frequently result in garbled text when the table structure is non-standard. LiteParse takes what the developers call a ‘beautifully lazy’ approach to tables. Rather than attempting to reconstruct a formal table object or a Markdown grid, it maintains the horizontal and vertical alignment of the text. Because modern LLMs are trained on vast amounts of ASCII art and formatted text files, they are often more capable of interpreting a spatially accurate text block than a poorly reconstructed Markdown table. This method reduces the computational cost of parsing while maintaining the relational integrity of the data for the LLM. Agentic Features: Screenshots and JSON Metadata LiteParse is specifically optimized for AI agents. In an agentic RAG workflow, an agent might need to verify the visual context of a document if the text extraction is ambiguous. To facilitate this, LiteParse includes a feature to generate page-level screenshots during the parsing process. When a document is processed, LiteParse can output: Spatial Text: The layout-preserved text version of the document. Screenshots: Image files for each page, allowing multimodal models (like GPT-4o or Claude 3.5 Sonnet) to visually inspect charts, diagrams, or complex formatting. JSON Metadata: Structured data containing page numbers and file paths, which helps agents maintain a clear ‘chain of custody’ for the information they retrieve. This multi-modal output allows engineers to build more robust agents that can switch between reading text for speed and viewing images for high-fidelity visual reasoning. Implementation and Integration LiteParse is designed to be a drop-in component within the LlamaIndex ecosystem. For developers already using VectorStoreIndex or IngestionPipeline, LiteParse provides a local alternative for the document loading stage. The tool can be installed via npm and offers a straightforward CLI: Copy CodeCopiedUse a different Browser npx @llamaindex/liteparse <path-to-pdf> –outputDir ./output This command processes the PDF and populates the output directory with the spatial text files and, if configured, the page screenshots. Key Takeaways TypeScript-Native Architecture: LiteParse is built on Node.js using PDF.js and Tesseract.js, operating with zero Python dependencies. This makes it a high-speed, lightweight alternative for developers working outside the traditional Python AI stack. Spatial Over Markdown: Instead of error-prone Markdown conversion, LiteParse uses Spatial Text Parsing. It preserves the document’s original layout through precise indentation and whitespace, leveraging an LLM’s natural ability to interpret visual structure and ASCII-style tables. Built for Multimodal Agents: To support agentic workflows, LiteParse generates page-level screenshots alongside text. This allows multimodal agents to ‘see’ and reason over complex elements like diagrams or charts that are difficult to capture in plain text. Local-First Privacy: All processing, including OCR, occurs on the local CPU. This eliminates the need for third-party API calls, significantly reducing latency and ensuring sensitive data never leaves the local security perimeter. Seamless Developer Experience: Designed for rapid deployment, LiteParse can be installed via npm and used as a CLI or library. It integrates directly into the LlamaIndex ecosystem, providing a ‘fast-mode’ ingestion path for production RAG pipelines. Check out Repo and Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post LlamaIndex Releases LiteParse: A CLI and TypeScript-Native Library for Spatial PDF Parsing in AI Agent Workflows appeared first on MarkTechPost.

LlamaIndex Releases LiteParse: A CLI and TypeScript-Native Library for Spatial PDF Parsing in AI Agent Workflows Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

Adaptive Decoding via Test-Time Policy Learning for Self-Improving Generation

arXiv:2603.18428v1 Announce Type: new Abstract: Decoding strategies largely determine the quality of Large Language Model (LLM) outputs, yet widely used heuristics such as greedy or fixed temperature/top-p decoding are static and often task-agnostic, leading to suboptimal or inconsistent generation quality across domains that demand stylistic or structural flexibility. We introduce a reinforcement learning-based decoder sampler that treats decoding as sequential decision-making and learns a lightweight policy to adjust sampling parameters at test-time while keeping LLM weights frozen. We evaluated summarization datasets including BookSum, arXiv, and WikiHow using Granite-3.3-2B and Qwen-2.5-0.5B. Our policy sampler consistently outperforms greedy and static baselines, achieving relative gains of up to +88% (BookSum, Granite) and +79% (WikiHow, Qwen). Reward ablations show that overlap-only objectives underperform compared to composite rewards, while structured shaping terms (length, coverage, repetition, completeness) enable stable and sustained improvements. These findings highlight reinforcement learning as a practical mechanism for test-time adaptation in decoding, enabling domain-aware and user-controllable generation without retraining large models.

Adaptive Decoding via Test-Time Policy Learning for Self-Improving Generation Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

Mind-altering substances are (still) falling short in clinical trials

This week I want to look at where we are with psychedelics, the mind-altering substances that have somehow made the leap from counterculture to major focus of clinical research. Compounds like psilocybin—which is found in magic mushrooms—are being explored for all sorts of health applications, including treatments for depression, PTSD, addiction, and even obesity. Over the last decade, we’ve seen scientific interest in these drugs explode. But most clinical trials of psychedelics have been small and plagued by challenges. And a lot of the trial results have been underwhelming or inconclusive. Two studies out earlier this week demonstrate just how difficult it is to study these drugs. And to my mind, they also show just how overhyped these substances have become. To some in the field, the hype is not necessarily a bad thing. Let me explain. The two new studies both focus on the effectiveness of psilocybin in treating depression. And they both attempt to account for one of the biggest challenges in trialing psychedelics: what scientists call “blinding.” The best way to test the effectiveness of a new drug is to perform a randomized controlled trial. In these studies, some volunteers receive the drug while others get a placebo. For a fair comparison, the volunteers shouldn’t know whether they’re getting the drug or placebo. That is almost impossible to do with psychedelics. Almost anyone can tell whether they’ve taken a dose of psilocybin or a dummy pill. The hallucinations are a dead giveaway. Still, the authors behind the two new studies have tried to overcome this challenge. In one, a team based in Germany gave 144 volunteers with treatment-resistant depression either a high or low dose of psilocybin or an “active” placebo, which has its own physical (but not hallucinatory) effects, along with psychotherapy. In their trial, neither the volunteers nor the investigators knew who was getting the drug. The volunteers who got psilocybin did show some improvement—but it was not significantly any better than the improvement experienced by those who took the placebo. And while those who took psilocybin did have a bigger reduction in their symptoms six weeks later, “the divergence between [the two results] renders the findings inconclusive,” the authors write. Not great news so far. The authors of the second study took a different approach. Balázs Szigeti at UCSF and his colleagues instead looked at what are known as “open label” studies of both psychedelics and traditional antidepressants. In those studies, the volunteers knew when they were getting a psychedelic—but they also knew when they were getting an antidepressant. The team assessed 24 such trials to find that … psychedelics were no more effective than traditional antidepressants. Sad trombone. “When I set up the study, I wanted to be a really cool psychedelic scientist to show that even if you consider this blinding problem, psychedelics are so much better than traditional antidepressants,” says Szigeti. “But unfortunately, the data came out the other way around.” His study highlights another problem, too. In trials of traditional antidepressant drugs, the placebo effect is pretty strong. Depressive symptoms are often measured using a scale, and in trials, antidepressant drugs typically lower symptoms by around 10 points on that scale. Placebos can lower symptoms by around eight points. When a drug regulator looks at those results, the takeaway is that the antidepressant drug lowers symptoms by an additional two points on the scale, relative to a placebo. But with psychedelics, the difference between active drug and placebo is much greater. That’s partly because people who get the psychedelic drug know they’re getting it and are expecting the drug to improve their symptoms, says David Owens, emeritus professor of clinical psychiatry at the University of Edinburgh, UK. But it’s also partly because of the effect on those who know they’re not getting it. It’s pretty obvious when you’re getting a placebo, says Szigeti, and it can be disappointing. Scientists have long recognized the “nocebo” effect as placebo’s “evil twin”—essentially, when you expect to feel worse, you will. The disappointment of getting a placebo is slightly different, and Szigeti calls it the “knowcebo effect.” “It’s kind of like a negative psychedelic effect, because you have figured out that you’re taking the placebo,” he says. This phenomenon can distort the results of psychedelic drug trials. While a placebo in a traditional antidepressant drug trial improves symptoms by eight points, placebos in psychedelic trials improve symptoms by a mere four points, says Szigeti. If the active drug similarly improves symptoms by around 10 points, that makes it look as though the psychedelic is improving symptoms by around six points compared with a placebo. It “gives the illusion” of a huge effect, says Szigeti. So why have those smaller trials of the past received so much attention? Many have been published in high-end journals, accompanied by breathless press releases and media coverage. Even the inconclusive ones. I’ve often thought that those studies might not have seen the light of day if they’d been investigating any other drug. “Yeah, nobody would care,” Szigeti agrees. It’s partly because people who work in mental health are so desperate for new treatments, says Owens. There has been little innovation in the last 40 years or so, since the advent of selective serotonin reuptake inhibitors. “Psychiatry is hemmed in with old theories … and we don’t need another SSRI for depression,” he says. But it’s also because psychedelics are inherently fascinating, says Szigeti. “Psychedelics are cool,” he says. “Culturally, they are exciting.” I’ve often worried that psychedelics are overhyped—that people might get the mistaken impression they are cure-alls for mental-health disorders. I’ve worried that vulnerable people might be harmed by self-experimentation. Szigeti takes a different view. Given how effective we know the placebo effect can be, maybe hype isn’t a totally bad thing, he says. “The placebo response is the expectation of a benefit,” he says. “The better response patients are expecting, the better they’re going to get.” Tempering the hype might end up making those drugs less effective, he

Mind-altering substances are (still) falling short in clinical trials Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation

arXiv:2603.12522v1 Announce Type: new Abstract: As large language models (LLMs) are deployed widely, detecting and understanding bias in their outputs is critical. We present LLM BiasScope, a web application for side-by-side comparison of LLM outputs with real-time bias analysis. The system supports multiple providers (Google Gemini, DeepSeek, MiniMax, Mistral, Meituan, Meta Llama) and enables researchers and practitioners to compare models on the same prompts while analyzing bias patterns. LLM BiasScope uses a two-stage bias detection pipeline: sentence-level bias detection followed by bias type classification for biased sentences. The analysis runs automatically on both user prompts and model responses, providing statistics, visualizations, and detailed breakdowns of bias types. The interface displays two models side-by-side with synchronized streaming responses, per-model bias summaries, and a comparison view highlighting differences in bias distributions. The system is built on Next.js with React, integrates Hugging Face inference endpoints for bias detection, and uses the Vercel AI SDK for multi-provider LLM access. Features include real-time streaming, export to JSON/PDF, and interactive visualizations (bar charts, radar charts) for bias analysis. LLM BiasScope is available as an open-source web application, providing a practical tool for bias evaluation and comparative analysis of LLM behaviour.

LLM BiasScope: A Real-Time Bias Analysis Platform for Comparative LLM Evaluation Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

From XAI to Stories: A Factorial Study of LLM-Generated Explanation Quality

arXiv:2601.02224v2 Announce Type: replace Abstract: Explainable AI (XAI) methods like SHAP and LIME produce numerical feature attributions that remain inaccessible to non expert users. Prior work has shown that Large Language Models (LLMs) can transform these outputs into natural language explanations (NLEs), but it remains unclear which factors contribute to high-quality explanations. We present a systematic factorial study investigating how Forecasting model choice, XAI method, LLM selection, and prompting strategy affect NLE quality. Our design spans four models (XGBoost (XGB), Random Forest (RF), Multilayer Perceptron (MLP), and SARIMAX – comparing black-box Machine-Learning (ML) against classical time-series approaches), three XAI conditions (SHAP, LIME, and a no-XAI baseline), three LLMs (GPT-4o, Llama-3-8B, DeepSeek-R1), and eight prompting strategies. Using G-Eval, an LLM-as-a-judge evaluation method, with dual LLM judges and four evaluation criteria, we evaluate 660 explanations for time-series forecasting. Our results suggest that: (1) XAI provides only small improvements over no-XAI baselines, and only for expert audiences; (2) LLM choice dominates all other factors, with DeepSeek-R1 outperforming GPT-4o and Llama-3; (3) we observe an interpretability paradox: in our setting, SARIMAX yielded lower NLE quality than ML models despite higher prediction accuracy; (4) zero-shot prompting is competitive with self-consistency at 7-times lower cost; and (5) chain-of-thought hurts rather than helps.

From XAI to Stories: A Factorial Study of LLM-Generated Explanation Quality Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

IBM AI Releases Granite 4.0 1B Speech as a Compact Multilingual Speech Model for Edge AI and Translation Pipelines

IBM has released Granite 4.0 1B Speech, a compact speech-language model designed for multilingual automatic speech recognition (ASR) and bidirectional automatic speech translation (AST). The release targets enterprise and edge-style speech deployments where memory footprint, latency, and compute efficiency matter as much as raw benchmark quality. What Changed in Granite 4.0 1B Speech At the center of the release is a straightforward design goal: reduce model size without dropping the core capabilities expected from a modern multilingual speech system. Granite 4.0 1B Speech has half the number of parameters of granite-speech-3.3-2b, while adding Japanese ASR, keyword list biasing, and improved English transcription accuracy. The model provides faster inference through better encoder training and speculative decoding. That makes the release less about pushing model scale upward and more about tightening the efficiency-quality tradeoff for practical deployment. Training Approach and Modality Alignment Granite-4.0-1b-speech is a compact and efficient speech-language model trained for multilingual ASR and bidirectional AST. The training mix includes public ASR and AST corpora along with synthetic data used to support Japanese ASR, keyword-biased ASR, and speech translation. This is an important detail for devs because it shows IBM’s team did not build a separate closed speech stack from scratch; it adapted a Granite 4.0 base language model into a speech-capable model through alignment and multimodal training. Language Coverage and Intended Use The supported language set includes English, French, German, Spanish, Portuguese, and Japanese. IBM positions the model for speech-to-text and speech translation to and from English for those languages. It also support for English-to-Italian and English-to-Mandarin translation scenarios. The model is released under the Apache 2.0 license, which makes it more straightforward for teams evaluating open deployment options compared with speech systems that carry commercial restrictions or API-only access patterns. Two-Pass Design and Pipeline Structure IBM’s Granite Speech Team describes the Granite Speech family as using a two-pass design. In that setup, an initial call transcribes audio into text, and any downstream language-model reasoning over the transcript requires a second explicit call to the Granite language model. That differs from integrated architectures that combine speech and language generation into a single pass. For developers, this matters because it affects orchestration. A transcription pipeline built around Granite Speech is modular by design: speech recognition comes first, and language-level post-processing is a separate step. Benchmark Results and Efficiency Positioning Granite 4.0 1B Speech recently ranked #1 on the OpenASR leaderboard. The Open ASR leaderboard row states with an Average WER of 5.52 and RTFx of 280.02, alongside dataset-specific WER values such as 1.42 on LibriSpeech Clean, 2.85 on LibriSpeech Other, 3.89 on SPGISpeech, 3.1 on Tedlium, and 5.84 on VoxPopuli. Deployment Details For deployment, Granite 4.0 1B Speech is supported natively in transformers>=4.52.1 and can be served through vLLM, giving teams both standard Python inference and API-style serving options. IBM’s reference transformers flow uses AutoModelForSpeechSeq2Seq and AutoProcessor, expects mono 16 kHz audio, and formats requests by prepending <|audio|> to the user prompt; keyword biasing can be added directly in the prompt as Keywords: <kw1>, <kw2> …. For lower-resource environments, IBM’s vLLM example sets max_model_len=2048 and limit_mm_per_prompt={“audio”: 1}, while online serving can be exposed through vllm serve with an OpenAI-compatible API interface. Key Takeaways Granite 4.0 1B Speech is a compact speech-language model for multilingual ASR and bidirectional AST. The model has half the parameters of granite-speech-3.3-2b while improving deployment efficiency. The release adds Japanese ASR and keyword list biasing for more targeted transcription workflows. It supports deployment through Transformers, vLLM, and mlx-audio, including Apple Silicon environments. The model is positioned for resource-constrained devices where latency, memory, and compute cost are critical. Check out Model Page, Repo and Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post IBM AI Releases Granite 4.0 1B Speech as a Compact Multilingual Speech Model for Edge AI and Translation Pipelines appeared first on MarkTechPost.

IBM AI Releases Granite 4.0 1B Speech as a Compact Multilingual Speech Model for Edge AI and Translation Pipelines Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

Moonshot AI Releases 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔 to Replace Fixed Residual Mixing with Depth-Wise Attention for Better Scaling in Transformers

Residual connections are one of the least questioned parts of modern Transformer design. In PreNorm architectures, each layer adds its output back into a running hidden state, which keeps optimization stable and allows deep models to train. Moonshot AI researchers argue that this standard mechanism also introduces a structural problem: all prior layer outputs are accumulated with fixed unit weights, which causes hidden-state magnitude to grow with depth and progressively weakens the contribution of any single layer. The research team proposes Attention Residuals (AttnRes) as a drop-in replacement for standard residual accumulation. Instead of forcing every layer to consume the same uniformly mixed residual stream, AttnRes lets each layer aggregate earlier representations using softmax attention over depth. The input to layer (l) is a weighted sum of the token embedding and previous layer outputs, where the weights are computed over prior depth positions rather than over sequence positions. The core idea is simple: if attention improved sequence modeling by replacing fixed recurrence over time, a similar idea can be applied to the depth dimension of a network. https://github.com/MoonshotAI/Attention-Residuals/tree/master?tab=readme-ov-file Why Standard Residuals Become a Bottleneck The research team identified three issues with standard residual accumulation. First, there is no selective access: all layers receive the same aggregated state even though attention layers and feed-forward or MoE layers may benefit from different mixtures of earlier information. Second, there is irreversible loss: once information is blended into a single residual stream, later layers cannot selectively recover specific earlier representations. Third, there is output growth: deeper layers tend to produce larger outputs to remain influential inside an ever-growing accumulated state, which can destabilize training. This is the research team’s main framing: standard residuals behave like a compressed recurrence over layers. AttnRes replaces that fixed recurrence with explicit attention over previous layer outputs. Full AttnRes: Attention Over All Previous Layers In Full AttnRes, each layer computes attention weights over all preceding depth sources. The default design does not use an input-conditioned query. Instead, each layer has a learned layer-specific pseudo-query vector wl ∈ Rd, while keys and values come from the token embedding and previous layer outputs after RMSNorm. The RMSNorm step is important because it prevents large-magnitude layer outputs from dominating the depth-wise attention weights. Full AttnRes is straightforward, but it increases cost. Per token, it requires O(L2 d) arithmetic and (O(Ld)) memory to store layer outputs. In standard training this memory largely overlaps with activations already needed for backpropagation, but under activation re-computation and pipeline parallelism the overhead becomes more significant because those earlier outputs must remain available and may need to be transmitted across stages. Block AttnRes: A Practical Variant for Large Models To make the method usable at scale, Moonshot AI research team introduces Block AttnRes. Instead of attending over every earlier layer output, the model partitions layers into N blocks. Within each block, outputs are accumulated into a single block representation, and attention is applied only over those block-level representations plus the token embedding. This reduces memory and communication overhead from O(Ld) to O(Nd). The research team describes cache-based pipeline communication and a two-phase computation strategy that make Block AttnRes practical in distributed training and inference. This results in less than 4% training overhead under pipeline parallelism, while the repository reports less than 2% inference latency overhead on typical workloads. Scaling Results The research team evaluates five model sizes and compares three variants at each size: a PreNorm baseline, Full AttnRes, and Block AttnRes with about eight blocks. All variants within each size group share the same hyperparameters chosen under the baseline, which the research team note makes the comparison conservative. The fitted scaling laws are reported as: Baseline: L = 1.891 x C-0.057Block AttnRes: L = 1.870 x C-0.058Full AttnRes: L = 1.865 x C-0.057 The practical implication is that AttnRes achieves lower validation loss across the tested compute range, and the Block AttnRes matches the loss of a baseline trained with about 1.25× more compute. Integration into Kimi Linear Moonshot AI also integrates AttnRes into Kimi Linear, its MoE architecture with 48B total parameters and 3B activated parameters, and pre-trains it on 1.4T tokens. According to the research paper, AttnRes mitigates PreNorm dilution by keeping output magnitudes more bounded across depth and distributing gradients more uniformly across layers. Another implementation detail is that all pseudo-query vectors are initialized to zero so the initial attention weights are uniform across source layers, effectively reducing AttnRes to equal-weight averaging at the start of training and avoiding early instability. On downstream evaluation, the reported gains are consistent across all listed tasks. It reports improvements from 73.5 to 74.6 on MMLU, 36.9 to 44.4 on GPQA-Diamond, 76.3 to 78.0 on BBH, 53.5 to 57.1 on Math, 59.1 to 62.2 on HumanEval, 72.0 to 73.9 on MBPP, 82.0 to 82.9 on CMMLU, and 79.6 to 82.5 on C-Eval. Key Takeaways Attention Residuals replaces fixed residual accumulation with softmax attention over previous layers. The default AttnRes design uses a learned layer-specific pseudo-query, not an input-conditioned query. Block AttnRes makes the method practical by reducing depth-wise memory and communication from O(Ld) to O(Nd). Moonshot research teamreports lower scaling loss than the PreNorm baseline, with Block AttnRes matching about 1.25× more baseline compute. In Kimi Linear, AttnRes improves results across reasoning, coding, and evaluation benchmarks with limited overhead. Check out Paper and Repo. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Moonshot AI Releases 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔 to Replace Fixed Residual Mixing with Depth-Wise Attention for Better Scaling in Transformers appeared first on MarkTechPost.

Moonshot AI Releases 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔 to Replace Fixed Residual Mixing with Depth-Wise Attention for Better Scaling in Transformers Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

How to Build Type-Safe, Schema-Constrained, and Function-Driven LLM Pipelines Using Outlines and Pydantic

In this tutorial, we build a workflow using Outlines to generate structured and type-safe outputs from language models. We work with typed constraints like Literal, int, and bool, and design prompt templates using outlines.Template, and enforce strict schema validation with Pydantic models. We also implement robust JSON recovery and a function-calling style that generates validated arguments and executes Python functions safely. Throughout the tutorial, we focus on reliability, constraint enforcement, and production-grade structured generation. Copy CodeCopiedUse a different Browser import os, sys, subprocess, json, textwrap, re subprocess.check_call([sys.executable, “-m”, “pip”, “install”, “-q”, “outlines”, “transformers”, “accelerate”, “sentencepiece”, “pydantic”]) import torch import outlines from transformers import AutoTokenizer, AutoModelForCausalLM from typing import Literal, List, Union, Annotated from pydantic import BaseModel, Field from enum import Enum print(“Torch:”, torch.__version__) print(“CUDA available:”, torch.cuda.is_available()) print(“Outlines:”, getattr(outlines, “__version__”, “unknown”)) device = “cuda” if torch.cuda.is_available() else “cpu” print(“Using device:”, device) MODEL_NAME = “HuggingFaceTB/SmolLM2-135M-Instruct” tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME, use_fast=True) hf_model = AutoModelForCausalLM.from_pretrained( MODEL_NAME, torch_dtype=torch.float16 if device == “cuda” else torch.float32, device_map=”auto” if device == “cuda” else None, ) if device == “cpu”: hf_model = hf_model.to(device) model = outlines.from_transformers(hf_model, tokenizer) def build_chat(user_text: str, system_text: str = “You are a precise assistant. Follow instructions exactly.”) -> str: try: msgs = [{“role”: “system”, “content”: system_text}, {“role”: “user”, “content”: user_text}] return tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True) except Exception: return f”{system_text}nnUser: {user_text}nAssistant:” def banner(title: str): print(“n” + “=” * 90) print(title) print(“=” * 90) We install all required dependencies and initialize the Outlines pipeline with a lightweight instruct model. We configure device handling so that the system automatically switches between CPU and GPU based on availability. We also build reusable helper functions for chat formatting and clean section banners to structure the workflow. Copy CodeCopiedUse a different Browser def extract_json_object(s: str) -> str: s = s.strip() start = s.find(“{“) if start == -1: return s depth = 0 in_str = False esc = False for i in range(start, len(s)): ch = s[i] if in_str: if esc: esc = False elif ch == “\”: esc = True elif ch == ‘”‘: in_str = False else: if ch == ‘”‘: in_str = True elif ch == “{“: depth += 1 elif ch == “}”: depth -= 1 if depth == 0: return s[start:i + 1] return s[start:] def json_repair_minimal(bad: str) -> str: bad = bad.strip() last = bad.rfind(“}”) if last != -1: return bad[:last + 1] return bad def safe_validate(model_cls, raw_text: str): raw = extract_json_object(raw_text) try: return model_cls.model_validate_json(raw) except Exception: raw2 = json_repair_minimal(raw) return model_cls.model_validate_json(raw2) banner(“2) Typed outputs (Literal / int / bool)”) sentiment = model( build_chat(“Analyze the sentiment: ‘This product completely changed my life!’. Return one label only.”), Literal[“Positive”, “Negative”, “Neutral”], max_new_tokens=8, ) print(“Sentiment:”, sentiment) bp = model(build_chat(“What’s the boiling point of water in Celsius? Return integer only.”), int, max_new_tokens=8) print(“Boiling point (int):”, bp) prime = model(build_chat(“Is 29 a prime number? Return true or false only.”), bool, max_new_tokens=6) print(“Is prime (bool):”, prime) We implement robust JSON extraction and minimal repair utilities to safely recover structured outputs from imperfect generations. We then demonstrate strongly typed generation using Literal, int, and bool, ensuring the model returns values that are strictly constrained. We validate how Outlines enforces deterministic type-safe outputs directly at generation time. Copy CodeCopiedUse a different Browser banner(“3) Prompt templating (outlines.Template)”) tmpl = outlines.Template.from_string(textwrap.dedent(“”” <|system|> You are a strict classifier. Return ONLY one label. <|user|> Classify sentiment of this text: {{ text }} Labels: Positive, Negative, Neutral <|assistant|> “””).strip()) templated = model(tmpl(text=”The food was cold but the staff were kind.”), Literal[“Positive”,”Negative”,”Neutral”], max_new_tokens=8) print(“Template sentiment:”, templated) We use outlines.Template to build structured prompt templates with strict output control. We dynamically inject user input into the template while preserving role formatting and classification constraints. We demonstrate how templating improves reusability and ensures consistent, constrained responses. Copy CodeCopiedUse a different Browser banner(“4) Pydantic structured output (advanced constraints)”) class TicketPriority(str, Enum): low = “low” medium = “medium” high = “high” urgent = “urgent” IPv4 = Annotated[str, Field(pattern=r”^((25[0-5]|2[0-4]d|[01]?dd?).){3}(25[0-5]|2[0-4]d|[01]?dd?)$”)] ISODate = Annotated[str, Field(pattern=r”^d{4}-d{2}-d{2}$”)] class ServiceTicket(BaseModel): priority: TicketPriority category: Literal[“billing”, “login”, “bug”, “feature_request”, “other”] requires_manager: bool summary: str = Field(min_length=10, max_length=220) action_items: List[str] = Field(min_length=1, max_length=6) class NetworkIncident(BaseModel): affected_service: Literal[“dns”, “vpn”, “api”, “website”, “database”] severity: Literal[“sev1”, “sev2”, “sev3”] public_ip: IPv4 start_date: ISODate mitigation: List[str] = Field(min_length=2, max_length=6) email = “”” Subject: URGENT – Cannot access my account after payment I paid for the premium plan 3 hours ago and still can’t access any features. I have a client presentation in an hour and need the analytics dashboard. Please fix this immediately or refund my payment. “””.strip() ticket_text = model( build_chat( “Extract a ServiceTicket from this message.n” “Return JSON ONLY matching the ServiceTicket schema.n” “Action items must be distinct.nnMESSAGE:n” + email ), ServiceTicket, max_new_tokens=240, ) ticket = safe_validate(ServiceTicket, ticket_text) if isinstance(ticket_text, str) else ticket_text print(“ServiceTicket JSON:n”, ticket.model_dump_json(indent=2)) We define advanced Pydantic schemas with enums, regex constraints, field limits, and structured lists. We extract a complex ServiceTicket object from raw email text and validate it using schema-driven decoding. We also apply safe validation logic to handle edge cases and ensure robustness at production scale. Copy CodeCopiedUse a different Browser banner(“5) Function-calling style (schema -> args -> call)”) class AddArgs(BaseModel): a: int = Field(ge=-1000, le=1000) b: int = Field(ge=-1000, le=1000) def add(a: int, b: int) -> int: return a + b args_text = model( build_chat(“Return JSON ONLY with two integers a and b. Make a odd and b even.”), AddArgs, max_new_tokens=80, ) args = safe_validate(AddArgs, args_text) if isinstance(args_text, str) else args_text print(“Args:”, args.model_dump()) print(“add(a,b) =”, add(args.a, args.b)) print(“Tip: For best speed and fewer truncations, switch Colab Runtime → GPU.”) We implement a function-calling style workflow by generating structured arguments that conform to a defined schema. We validate the generated arguments, then safely execute a Python function with those validated inputs. We demonstrate how schema-first generation enables controlled tool invocation and reliable LLM-driven computation. In conclusion, we implemented a fully structured generation pipeline using Outlines with strong typing, schema validation, and controlled decoding. We demonstrated how to move from simple typed outputs to advanced Pydantic-based extraction and function-style execution patterns. We also built resilience through JSON salvage and validation mechanisms, making

How to Build Type-Safe, Schema-Constrained, and Function-Driven LLM Pipelines Using Outlines and Pydantic Leggi l'articolo »

AI, Committee, Notizie, Uncategorized

Zhipu AI Introduces GLM-OCR: A 0.9B Multimodal OCR Model for Document Parsing and Key Information Extraction (KIE)

Why Document OCR Still Remains a Hard Engineering Problem? What does it take to make OCR useful for real documents instead of clean demo images? And can a compact multimodal model handle parsing, tables, formulas, and structured extraction without turning inference into a resource bonfire? That is the problem targeted by GLM-OCR, introduced by researchers from Zhipu AI and Tsinghua University. The research team presents GLM-OCR as a 0.9B-parameter compact multimodal model for document understanding. It combines a 0.4B CogViT visual encoder, a lightweight cross-modal connector, and a 0.5B GLM language decoder. The stated goal is to balance document recognition quality with lower latency and lower computational cost than larger multimodal systems. Traditional OCR systems are often good at plain text transcription, but they struggle when documents contain mixed layouts, tables, formulas, code blocks, seals, and structured fields. Recent multimodal large language models improve document understanding, but the research team argue that their size and standard autoregressive decoding make them expensive for edge deployment and large-scale production. GLM-OCR is positioned as a smaller system built for these deployment constraints rather than as a general-purpose vision-language model adapted to OCR as an afterthought. A Compact Architecture Built for OCR Workloads A key technical point for this research is the use of Multi-Token Prediction (MTP). Standard autoregressive decoding predicts one token at a time, which is not ideal for OCR-style tasks where outputs are often deterministic and locally structured. GLM-OCR instead predicts multiple tokens per step. The model is trained to predict 10 tokens per step and generates 5.2 tokens per decoding step on average at inference time, yielding about 50% throughput improvement. To keep memory overhead manageable, the implementation uses a parameter-sharing scheme across the draft models. Two-Stage Layout Parsing Instead of Flat Page Reading At the system level, GLM-OCR adopts a two-stage pipeline. The first stage uses PP-DocLayout-V3 for layout analysis, which detects structured regions on the page. The second stage performs parallel region-level recognition over those detected areas. This is important because the model is not simply reading a whole page left-to-right as a generic vision-language model might. It first breaks down the page into semantically meaningful regions, which improves efficiency and makes the system more robust on documents with complicated layouts. Document Parsing and KIE Use Different Output Paths The architecture also separates two related document tasks. For document parsing, the pipeline uses layout detection and region processing to produce structured outputs such as Markdown and JSON. For Key Information Extraction (KIE), the research team describes a different path: the full document image is fed to the model with a task prompt, and the model directly generates JSON containing the extracted fields. That distinction matters because GLM-OCR is not presented as a single monolithic page-to-text model. It is a structured generation system with different operating modes depending on the task. A Four-Stage Training Pipeline with Task-Specific Rewards The training recipe is split into 4 stages. Stage 1 trains the vision encoder on image-text pairs and grounding or retrieval data. Stage 2.1 performs multimodal pretraining on image-text, document parsing, grounding, and VQA data. Stage 2.2 adds the MTP objective. Stage 3 is supervised fine-tuning on OCR-specific tasks including text recognition, formula transcription, table structure recovery, and KIE. Stage 4 applies reinforcement learning using GRPO. The reward design is task-specific: Normalized Edit Distance for text recognition, CDM score for formula recognition, TEDS score for table recognition, and field-level F1 for KIE, along with structural penalties such as repetition penalties, malformed structure penalties, and JSON validation constraints. Benchmark Results Show Strong Performance, With Important Caveats On public benchmarks, GLM-OCR reports strong results across several document tasks. It scores 94.6 on OmniDocBench v1.5, 94.0 on OCRBench (Text), 96.5 on UniMERNet, 85.2 on PubTabNet, and 86.0 on TEDS_TEST. For KIE, it reports 93.7 on Nanonets-KIE and 86.1 on Handwritten-KIE. The research team notes that results for Gemini-3-Pro and GPT-5.2-2025-12-11 are shown only for reference and are excluded from the best-score ranking, which is an important detail when interpreting claims about model leadership. https://arxiv.org/pdf/2603.10910 The benchmark story is strong, but it needs careful phrasing. GLM-OCR achieves the highest reported scores among the evaluated non-reference models on OmniDocBench v1.5, OCRBench (Text), UniMERNet, and TEDS_TEST. On PubTabNet, however, it does not lead overall; MinerU 2.5 reports 88.4 versus GLM-OCR’s 85.2. For KIE, GLM-OCR outperforms the listed open-source competitors in the above table, but Gemini-3-Pro scores higher on both Nanonets-KIE and Handwritten-KIE in the reference column. So the reserach team supports a strong competitive claim, but not a blanket ‘best at everything’ claim. Deployment Details The research team state that GLM-OCR supports vLLM, SGLang, and Ollama, and can be fine-tuned through LLaMA-Factory. They also report throughput of 0.67 images/s and 1.86 PDF pages/s under their evaluation setup. In addition, they describe a MaaS API priced at 0.2 RMB per million tokens, with example cost estimates for scanned images and simple-layout PDFs. These details suggest that GLM-OCR is being framed as both a research model and a deployable system. Key Takeaways GLM-OCR is a compact 0.9B multimodal OCR model built with a 0.4B CogViT encoder and 0.5B GLM decoder. It uses Multi-Token Prediction (MTP) to improve decoding efficiency, reaching 5.2 tokens per step on average and about 50% higher throughput. The model uses a two-stage pipeline: PP-DocLayout-V3 handles layout analysis, then GLM-OCR performs parallel region-level recognition. It supports both document parsing and KIE: parsing outputs Markdown/JSON, while KIE directly generates JSON from the full document image. Benchmark results are strong but not universal wins: GLM-OCR leads several reported non-reference benchmarks, but MinerU 2.5 is higher on PubTabNet, and Gemini-3-Pro is higher on the reference-only KIE scores. Check out Paper, Repo and Model Page. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Zhipu AI Introduces GLM-OCR: A 0.9B Multimodal OCR Model for Document Parsing and Key Information Extraction (KIE) appeared first on MarkTechPost.

Zhipu AI Introduces GLM-OCR: A 0.9B Multimodal OCR Model for Document Parsing and Key Information Extraction (KIE) Leggi l'articolo »

We use cookies to improve your experience and performance on our website. You can learn more at Politica sulla privacy and manage your privacy settings by clicking Settings.

Privacy Preferences

You can choose your cookie settings by turning on/off each type of cookie as you wish, except for essential cookies.

Allow All
Manage Consent Preferences
  • Always Active

Save
it_IT