Uncategorized

In the current landscape of computer vision, the standard operating procedure involves a modular ‘Lego-brick’ approach: a pre-trained vision encoder for feature extraction paired with a separate decoder for task prediction. While effective, this architectural separation complicates scaling and bottlenecks the interaction between language and vision. The Technology Innovation Institute (TII) research team is challenging this paradigm with Falcon Perception, a 600M-parameter unified dense Transformer. By processing image patches and text tokens in a shared parameter space from the very first layer, TII research team has developed an early-fusion stack that handles perception and task modeling with extreme efficiency. https://arxiv.org/pdf/2603.27365 The Architecture: A Single Stack for Every Modality The core design of Falcon Perception is built on the hypothesis that a single Transformer can simultaneously learn visual representations and perform task-specific generation. Hybrid Attention and GGROPE Unlike standard language models that use strict causal masking, Falcon Perception employs a hybrid attention strategy. Image tokens attend to each other bidirectionally to build a global visual context, while text and task tokens attend to all preceding tokens (causal masking) to enable autoregressive prediction. To maintain 2D spatial relationships in a flattened sequence, the research team uses 3D Rotary Positional Embeddings. This decomposes the head dimension into a sequential component and a spatial component using Golden Gate ROPE (GGROPE). GGROPE allows attention heads to attend to relative positions along arbitrary angles, making the model robust to rotation and aspect ratio variations. Minimalist Sequence Logic The basic architectural sequence follows a Chain-of-Perception format: [Image] [Text] <coord> <size> <seg> … <eos>. This ensures that the model resolves spatial ambiguity (position and size) as a conditioning signal before generating the final segmentation mask. Engineering for Scale: Muon, FlexAttention, and Raster Ordering TII research team introduced several optimizations to stabilize training and maximize GPU utilization for these heterogeneous sequences. Muon Optimization: The research team report that employing the Muon optimizer for specialized heads (coordinates, size, and segmentation) led to lower training losses and improved performance on benchmarks compared to standard AdamW. FlexAttention and Sequence Packing: To process images at native resolutions without wasting compute on padding, the model uses a scatter-and-pack strategy. Valid patches are packed into fixed-length blocks, and FlexAttention is used to restrict self-attention within each image sample’s boundaries. Raster Ordering: When multiple objects are present, Falcon Perception predicts them in raster order (top-to-bottom, left-to-right). This was found to converge faster and produce lower coordinate loss than random or size-based ordering. The Training Recipe: Distillation to 685GT The model uses multi-teacher distillation for initialization, distilling knowledge from DINOv3 (ViT-H) for local features and SigLIP2 (So400m) for language-aligned features. Following initialization, the model undergoes a three-stage perception training pipeline totaling approximately 685 Gigatokens (GT): In-Context Listing (450 GT): Learning to ‘list’ the scene inventory to build global context. Task Alignment (225 GT): Transitioning to independent-query tasks using Query Masking to ensure the model grounds each query solely on the image. Long-Context Finetuning (10 GT): Short adaptation for extreme density, increasing the mask limit to 600 per expression. During these stages, the task-specific serialization is used: <image>expr1<present><coord><size><seg> <eoq>expr2<absent> <eoq> <eos>. The <present> and <absent> tokens force the model to commit to a binary decision on an object’s existence before localization. PBench: Profiling Capabilities Beyond Saturated Baselines To measure progress, TII research team introduced PBench, a benchmark that organizes samples into five levels of semantic complexity to disentangle model failure modes. Main Results: Falcon Perception vs. SAM 3 (Macro-F1) Benchmark Split SAM 3 Falcon Perception (600M) L0: Simple Objects 64.3 65.1 L1: Attributes 54.4 63.6 L2: OCR-Guided 24.6 38.0 L3: Spatial Understanding 31.6 53.5 L4: Relations 33.3 49.1 Dense Split 58.4 72.6 Falcon Perception significantly outperforms SAM 3 on complex semantic tasks, particularly showing a +21.9 point gain on spatial understanding (Level 3). https://arxiv.org/pdf/2603.27365 FalconOCR: The 300M Document specialist TII team also extended this early-fusion recipe to FalconOCR, a compact 300M-parameter model initialized from scratch to prioritize fine-grained glyph recognition. FalconOCR is competitive with several larger proprietary and modular OCR systems: olmOCR: Achieves 80.3% accuracy, matching or exceeding Gemini 3 Pro (80.2%) and GPT 5.2 (69.8%). OmniDocBench: Reaches an overall score of 88.64, ahead of GPT 5.2 (86.56) and Mistral OCR 3 (85.20), though it trails the top modular pipeline PaddleOCR VL 1.5 (94.37). Key Takeaways Unified Early-Fusion Architecture: Falcon Perception replaces modular encoder-decoder pipelines with a single dense Transformer that processes image patches and text tokens in a shared parameter space from the first layer. It utilizes a hybrid attention mask—bidirectional for visual tokens and causal for task tokens—to act simultaneously as a vision encoder and an autoregressive decoder. Chain-of-Perception Sequence: The model serializes instance segmentation into a structured sequence (⟨coord⟩→⟨size⟩→⟨seg⟩)(langle coordrangle rightarrow langle sizerangle rightarrow langle segrangle), which forces it to resolve spatial position and size as a conditioning signal before generating the pixel-level mask. Specialized Heads and GGROPE: To manage dense spatial data, the model uses Fourier Feature encoders for high-dimensional coordinate mapping and Golden Gate ROPE (GGROPE) to enable isotropic 2D spatial attention. The Muon optimizer is employed for these specialized heads to balance learning rates against the pre-trained backbone. Semantic Performance Gains: On the new PBench benchmark, which disentangles semantic capabilities (Levels 0-4), the 600M model demonstrates significant gains over SAM 3 in complex categories, including a +13.4 point lead in OCR-guided queries and a +21.9 point lead in spatial understanding. High-Efficiency OCR Extension: The architecture scales down to Falcon OCR, a 300M-parameter model that achieves 80.3% on olmOCR and 88.64 on OmniDocBench. It matches or exceeds the accuracy of much larger systems like Gemini 3 Pro and GPT 5.2 while maintaining high throughput for large-scale document processing. Check out the Paper, Model Weight, Repo and Technical details.  Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post TII Releases Falcon Perception: A 0.6B-Parameter Early-Fusion Transformer for Open-Vocabulary Grounding and Segmentation from Natural Language Prompts appeared first on MarkTechPost.

IBM has announced the release of Granite 4.0 3B Vision, a vision-language model (VLM) engineered specifically for enterprise-grade document data extraction. Departing from the monolithic approach of larger multimodal models, the 4.0 Vision release is architected as a specialized adapter designed to bring high-fidelity visual reasoning to the Granite 4.0 Micro language backbone. This release represents a transition toward modular, extraction-focused AI that prioritizes structured data accuracy—such as converting complex charts to code or tables to HTML—over general-purpose image captioning. Architecture: Modular LoRA and DeepStack Integration The Granite 4.0 3B Vision model is delivered as a LoRA (Low-Rank Adaptation) adapter with approximately 0.5B parameters. This adapter is designed to be loaded on top of the Granite 4.0 Micro base model, a 3.5B parameter dense language model. This design allows for a ‘dual-mode’ deployment: the base model can handle text-only requests independently, while the vision adapter is activated only when multimodal processing is required. Vision Encoder and Patch Tiling The visual component utilizes the google/siglip2-so400m-patch16-384 encoder. To maintain high resolution across diverse document layouts, the model employs a tiling mechanism. Input images are decomposed into 384×384 patches, which are processed alongside a downscaled global view of the entire image. This approach ensures that fine details—such as subscripts in formulas or small data points in charts—are preserved before they reach the language backbone. The DeepStack Backbone To bridge the vision and language modalities, IBM utilizes a variant of the DeepStack architecture. This involves deeply stacking visual tokens into the language model across 8 specific injection points. By routing visual features into multiple layers of the transformer, the model achieves a tighter alignment between the ‘what’ (semantic content) and the ‘where’ (spatial layout), which is critical for maintaining structure during document parsing. Training Curriculum: Focused on Chart and Table Extraction The training of Granite 4.0 3B Vision reflects a strategic shift toward specialized extraction tasks. Rather than relying solely on general image-text datasets, IBM utilized a curated mixture of instruction-following data focused on complex document structures. ChartNet Dataset: The model was refined using ChartNet, a million-scale multimodal dataset designed for robust chart understanding. Code-Guided Pipeline: A key technical highlight of the training involves a “code-guided” approach for chart reasoning. This pipeline uses aligned data consisting of the original plotting code, the resulting rendered image, and the underlying data table, allowing the model to learn the structural relationship between visual representations and their source data. Extraction Tuning: The model was fine-tuned on a mixture of datasets focusing on Key-Value Pair (KVP) extraction, table structure recognition, and converting visual charts into machine-readable formats like CSV, JSON, and OTSL. Performance and Evaluation Benchmarks In technical evaluations, Granite 4.0 3B Vision has been benchmarked against several industry-standard suites for document understanding. It is important to note that datasets like PubTables-v2 and OmniDocBench are utilized as evaluation benchmarks to verify the model’s zero-shot performance in real-world scenarios. Task Evaluation Benchmark Metric KVP Extraction VAREX 85.5% Exact Match (Zero-Shot) Chart Reasoning ChartNet (Human-Verified Test Set) High Accuracy in Chart2Summary Table Extraction TableVQA-Bench & OmniDocBench Evaluated via TEDS and HTML extraction The model currently ranks 3rd among models in the 2–4B parameter class on the VAREX leaderboard (as of March 2026), demonstrating its efficiency in structured extraction despite its compact size. https://huggingface.co/blog/ibm-granite/granite-4-vision https://huggingface.co/blog/ibm-granite/granite-4-vision Key Takeaways Modular LoRA Architecture: The model is a 0.5B parameter LoRA adapter that operates on the Granite 4.0 Micro (3.5B) backbone. This design allows a single deployment to handle text-only workloads efficiently while activating vision capabilities only when needed. High-Resolution Tiling: Utilizing the google/siglip2-so400m-patch16-384 encoder, the model processes images by tiling them into 384×384 patches alongside a global downscaled view, ensuring that fine details in complex documents are preserved. DeepStack Injection: To improve layout awareness, the model uses a DeepStack approach with 8 injection points. This routes semantic features to earlier layers and spatial details to later layers, which is critical for accurate table and chart extraction. Specialized Extraction Training: Beyond general instruction following, the model was refined using ChartNet and a ‘code-guided’ pipeline that aligns plotting code, images, and data tables to help the model internalize the logic of visual data structures. Developer-Ready Integration: The release is Apache 2.0 licensed and features native support for vLLM (via a custom model implementation) and Docling, IBM’s tool for converting unstructured PDFs into machine-readable JSON or HTML. Check out the Technical details and Model Weight.  Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post IBM Releases Granite 4.0 3B Vision: A New Vision Language Model for Enterprise Grade Document Data Extraction appeared first on MarkTechPost.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Fuel prices are soaring. Plastic could be next.  As the war in Iran continues, one of the most visible global economic ripple effects has been fossil-fuel prices. But looking ahead, further consequences could be looming for plastics.  Plastics are made from petrochemicals, and the supply chain impacts from the conflict are starting to build up. Americans will likely feel the ripples.   Read the full story to grasp the unpredictable impacts.  —Casey Crownhart  This story is from The Spark, our weekly climate newsletter. Sign up to get it in your inbox every Wednesday.  The must-reads  I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.  1 SpaceX has filed for an IPO It’s set to be the largest ever, targeting a $1.75 trillion valuation. (NYT $)  + Which would make Elon Musk the world’s first trillionaire. (Al Jazeera) + But the IPO could hinge on the success of Moon missions. (LA Times $) + And the conflicts of interest are staggering. (The Next Web) + Meanwhile, rivals are rising to challenge SpaceX. (MIT Technology Review)   2 Artemis II is on its way to the Moon NASA successfully launched the four astronauts on its rocket yesterday. (Axios) + The lunar plans could violate international law. (The Verge) + But the potential scientific advances are tremendous. (Nature)  + Check out our roundtable on the next era of space exploration. (MIT Technology Review)   3 Iran has struck Amazon’s cloud business in Bahrain again It promised to hit US companies only yesterday. (FT $) + Other targets include Google, Microsoft, Apple, and Nvidia. (CNBC) + AWS data centers in Bahrain were also hit last month. (Reuters $)  4 OpenAI was secretly behind a child safety campaign group It pushed for age verification requirements for AI. (The San Francisco Standard $) + OpenAI had backed the legislation as a compromise measure. (WSJ $) + Coincidentally, Sam Altman heads a company providing age verification. (Engadget)  5 Anthropic is scrambling to limit the Claude Code leak It’s trying to remove 8,000 copies of the exposed code from GitHub. (Gizmodo) + An executive blamed the leak on “process errors.” (Bloomberg $) + Here’s what it reveals about Anthropic’s plans. (Ars Technica) + AI is making online crimes easier—and it could get much worse. (MIT Technology Review)  6 A new Russian “super-app” aims to emulate China’s WeChat And give the Kremlin new surveillance powers. (WSJ $)  7 America’s AI boom is leaving the rest of the world behind  And it’s concentrating power and wealth in a handful of companies. (Rest of World)  8 Chinese chipmakers have claimed nearly half the country’s market Nvidia’s lead is shrinking rapidly. (Reuters $)  9 The first quantum computer to break encryption is imminent  New research reveals how it could happen. (New Scientist)  10 The world’s oldest tortoise has been embroiled in a crypto scam Reports that Jonathan died at just 194 years old are thankfully false. (Guardian)  Quote of the day  “Starlink is the only reason this valuation is defensible.”  —Shay Boloor, chief market strategist at Futurum Equities, tells Reuters why SpaceX has such high hopes for its IPO.  One More Thing  These companies are creating food out of thin air  Dried cells—it’s what’s for dinner. At least that’s what a new crop of biotech startups, armed with carbon-guzzling bacteria and plenty of capital, are hoping to convince us.   Their claims sound too good to be true: they say they can make food out of thin air. But that’s exactly how certain soil-dwelling bacteria work.  Startups are replicating the process to turn abundant carbon dioxide into nutritious “air protein.” They believe it could dramatically lower farming emissions—and even disrupt agriculture altogether. Read the full story.  —Claire L. Evans  We can still have nice things  A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.)  + Need more Artemis II in your life? This site takes you inside the flight. + Here’s a fascinating look at the recording errors that improved songs. + Good news: the elusive Nightjar bird is making a comeback. + Finally, a master chef has baked clam chowder donuts. 

In the current landscape of generative AI, the ‘scaling laws’ have generally dictated that more parameters equal more intelligence. However, Liquid AI is challenging this convention with the release of LFM2.5-350M. This model is actually a technical case study in intelligence density with additional pre-training (from 10T to 28T tokens) and large-scale reinforcement learning The significance of LFM2.5-350M lies in its architecture and training efficiency. While the most AI companies has been focused on frontier models, Liquid AI is targeting the ‘edge’—devices with limited memory and compute—by proving that a 350-million parameter model can outperform models more than twice its size on several evaluated benchmarks. https://www.liquid.ai/blog/lfm2-5-350m-no-size-left-behind Architecture: The Hybrid LIV Backbone The core technical differentiator of the LFM2.5-350M is its departure from the pure Transformer architecture. It utilizes a hybrid structure built on Linear Input-Varying Systems (LIVs). Traditional Transformers rely entirely on self-attention mechanisms, which suffer from quadratic scaling issues: as the context window grows, the memory and computational requirements for the Key-Value (KV) cache increase. Liquid AI addresses this by using a hybrid backbone consisting of: 10 Double-Gated LIV Convolution Blocks: These handle the majority of the sequence processing. LIVs function similarly to advanced Recurrent Neural Networks (RNNs) but are designed to be more parallelizable and stable during training. They maintain a constant-state memory, reducing the I/O overhead. 6 Grouped Query Attention (GQA) Blocks: By integrating a small number of attention blocks, the model retains high-precision retrieval and long-range context handling without the full memory overhead of a standard Transformer. This hybrid approach allows the LFM2.5-350M to support a 32k context window (32,768 tokens) while maintaining an extremely lean memory footprint. Performance and Intelligence Density The LFM2.5-350M was pre-trained on 28 trillion tokens with an extremely high training-to-parameter ratio. This ensures that the model’s limited parameter count is utilized to its maximum potential, resulting in high ‘intelligence density.’ Benchmarks and Use Cases The LFM2.5-350M is a specialist model designed for high-speed, agentic tasks rather than general-purpose reasoning. Benchmark Score IFEval (Instruction Following) 76.96 GPQA Diamond 30.64 MMLU-Pro 20.01 The high IFEval score indicates the model is efficient at following complex, structured instructions, making it suitable for tool use, function calling, and structured data extraction (e.g., JSON). However, the documentation explicitly states that LFM2.5-350M is not recommended for mathematics, complex coding, or creative writing. For those tasks, the reasoning capabilities of larger parameter counts remain necessary. https://www.liquid.ai/blog/lfm2-5-350m-no-size-left-behind Hardware Optimization and Inference Efficiency A major hurdle for AI devs is the ‘memory wall’—the bottleneck created by moving data between the processor and memory. Because the LFM2.5-350M utilizes LIVs and GQA, it drastically reduces KV cache size, boosting throughput. On a single NVIDIA H100 GPU, the model can reach a throughput of 40.4K output tokens per second at high concurrency. Liquid AI team reports device-specific low-memory inference results that make local deployment viable: Snapdragon 8 Elite NPU: 169MB peak memory using RunAnywhere Q4. Snapdragon GPU: 81MB peak memory using RunAnywhere Q4. Raspberry Pi 5: 300MB using Cactus Engine int8. Key Takeaways Extreme Intelligence Density: By training a 350M parameter model on 28 trillion tokens, Liquid AI team achieved an super high 80,000:1 token-to-parameter ratio, allowing it to outperform models more than twice its size on several benchmarks. Hybrid LIV Architecture: The model departs from pure Transformers by using Linear Input-Varying Systems (LIVs) combined with a small number of Grouped Query Attention (GQA) blocks, significantly reducing the memory overhead of the KV cache. Edge-First Efficiency: It is designed for local deployment with a 32k context window and a remarkably low memory footprint—reaching as low as 81MB on mobile GPUs and 169MB on NPUs via specialized inference engines. Specialized Agentic Capability: The model is highly optimized for instruction following (IFEval: 76.96) and tool use, though it is explicitly not recommended for complex coding, mathematics, or creative writing. Massive Throughput: The architectural efficiency enables high-speed utility, processing up to 40.4K output tokens per second on a single H100, making it ideal for high-volume data extraction and real-time classification. Check out the Technical details and Model Weight. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Liquid AI Released LFM2.5-350M: A Compact 350M Parameter Model Trained on 28T Tokens with Scaled Reinforcement Learning appeared first on MarkTechPost.

Google has announced the release of Veo 3.1 Lite, a new model tier within its generative video portfolio designed to address the primary bottleneck for production-scale deployments: pricing. While the generative video space has seen rapid progress in visual fidelity, the cost per second of generated content has remained high, often prohibitive for developers building high-volume applications. Veo 3.1 Lite is now available via the Gemini API and Google AI Studio for users in the paid tier. By offering the same generation speed as the existing Veo 3.1 Fast model at approximately half the cost, Google is positioning this model as the standard for developers focused on programmatic video generation and iterative prototyping. https://blog.google/innovation-and-ai/technology/ai/veo-3-1-lite/ Technical Architecture: The Diffusion Transformer (DiT) The most significant aspect of the Veo 3.1 family is its underlying Diffusion Transformer (DiT) architecture. Traditional generative video models often relied on U-Net-based diffusion, which can struggle with high-dimensional data and long-range temporal dependencies. Veo 3.1 Lite utilizes a transformer-based backbone that operates on spatio-temporal patches. In this architecture, video frames are not processed as static 2D images but as a continuous sequence of tokens in a latent space. By applying self-attention across these patches, the model maintains better temporal consistency. This ensures that objects, lighting, and textures remain coherent across the duration of the clip, reducing the artifacts commonly seen in earlier models. The model performs its computation in a compressed latent space rather than pixel space. This allows the model to handle the high computational demands of video generation while maintaining a lower memory footprint. For developers, this translates to a model that can generate high-definition content without the exponential increase in compute time that usually accompanies resolution scaling. Performance and Output Specifications Veo 3.1 Lite provides specific parameters for resolution and duration, allowing AI devs to integrate it into structured workflows. Unlike the flagship Veo 3.1 model, which supports 4K resolution, the Lite version is optimized for high-definition (HD) outputs. Supported Resolutions: 720p and 1080p. Aspect Ratios: Native support for both landscape (16:9) and portrait (9:16) orientations. Clip Durations: Developers can specify generation lengths of 4, 6, or 8 seconds. Prompt Adherence: The model is optimized for ‘Cinematic Control,’ recognizing technical directives such as ‘pan,’ ’tilt,’ and specific lighting instructions. The ‘Lite’ tag does not refer to a reduction in generation speed compared to the ‘Fast’ tier. Instead, it refers to an optimized parameter set that allows Google team to offer the model at a significantly lower price point while maintaining the same low-latency performance characteristics of Veo 3.1 Fast. The Pricing Shift: Democratizing Video Inference The core value proposition of Veo 3.1 Lite is its cost structure. In the current market, high-quality video inference often costs several dollars per minute of footage, making it difficult to justify for applications like dynamic ad generation or social media automation. Veo 3.1 Lite pricing is structured as follows: 720p: $0.05 per second. 1080p: $0.08 per second. Deployment via Gemini API and AI Studio The accessibility is handled through the Gemini API. This allows for the integration of video generation into existing Python or Node.js applications using standard REST or gRPC calls. One critical technical feature for enterprise developers is the inclusion of SynthID. Developed by Google DeepMind, SynthID is a tool for watermarking and identifying AI-generated content. It embeds a digital watermark directly into the pixels of the video that is imperceptible to the human eye but detectable by specialized software. This is a mandatory component for developers concerned with safety, compliance, and distinguishing synthetic media from captured footage. Key Takeaways Half the Cost, Same Speed: Offers the same low-latency performance as the ‘Fast’ tier at less than 50% of the price ($0.05/sec for 720p). Scalable HD Output: Supports 720p and 1080p resolutions in 4, 6, or 8-second clips with native 16:9 and 9:16 aspect ratios. Architecture: Built on a Diffusion Transformer (DiT) using spatio-temporal patches for superior motion and physical consistency. Developer Ready: Available now via Gemini API (paid tier) and Google AI Studio, featuring built-in SynthID digital watermarking. Check out the Technical details. You can access the model via paid tier on the Gemini API and Google AI Studio. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Google AI Releases Veo 3.1 Lite: Giving Developers Low Cost High Speed Video Generation via The Gemini API appeared first on MarkTechPost.