The 6 Best AI Agent Memory Frameworks You Should Try in 2026
Memory helps
The 6 Best AI Agent Memory Frameworks You Should Try in 2026 Leer entrada »
What if the AI industry is optimizing for a goal that cannot be clearly defined or reliably measured? That is the central argument of a new paper by Yann LeCun, and his team, which claims that Artificial General Intelligence has become an overloaded term used in inconsistent ways across academia and industry. The research team argued that because AGI lacks a stable operational definition, it has become a weak scientific target for evaluating progress or guiding research. Why Human Intelligence Is Not Truly ‘General‘ The research team in the paper starts by challenging a common assumption behind many AGI discussions: that human intelligence is a meaningful template for’ ‘general’ intelligence. The research team argue that humans only appear general because we evaluate intelligence from inside the task distribution shaped by human biology and survival. We are good at the kinds of tasks that mattered for our existence, such as perception, motor control, planning, and social reasoning. But outside that range, human ability is limited, and in many cases machines already outperform us. The research paper’s point is not that humans are narrow in every sense, but that human intelligence is better understood as specialized and adaptable rather than general in any universal sense. The Problem With Human-Centered AGI Definitions That distinction matters because many AGI definitions quietly inherit a human-centered benchmark. The research team argues there is no real consensus on what AGI means across academia or industry. Some definitions focus on doing everything a human can do. Others focus on economic usefulness, broad task competence, open-ended reasoning, or the ability to learn. These are not equivalent definitions, and they do not produce one clean evaluation target. The research team therefore argue that existing AGI definitions are insufficient because they are often ambiguous, difficult to assess, or not truly general once examined closely. The Shift From AGI to SAI The research paper’s alternative is Superhuman Adaptable Intelligence, or SAI. It defines SAI as intelligence that can adapt to exceed humans at any task humans can do, while also adapting to useful tasks outside the human domain. That is a subtle but important shift. Instead of asking whether a system already matches humans across a fixed checklist of tasks, the research team asks how quickly the system can learn something new and how broadly it can continue adapting. In this framework, the key metric is adaptation speed: the speed with which an agent acquires new skills and learns new tasks. Why Adaptation Speed Matters More Than Static Benchmarks This reframes the problem in a more engineering-friendly way. A benchmark based on a growing catalog of tasks becomes messy fast; the space of possible skills is effectively unbounded. The research team argued that evaluating intelligence as a static inventory of competencies is the wrong abstraction. What matters more is whether a system can specialize rapidly when it encounters a new domain, new objective, or new environment. That is why the research paper treats adaptability, rather than generality, as the better North Star. Specialization as a Feature, Not a Failure A second major claim in the research paper is that AI progress should not be framed as a march toward one universal model that does everything equally well. The research team argued that specialization is not a weakness of intelligence but a practical route to high performance. Humans themselves are not a counterexample; they are part of the evidence. The research paper suggests that future AI systems will likely need internal specialization, hierarchy, and diversity across models and modalities rather than a single monolithic system. In plain terms, the research paper argues that one model should not be expected to master all domains with equal efficiency just because current marketing language likes the word ‘general.’ Why the Research Paper Points to Self-Supervised Learning? From there, the research paper connects SAI to self-supervised learning. The logic is straightforward. If the goal is fast adaptation across a very large task space, then relying only on supervised learning becomes limiting because supervised methods assume access to large, reliable labeled datasets. In real settings, that assumption often fails. The research team argues that self-supervised learning is a promising pathway because it can exploit structure in raw data and has already driven strong results across domains. Importantly, they do not claim that SAI requires one specific architecture. They present self-supervised learning as a promising route, not a final architectural answer. World Models and the Limits of Surface-Level Prediction The research paper also argues that strong adaptation likely benefits from world models. Here the research team move away from the idea that token-level or pixel-level prediction alone is enough for robust intelligence in the physical world. They argue that what matters is learning compact representations that capture system dynamics. In that view, a world model supports simulation and planning, which in turn support zero-shot and few-shot adaptation. The research paper points to latent prediction architectures such as JEPA, Dreamer 4, and Genie 2 as examples of the kind of direction the field should explore, while again stating that SAI does not dictate a single architecture. A Warning Against Architectural Monoculture The research team also criticize the current level of architectural homogeneity in advanced AI. They note that autoregressive LLMs and LMMs dominate the ‘general’ AI landscape in part because shared tooling and benchmarks create momentum. But the research paper argues that this concentration narrows the search space and can slow progress. It further claims that autoregressive systems have well-known weaknesses, including error accumulation over long horizons, which makes long-horizon interaction brittle. Their broader point is not that current large models are useless. It is that the field should avoid treating one successful paradigm as the final template for intelligence. Key Takeaways The research paper argues AGI is not a precise scientific target: According to the research team, AGI is used inconsistently across academia and industry, making it difficult to define, measure, or use as a stable research goal. Human intelligence should not be treated as the definition of
In this tutorial, we explore tqdm in depth and demonstrate how we build powerful, real-time progress tracking into modern Python workflows. We begin with nested progress bars and manual progress control, then move into practical scenarios such as streaming downloads, pandas data processing, parallel execution, structured logging, and asynchronous tasks. Throughout this tutorial, we focus on writing clean, production-ready code that runs in Colab while showcasing the advanced capabilities of tqdm beyond simple loops. Copy CodeCopiedUse a different Browser !pip -q install -U tqdm import time, math, random, asyncio, hashlib, logging import pandas as pd import requests from tqdm.auto import tqdm, trange from tqdm.contrib.concurrent import thread_map, process_map from tqdm.contrib.logging import logging_redirect_tqdm import tqdm as tqdm_pkg print(“tqdm version:”, tqdm_pkg.__version__) print(“pandas version:”, pd.__version__) print(“requests version:”, requests.__version__) We install and configure tqdm in a Colab-safe manner while preserving the existing environment dependencies. We import all required libraries, including concurrency and logging helpers from tqdm.contrib. We also print version information to verify that our runtime setup is stable before proceeding. Copy CodeCopiedUse a different Browser print(“1) Nested progress bars (position/leave) + tqdm.write()”) outer = trange(5, desc=”Outer loop”, leave=True) for i in outer: inner = trange(20, desc=f”Inner loop {i}”, leave=False, position=1) for j in inner: time.sleep(0.01) if j in (0, 10, 19): tqdm.write(f” note: i={i}, j={j}”) print() print(“2) Manual progress (unknown -> known total, update(), set_postfix())”) items = list(range(1, 101)) pbar = tqdm(total=None, desc=”Processing (discovering total)”, unit=”item”) seen = 0 for x in items: time.sleep(0.005) seen += 1 if seen == 25: pbar.total = len(items) pbar.refresh() pbar.update(1) if x % 20 == 0: pbar.set_postfix(last=x, sqrt=round(math.sqrt(x), 3)) pbar.close() print() We demonstrate nested progress bars and show how we manage multiple levels of iteration cleanly using position and leave. We also explore manual progress control by dynamically setting totals and updating progress explicitly. By using set_postfix, we enrich the progress bar with live metadata as it runs. Copy CodeCopiedUse a different Browser print(“3) Download with streaming progress”) url = “https://raw.githubusercontent.com/tqdm/tqdm/master/README.rst” out_path = “/content/tqdm_README.rst” with requests.get(url, stream=True, timeout=30) as r: r.raise_for_status() total = int(r.headers.get(“Content-Length”, 0)) or None chunk = 1024 * 32 with open(out_path, “wb”) as f, tqdm( total=total, unit=”B”, unit_scale=True, unit_divisor=1024, desc=”Downloading README”, miniters=1, ) as bar: for part in r.iter_content(chunk_size=chunk): if not part: continue f.write(part) bar.update(len(part)) print(“Saved:”, out_path) print() We implement a real-world streaming download scenario using requests with chunk-based updates. We track byte-level progress with automatic unit scaling and accurate handling of content length. This shows how we efficiently and transparently monitor external I/O operations. Copy CodeCopiedUse a different Browser print(“4) pandas progress_apply (Series) + DataFrame row-wise progress (safe)”) tqdm.pandas() df = pd.DataFrame({ “user_id”: range(1, 2001), “value”: [random.random() for _ in range(2000)], }) def heavy_fn(v: float) -> str: time.sleep(0.0005) s = f”{v:.10f}”.encode(“utf-8”) return hashlib.sha256(s).hexdigest()[:10] df[“hash”] = df[“value”].progress_apply(heavy_fn) df2 = df[[“value”]].copy() df2[“hash2”] = [ heavy_fn(float(v)) for v in tqdm(df2[“value”].to_list(), desc=”Row-wise hash2″, total=len(df2)) ] df[“hash2”] = df2[“hash2”] print(df.head(3)) print() We integrate tqdm with pandas to monitor vectorized operations using progress_apply. We implement a hashing workload to simulate realistic, computationally heavy transformations. We also demonstrate a safe row-wise progress pattern to ensure compatibility with Colab’s pinned pandas version. Copy CodeCopiedUse a different Browser print(“5) Concurrency progress: thread_map / process_map”) def cpuish(n: int) -> int: x = 0 for i in range(50_000): x = (x + (n * i)) % 1_000_003 return x nums = list(range(80)) thread_results = thread_map(cpuish, nums, max_workers=8, desc=”thread_map”) print(“thread_map done:”, len(thread_results)) proc_results = process_map(cpuish, nums[:20], max_workers=2, chunksize=2, desc=”process_map”) print(“process_map done:”, len(proc_results)) print() print(“6) logging_redirect_tqdm (logs won’t break bars)”) logger = logging.getLogger(“demo”) logger.setLevel(logging.INFO) handler = logging.StreamHandler() handler.setFormatter(logging.Formatter(“%(levelname)s: %(message)s”)) logger.handlers = [handler] with logging_redirect_tqdm(): for k in tqdm(range(60), desc=”Work with logs”): time.sleep(0.01) if k in (5, 25, 45): logger.info(f”checkpoint k={k}”) print() print(“7) asyncio progress (as_completed) — Colab/Jupyter-safe”) async def io_task(i: int): await asyncio.sleep(random.uniform(0.02, 0.12)) return i, random.random() async def run_async(): tasks = [asyncio.create_task(io_task(i)) for i in range(80)] results = [] for fut in tqdm(asyncio.as_completed(tasks), total=len(tasks), desc=”async tasks”): results.append(await fut) return results results = await run_async() print(“async done:”, len(results), “results”) We explore advanced execution patterns including multithreading, multiprocessing, structured logging, and asynchronous task tracking. We use thread_map and process_map to parallelize CPU-bound workloads with visible progress. Also, we handle asyncio safely in a notebook environment using top-level await, ensuring smooth progress tracking without event loop conflicts. In conclusion, we integrated tqdm across synchronous, parallel, logging-aware, and asynchronous environments. We saw how progress bars enhance observability, improve debugging clarity, and make long-running workloads more transparent and user-friendly. With these advanced patterns, we now have a solid foundation to incorporate robust progress monitoring into data pipelines, machine learning workflows, distributed systems, and real-world production applications. Check out the Full Codes here. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post How to Build Progress Monitoring Using Advanced tqdm for Async, Parallel, Pandas, Logging, and High-Performance Workflows appeared first on MarkTechPost.
The ongoing public feud between the Department of Defense and the AI company Anthropic has raised a deep and still unanswered question: Does the law actually allow the US government to conduct mass surveillance on Americans? Surprisingly, the answer is not straightforward. More than a decade after Edward Snowden exposed the NSA’s collection of bulk metadata from the phones of Americans, the US is still navigating a gap between what ordinary people think and what the law allows. The flashpoint in the standoff between Anthropic and the government was the Pentagon’s desire to use Anthropic’s AI Claude to analyze bulk commercial data on Americans. Anthropic demanded that its AI not be used for mass domestic surveillance (or for autonomous weapons, which are machines that can kill targets without human oversight). A week after negotiations broke down, the Pentagon designated Anthropic a supply chain risk, a label typically reserved for foreign companies that pose a threat to national security. Meanwhile, OpenAI, the rival AI company behind ChatGPT, sealed a deal that allowed the Pentagon to use its AI for “all lawful purposes”—language that critics say left the door open to domestic surveillance. Over the following weekend, users uninstalled ChatGPT in droves. Protesters chalked messages around OpenAI’s headquarters in San Francisco: “What are your redlines?” OpenAI announced on Monday that it had reworked its deal to make sure that its AI will not be used for domestic surveillance. The company added that its services will not be used by intelligence agencies, such as the NSA. CEO Sam Altman suggested that existing law prohibits domestic surveillance by the Department of Defense (now sometimes called the Department of War) and that OpenAI’s contract simply needed to reference this law. “The DoW agrees with these principles, reflects them in law and policy, and we put them into our agreement,” he wrote on X. Anthropic CEO Dario Amodei argued the opposite. “To the extent that such surveillance is currently legal, this is only because the law has not yet caught up with the rapidly growing capabilities of AI,” he wrote in a policy statement. So, who is right? Does the law allow the Pentagon to surveil Americans using AI? Supercharged surveillance The answer depends on what we think counts as surveillance. “A lot of stuff that normal people would consider a search or surveillance … is not actually considered a search or surveillance by the law,” says Alan Rozenshtein, a law professor at the University of Minnesota Law School. That means public information—such as social media posts, surveillance camera footage, and voter registration records—is fair game. So is information on Americans picked up incidentally from surveillance of foreign nationals. Most notably, the government can purchase commercial data from companies, which can include sensitive personal information like mobile location and web browsing records. In recent years, agencies from ICE and IRS to the FBI and NSA have increasingly tapped into this data marketplace, fueled by an internet economy that harvests user data for advertising. These data sets can let the government access information that might not be available without a warrant or subpoena, which are normally required to obtain sensitive personal data. “There’s a huge amount of information that the government can collect on Americans that is not itself regulated either by the Constitution, which is the Fourth Amendment, or statute,” says Rozenshtein. And there aren’t meaningful limits on what the government can do with all this data. That’s because until the last several decades, people weren’t generating massive clouds of data that opened up new possibilities for surveillance. The Fourth Amendment, which protects against unreasonable search and seizure, was written when collecting information meant entering people’s homes. Subsequent laws, like the Foreign Intelligence Surveillance Act of 1978 or the Electronic Communications Privacy Act of 1986, were passed when surveillance involved wiretapping phone calls and intercepting emails. The bulk of laws governing surveillance were on the books before the internet took off. We weren’t generating vast trails of online data, and the government didn’t have sophisticated tools to analyze the data. Now we do, and AI supercharges what kind of surveillance can be carried out. “What AI can do is it can take a lot of information, none of which is by itself sensitive, and therefore none of which by itself is regulated, and it can give the government a lot of powers that the government didn’t have before,” says Rozenshtein. AI can aggregate individual pieces of information to spot patterns, draw inferences, and build detailed profiles of people—at massive scale. And as long as the government collects the information lawfully, it can do whatever it wants with that information, including feeding it to AI systems. “The law has not caught up with technological reality,” says Rozenshtein. While surveillance can raise serious privacy concerns, the Pentagon can have legitimate national security interests in collecting and analyzing data on Americans. “In order to collect information on Americans, it has to be for a very specific subset of missions,” says Loren Voss, a former military intelligence officer at the Pentagon. For example, a counterintelligence mission might require information about an American who is working for a foreign country, or plotting to engage in international terrorist activities. But targeted intelligence can sometimes stretch into collecting more data. “This kind of collection does make people nervous,” says Voss. Lawful use OpenAI has amended its contract to say that the company’s AI system “shall not be intentionally used for domestic surveillance of U.S. persons and nationals,” in line with relevant laws. The amendment clarifies that this prohibits “deliberate tracking, surveillance or monitoring of U.S. persons or nationals, including through the procurement or use of commercially acquired personal or identifiable information.” But the added language might not do much to override the clause that the Pentagon may use the company’s AI system for all lawful purposes, which could include collecting and analyzing sensitive personal information. “OpenAI can say whatever it wants in its agreement … but the Pentagon’s gonna use the tech
Is the Pentagon allowed to surveil Americans with AI? Leer entrada »
This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. Coming soon: our 10 Things That Matter in AI Right Now For years, MIT Technology Review’s newsroom has been ahead of the curve, tracking the developments in AI that matter and explaining what they mean. Now, our world-leading AI team is creating something definitive: the 10 Things That Matter in AI Right Now. Publishing in April to be launched at our flagship AI event, EmTech AI, this special report will reveal what our expert journalists are tracking most closely, what breakthroughs have excited them, and what transformations they see on the horizon. It’s our authoritative snapshot of where AI is heading in the year ahead—a curated expert list of 10 technologies, emerging trends, bold ideas, and powerful movements reshaping our world. Attendees at EmTech AI will get much more than an exclusive heads-up of what made our 10 Things That Matter in AI Right Now list. We’re at a pivotal moment as AI moves from pilot testing into core business infrastructure, and to reflect that we’ve curated a program that will help you navigate what’s going on, and get ahead of what’s coming next. We’ll hear from top leaders at OpenAI, Walmart, General Motors, Poolside, MIT, the Allen Institute for AI (Ai2) and SAG-AFTRA. Topics will include everything from how organizations are preparing for AI agents to how AI will change the future of human expression. As well as networking with speakers, you’ll have the chance to mingle with MIT Technology Review’s editors too. Download readers get 10% off tickets, so what are you waiting for? See you there! The must-reads I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology. 1 Anthropic says it plans to sue the PentagonIt believes the DoD’s ban on its software is unlawful. (BBC) + CEO Dario Amodei has nonetheless apologized for a leaked memo criticizing Trump. (Axios)+ Trump, meanwhile, says he fired Anthropic “like dogs.” (The Guardian)+ In happier news for Anthropic, its models can remain in Microsoft products.(CNBC) 2 The Pentagon has been secretly testing OpenAI models for yearsWhich shows exactly how effective OpenAI’s ban on military use of its models has been. (Wired $) 3 A new lawsuit says Trump’s TikTok deal helped firms that ‘personally enriched’ himThe suit aims to reverse the sale of the app’s US operations. (CBS News)+ It could shed light on the majority American-owned joint venture for TikTok. (Reuters) 4 AI could give smart homes a reboot Google and Amazon are betting on smarter assistants—but not everyone’s convinced (NYT) 5 Iran has struck Amazon data centers, rattling the Gulf’s AI ambitionsThe first military hit on a US hyperscaler has shaken the region’s tech sector. (FT $)+ The conflict has thrown a spotlight on AI’s current use in warfare—and what’s next. (Nature) 6 Trump and tech CEOs have promised to protect consumers from AI’s energy costsGoogle, Microsoft, Meta, Amazon, OpenAI, Oracle and xAI have all signed the pledge. (Axios)+ But what is AI’s true energy footprint? We did the math. (MIT Technology Review) 7 Meta’s getting sued over surveillance through smart glasses The suit claims Meta misled users over the devices’ privacy features. (TechCrunch) 8 There’s a new field of study: researching ‘AI societies’Scientists are examining human behavior without even involving humans. (Nature)+ Hundreds of AI agents built their own society in Minecraft. (MIT Technology Review) 9 Oh great, teenage boys are using ChatGPT to chat up girlsOf all the things to outsource to AI, flirting surely ain’t it. (Vox) 10 The mythical Nintendo PlayStation has a new home The US National Video Museum has bought the fabled console’s development kit. (Engadget) Quote of the day “It’s sort of bitterly ironic.” —Dean Ball, a former Trump administration AI adviser, tells Politico that the Anthropic spat contradicts the president’s pledge to cut bureaucratic red tape for tech. One more thing KATHERINE LAM These scientists are working to extend the life span of pet dogs—and their owners Gavesh’s journey began with a Facebook job advert promising a better life. Instead, he was trafficked into “pig butchering”—a form of fraud where scammers build close relationships with online targets to extract money. We spoke to Gavesh and five other workers from inside the scam industry, as well as anti-trafficking experts and technology specialists. Their testimony reveals how global tech platforms have industrialized this criminal trade—and why those same companies now hold the key to dismantling it. Read the full story. —Peter Guest and Emily Fishbein We can still have nice things A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line or skeet ’em at me.) + The Blood Moon of March 3 was sublime.+ Orysia Zabeida’s imperfect animations, drawn frame-by-frame from memory, are hypnotizing.+ This stunning snap of a white whale calf scooped the top prize at the World Nature Photography Awards.+ Two “Lazarus” marsupial species just came back from the dead in a big win for biodiversity.
The Download: 10 things that matter in AI, plus Anthropic’s plan to sue the Pentagon Leer entrada »
Microsoft has released Phi-4-reasoning-vision-15B, a 15 billion parameter open-weight multimodal reasoning model designed for image and text tasks that require both perception and selective reasoning. It is a compact model built to balance reasoning quality, compute efficiency, and training-data requirements, with particular strength in scientific and mathematical reasoning and understanding user interfaces. https://arxiv.org/pdf/2603.03975 What the model is built on? Phi-4-reasoning-vision-15B combines the Phi-4-Reasoning language backbone with the SigLIP-2 vision encoder using a mid-fusion architecture. In this setup, the vision encoder first converts images into visual tokens, then those tokens are projected into the language model embedding space and processed by the pretrained language model. This design acts as a practical trade-off: it preserves strong cross-modal reasoning while keeping training and inference costs manageable compared with heavier early-fusion designs. https://arxiv.org/pdf/2603.03975 Why Microsoft took the smaller-model route? Many recent vision-language models have grown in parameter count and token usage, which raises both latency and deployment cost. Phi-4-reasoning-vision-15B was built as a smaller alternative that still handles common multimodal workloads without relying on extremely large training datasets or excessive inference-time token generation. The model was trained on 200 billion multimodal tokens, building on Phi-4-Reasoning, which was trained on 16 billion tokens, and ultimately on the Phi-4 base model, which was trained on 400 billion unique tokens. Microsoft contrasts that with the more than 1 trillion tokens used to train several recent multimodal models such as Qwen 2.5 VL, Qwen 3 VL, Kimi-VL, and Gemma 3. https://arxiv.org/pdf/2603.03975 High-resolution perception was a core design choice Microsoft team explains one of the more useful technical lessons in their technical report that multimodal reasoning often fails because perception fails first. Models can miss the answer not because they lack reasoning ability, but because they fail to extract the relevant visual details from dense images such as screenshots, documents, or interfaces with small interactive elements. Phi-4-reasoning-vision-15B uses a dynamic resolution vision encoder with up to 3,600 visual tokens, which is intended to support high-resolution understanding for tasks such as GUI grounding and fine-grained document analysis. The Microsoft team states that high-resolution, dynamic-resolution encoders yield consistent improvements, and explicitly notes that accurate perception is a prerequisite for high-quality reasoning. Mixed reasoning instead of forcing reasoning everywhere A second important design decision is the model’s mixed reasoning and non-reasoning training strategy. Rather than forcing chain-of-thought-style reasoning for all tasks, Microsoft team trained the model to switch between two modes. Reasoning samples include <think>…</think> traces, while non-reasoning samples begin with <nothink> and are used for perception-focused tasks such as captioning, grounding, OCR, and simple VQA. The reasoning data makes up about 20% of the overall training mixture. The goal of this hybrid setup is to let the model respond directly on tasks where longer reasoning adds latency without improving accuracy, while still invoking structured reasoning on tasks such as math and science. Microsoft team also notes an important limitation: the boundary between these modes is learned implicitly, so switching is not always optimal. Users can override the default behavior through explicit prompting with <think> or <nothink> tokens. What areas are stronger? Microsoft team highlights 2 main application areas. The first is scientific and mathematical reasoning over visual inputs, including handwritten equations, diagrams, charts, tables, and quantitative documents. The second is computer-use agent tasks, where the model interprets screen content, localizes GUI elements, and supports interaction with desktop, web, or mobile interfaces. https://arxiv.org/pdf/2603.03975 Benchmark results Microsoft team reports the following benchmark scores for Phi-4-reasoning-vision-15B: 84.8 on AI2DTEST, 83.3 on ChartQATEST, 44.9 on MathVerseMINI, 36.2 on MathVisionMINI, 75.2 on MathVistaMINI, 54.3 on MMMUVAL, 64.5 on MMStar, 76.0 on OCRBench, and 88.2 on ScreenSpotv2. The technical report also notes that these results were generated using Eureka ML Insights and VLMEvalKit, with fixed evaluation settings, and that Microsoft team presents them as comparison results rather than leaderboard claims. Key Takeaways Phi-4-reasoning-vision-15B is a 15B open-weight multimodal model built by combining Phi-4-Reasoning with the SigLIP-2 vision encoder in a mid-fusion architecture. Microsoft team designed the model for compact multimodal reasoning, with a focus on math, science, document understanding, and GUI grounding, rather than scaling to a much larger parameter count. High-resolution visual perception is a core part of the system, with support for dynamic resolution encoding and up to 3,600 visual tokens, which helps on dense screenshots, documents, and interface-heavy tasks. The model uses mixed reasoning and non-reasoning training, allowing it to switch between <think> and <nothink> modes depending on whether a task needs explicit reasoning or direct perception-based output. Microsoft’s reported benchmarks show strong performance for its size, including results on AI2DTEST, ChartQATEST, MathVistaMINI, OCRBench, and ScreenSpotv2, which supports its positioning as a compact but capable vision-language reasoning model. Check out the Paper, Repo and Model Weights. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Microsoft Releases Phi-4-Reasoning-Vision-15B: A Compact Multimodal Model for Math, Science, and GUI Understanding appeared first on MarkTechPost.
Google has officially released TensorFlow 2.21. The most significant update in this release is the graduation of LiteRT from its preview stage to a fully production-ready stack. Moving forward, LiteRT serves as the universal on-device inference framework, officially replacing TensorFlow Lite (TFLite). This update streamlines the deployment of machine learning models to mobile and edge devices while expanding hardware and framework compatibility. LiteRT: Performance and Hardware Acceleration When deploying models to edge devices (like smartphones or IoT hardware), inference speed and battery efficiency are primary constraints. LiteRT addresses this with updated hardware acceleration: GPU Improvements: LiteRT delivers 1.4x faster GPU performance compared to the previous TFLite framework. NPU Integration: The release introduces state-of-the-art NPU acceleration with a unified, streamlined workflow for both GPU and NPU across edge platforms. This infrastructure is specifically designed to support cross-platform GenAI deployment for open models like Gemma. Lower Precision Operations (Quantization) To run complex models on devices with limited memory, developers use a technique called quantization. This involves lowering the precision—the number of bits—used to store a neural network’s weights and activations. TensorFlow 2.21 significantly expands the tf.lite operators’ support for lower-precision data types to improve efficiency: The SQRT operator now supports int8 and int16x8. Comparison operators now support int16x8. tfl.cast now supports conversions involving INT2 and INT4. tfl.slice has added support for INT4. tfl.fully_connected now includes support for INT2. Expanded Framework Support Historically, converting models from different training frameworks into a mobile-friendly format could be difficult. LiteRT simplifies this by offering first-class PyTorch and JAX support via seamless model conversion. Developers can now train their models in PyTorch or JAX and convert them directly for on-device deployment without needing to rewrite the architecture in TensorFlow first. Maintenance, Security, and Ecosystem Focus Google is shifting its TensorFlow Core resources to focus heavily on long-term stability. The development team will now exclusively focus on: Security and bug fixes: Quickly addressing security vulnerabilities and critical bugs by releasing minor and patch versions as required. Dependency updates: Releasing minor versions to support updates to underlying dependencies, including new Python releases. Community contributions: Continuing to review and accept critical bug fixes from the open-source community. These commitments apply to the broader enterprise ecosystem, including: TF.data, TensorFlow Serving, TFX, TensorFlow Data Validation, TensorFlow Transform, TensorFlow Model Analysis, TensorFlow Recommenders, TensorFlow Text, TensorBoard, and TensorFlow Quantum. Key Takeaways LiteRT Officially Replaces TFLite: LiteRT has graduated from preview to full production, officially becoming Google’s primary on-device inference framework for deploying machine learning models to mobile and edge environments. Major GPU and NPU Acceleration: The updated runtime delivers 1.4x faster GPU performance compared to TFLite and introduces a unified workflow for NPU (Neural Processing Unit) acceleration, making it easier to run heavy GenAI workloads (like Gemma) on specialized edge hardware. Aggressive Model Quantization (INT4/INT2): To maximize memory efficiency on edge devices, tf.lite operators have expanded support for extreme lower-precision data types. This includes int8/int16 for SQRT and comparison operations, alongside INT4 and INT2 support for cast, slice, and fully_connected operators. Seamless PyTorch and JAX Interoperability: Developers are no longer locked into training with TensorFlow for edge deployment. LiteRT now provides first-class, native model conversion for both PyTorch and JAX, streamlining the pipeline from research to production. Check out the Technical details and Repo. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Google Launches TensorFlow 2.21 And LiteRT: Faster GPU Performance, New NPU Acceleration, And Seamless PyTorch Edge Deployment Upgrades appeared first on MarkTechPost.
arXiv:2603.04759v1 Announce Type: new Abstract: The limited context window of contemporary large language models (LLMs) remains a primary bottleneck for their broader application across diverse domains. Although continual pre-training on long-context data offers a straightforward solution, it incurs prohibitive data acquisition and computational costs. To address this challenge, we propose~modelname, a novel framework based on multi-grained context compression and query-aware information acquisition. SharedLLM comprises two stacked short-context LLMs: a lower model serving as a compressor and an upper model acting as a decoder. The lower model compresses long inputs into compact, multi-grained representations, which are then forwarded to the upper model for context-aware processing. To maximize efficiency, this information transfer occurs exclusively at the lowest layers, bypassing lengthy forward passes and redundant cross-attention operations. This entire process, wherein the upper and lower models are derived from the same underlying LLM layers, is termed~textit{self-injection}. To support this architecture, a specialized tree-based data structure enables the efficient encoding and query-aware retrieval of contextual information. Despite being trained on sequences of only 8K tokens, modelname~effectively generalizes to inputs exceeding 128K tokens. Across a comprehensive suite of long-context modeling and understanding benchmarks, modelname~achieves performance superior or comparable to strong baselines, striking an optimal balance between efficiency and accuracy. Furthermore, these design choices allow modelname~to substantially reduce the memory footprint and yield notable inference speedups ($2times$ over streaming and $3times$ over encoder-decoder architectures).
Stacked from One: Multi-Scale Self-Injection for Context Window Extension Leer entrada »
arXiv:2507.07999v2 Announce Type: replace-cross Abstract: Models like OpenAI-o3 pioneer visual grounded reasoning by dynamically referencing visual regions, just like human “thinking with images”. However, no benchmark exists to evaluate these capabilities holistically. To bridge this gap, we propose TreeBench (Traceable Evidence Evaluation Benchmark), a diagnostic benchmark built on three principles: (1) focused visual perception of subtle targets in complex scenes, (2) traceable evidence via bounding box evaluation, and (3) second-order reasoning to test object interactions and spatial hierarchies beyond simple object localization. Prioritizing images with dense objects, we initially sample 1K high-quality images from SA-1B, and incorporate eight LMM experts to manually annotate questions, candidate options, and answers for each image. After three stages of quality control, TreeBench consists of 405 challenging visual question-answering pairs, even the most advanced models struggle with this benchmark, where none of them reach 60% accuracy, e.g., OpenAI-o3 scores only 54.87. Furthermore, we introduce TreeVGR (Traceable Evidence Enhanced Visual Grounded Reasoning), a training paradigm to supervise localization and reasoning jointly with reinforcement learning, enabling accurate localizations and explainable reasoning pathways. Initialized from Qwen2.5-VL-7B, it improves V* Bench (+16.8), MME-RealWorld (+12.6), and TreeBench (+13.4), proving traceability is key to advancing vision-grounded reasoning. The code is available at https://github.com/Haochen-Wang409/TreeVGR.
Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodology Leer entrada »
arXiv:2603.04772v1 Announce Type: new Abstract: Despite the exceptional reasoning capabilities of Multimodal Large Language Models (MLLMs), their adaptation into universal embedding models is significantly impeded by task conflict. To address this, we propose TSEmbed, a universal multimodal embedding framework that synergizes Mixture-of-Experts (MoE) with Low-Rank Adaptation (LoRA) to explicitly disentangle conflicting task objectives. Moreover, we introduce Expert-Aware Negative Sampling (EANS), a novel strategy that leverages expert routing distributions as an intrinsic proxy for semantic similarity. By dynamically prioritizing informative hard negatives that share expert activation patterns with the query, EANS effectively sharpens the model’s discriminative power and refines embedding boundaries. To ensure training stability, we further devise a two-stage learning paradigm that solidifies expert specialization before optimizing representations via EANS. TSEmbed achieves state-of-the-art performance on both the Massive Multimodal Embedding Benchmark (MMEB) and real-world industrial production datasets, laying a foundation for task-level scaling in universal multimodal embeddings.
TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddings Leer entrada »
We use cookies to improve your experience and performance on our website. You can learn more at Política de privacidad and manage your privacy settings by clicking Settings.