Uncategorized

NVIDIA researchers introduced ProRL AGENT, a scalable infrastructure designed for reinforcement learning (RL) training of multi-turn LLM agents. By adopting a ‘Rollout-as-a-Service’ philosophy, the system decouples agentic rollout orchestration from the training loop. This architectural shift addresses the inherent resource conflicts between I/O-intensive environment interactions and GPU-intensive policy updates that currently bottleneck agent development. The Core Problem: Tight Coupling Multi-turn agent tasks involve interacting with external environments, such as code repositories or operating systems, via iterative tool use. Many existing frameworks—including SkyRL, VeRL-Tool, Agent Lightning, rLLM, and GEM—embed rollout control directly within the training process. This tight coupling leads to two primary limitations: Conflicting System Requirements: Rollouts are I/O-bound, requiring sandbox creation, long-lived tool sessions, and asynchronous coordination. Training is GPU-intensive, centered on forward/backward passes and gradient synchronization. Running both in one process causes interference and reduces hardware efficiency. Maintenance Barriers: Embedding rollout logic in the trainer makes it difficult to migrate to different training backends or support new runtime environments without re-implementing the execution pipeline. https://arxiv.org/pdf/2603.18815 System Design: Rollout-as-a-Service ProRL AGENT operates as a standalone HTTP service that manages the full rollout lifecycle. The RL trainer interacts with the server solely through an API, remaining agnostic to the underlying rollout infrastructure. Three-Stage Asynchronous Pipeline To maximize throughput, the server orchestrates rollouts through an asynchronous three-stage ‘assembly line’: INIT: Initialization workers spin up sandbox containers and configure tools. RUN: Rollout workers drive the multi-turn agent loop and collect trajectories. EVAL: Evaluation workers score results against ground truth to produce reward signals. By assigning each stage to an independent worker pool, ProRL AGENT allows phases to overlap across different jobs, preventing slow evaluations (such as full test suite executions) from stalling the rollout process. https://arxiv.org/pdf/2603.18815 HPC-Compatible Sandboxing and Optimized Tools ProRL AGENT utilizes Singularity for its sandbox infrastructure. Unlike Docker-based platforms, Singularity allows rootless execution, which is required for deployment on shared HPC clusters managed by Slurm. The system includes several optimizations to reduce tool execution latency, which often dominates total rollout time: Efficient Bash: Replaces tmux-based terminal multiplexing with a ptyprocess-based direct pseudo-terminal, reducing shell command latency from 0.78s to 0.42s. Direct IPython API: Connects to persistent kernels via an in-process API instead of network gateways, removing networking overhead. Unix Domain Sockets (UDS): Replaces TCP loopback for communication between the agent and the execution server inside the container to shave off additional latency. Advanced Features for Scalable RL The infrastructure introduces mechanisms to improve training stability and hardware utilization: Load Balancing and Prefix Cache Reuse The server manages a pool of LLM inference backends (e.g., vLLM) using a min-heap keyed by assignment counts. When a task is assigned, all subsequent calls within that task are routed to the same backend. This strategy maximizes prefix cache reuse, reducing inference time across multiple agent turns. Token-in/Token-out Communication To eliminate re-tokenization drift—where the token sequence generated during rollout differs from what is used during training—ProRL AGENT uses token IDs as the canonical representation throughout the entire process. Log-probabilities and IDs are propagated unchanged from the inference backend to the trainer. Optimized DAPO Implementation The system supports Dynamic Sampling Policy Optimization (DAPO), which filters out ‘non-informative’ prompts that yield uniform rewards. ProRL AGENT uses an asynchronous replenishment mechanism to maintain maximum throughput, terminating redundant active jobs early once the target number of informative prompts is reached. Experimental Results on SWE-Bench Verified The system was validated using Qwen3 models across multiple scales. ProRL AGENT consistently improved performance compared to reproduced baselines. Model Scale Reproduced Baseline ProRL Agent (RL) Qwen3-4B 14.8 21.2 Qwen3-8B 9.6 18.0 Qwen3-14B 15.4 (reproduced baseline) 23.6 Note: The reported prior result for SkyRL-Agent-14B-v0 was 21.6. In addition to software engineering, the system demonstrated generality in STEM, Math, and Code domains, showing steady reward growth during RL training. Scalability tests confirmed that rollout throughput increases near-linearly as compute nodes are added. Key Takeaways Architectural Decoupling: ProRL Agent treats the full agentic rollout lifecycle—including environment initialization, tool execution, and reward scoring—as an independent HTTP service, separating I/O-intensive tasks from GPU-intensive policy training. Significant Performance Gains: This infrastructure enabled the Qwen3-8B model to nearly double its performance on the SWE-Bench Verified benchmark (from 9.6% to 18.0%), while the Qwen3-14B model improved from 15.4% to 23.6%. System Latency Reductions: Targeted optimizations, such as replacing tmux with ptyprocess for shell execution, reduced action latency from 0.78s to 0.42s, contributing to near-linear throughput scaling across compute nodes. Elimination of Tokenization Drift: The framework utilizes a token-in/token-out communication pipeline, ensuring that the exact token IDs generated during rollout are passed to the trainer without the risk of lossy re-tokenization. HPC-Native Deployment: By using Singularity instead of Docker, ProRL Agent supports rootless execution and native Slurm integration, allowing large-scale agent training on shared high-performance computing clusters. Check out the Paper and Repo. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post NVIDIA AI Unveils ProRL Agent: A Decoupled Rollout-as-a-Service Infrastructure for Reinforcement Learning of Multi-Turn LLM Agents at Scale appeared first on MarkTechPost.

“Think of this as a human body,” says Javier González. In front of me is essentially a metal box on wheels. Standing at around a meter in height, it reminds me of a stainless-steel counter in a restaurant kitchen. It is covered in flexible plastic tubing—which act as veins and arteries—connecting a series of transparent containers, the organs of this machine. What makes it extra special is the role of the cream-colored tub that sits on its surface. Ten months ago, González, a biomedical scientist who developed the device with his colleagues at the Carlos Simon Foundation, carefully placed a freshly donated human uterus in the tub. The team connected it to the device’s tubes and pumped in modified human blood. The device kept the uterus alive for a day—a new feat that could represent the first step to the long-term maintenance of uteruses outside the human body. The work has not yet been published.  The team members want to keep donated human uteruses alive long enough to see a full menstrual cycle. They hope this will help them study diseases of the uterus and learn more about how embryos burrow their way into the organ’s lining at the start of a pregnancy. They also hope that future iterations of their device might one day sustain the full gestation of a human fetus. The machine is technically called PUPER, which stands for “preservation of the uterus in perfusion.” But González’s colleague Xavier Santamaria says the team has adopted a nickname for it: “We call it ‘Mother.’” The organ in the machine González and Santamaria, medical vice president of the Carlos Simon Foundation, demonstrated how the device might work when I visited the foundation in Valencia, Spain, earlier this month (although it held no organs on that day).  Both are interested in learning more about implantation, the moment at which an embryo attaches itself to the lining of a uterus—essentially, the very first moment of pregnancy. The foundation’s founder and director, Carlos Simon, believes it’s a sticking point in IVF: Scientists have made many improvements to the technology over the years, but the failure of embryos to implant underlies plenty of unsuccessful IVF cycles, he says. Being able to carefully study how the process works in a real, living organ might give the team a better idea of how to prevent those failures. JESS HAMZELOU JAVIER GONZALES/CARLOS SIMON FOUNDATION Javier González demonstrates the perfusion machine. A previous iteration of the device kept a sheep’s uterus (right) alive for a day. The team took inspiration from advances in technologies designed to maintain donated organs for transplantation. In recent years, researchers around the world have created devices that deliver nutrients and filter waste so that organs can survive longer after being removed from donors’ bodies. The main goal here is to buy time. A human organ might last only a matter of hours outside the body, so a transplant may require frantic preparation for the recipient, sometimes in the middle of the night. With a little more time, doctors could find better donor-patient matches and potentially test the quality of donated organs. This approach is called normothermic or machine perfusion, and it is already being used clinically for some liver, kidney, and heart transplants. The team at the Carlos Simon Foundation built a similar machine for uteruses. A blood bag hangs on one side. From there, blood is ferried via plastic tubing to a pump, which functions as the heart. The pump shunts the blood through an oxygenator, which adds oxygen and removes carbon dioxide as the lungs would in a human body. The blood is warmed and passed through sensors that monitor the levels of glucose and oxygen, along with other factors. It passes through a “kidney” to remove waste. And finally the blood reaches the uterus, hooked up to its own plastic “arteries” and “veins.” The organ itself sits at a tilt, just as in the body, and is kept in a humid environment to stay moist. Mother’s first uterus The team first began testing an early prototype of the device with sheep uteruses around four years ago. That meant carting the machine to an animal research center in Zaragoza, around 200 miles away. Over the course of the preliminary study, veterinary surgeons removed the uteruses of six sheep and hooked them up to the machine. They kept each uterus alive for a day, using blood from the same animals. After the sheep experiments, the researchers carted their machine back to Valencia and modified it to achieve its current incarnation, “Mother.” They started working with a local hospital that performed hysterectomies. And in May last year, they were offered their first human uterus. The team needed to be quick. “You need to put [the uterus in the machine] within a couple of hours, maximum, of the extraction,” says Santamaria. He and his colleagues also needed to connect the uterus’s blood vessels to the tubing delicately, taking care to avoid any blockages (clotting is a major challenge in organ perfusion). The organ was hooked up to human blood obtained from a blood bank. It seemed to work—at least temporarily. “We kept it alive for one day,” says Santamaria. “As a proof of concept, it is impressive,” says Keren Ladin, a bioethicist who has focused on organ transplantation and perfusion at Tufts University. “These are early days.” It might not sound like much, but 24 hours is a long time for an organ to be out of the body. Maintaining a donated uterus for that long could expand the options for uterus transplant, a fairly new procedure offered to some people who want to be pregnant but don’t have a functional uterus, says Gerald Brandacher, professor of experimental and translational transplant surgery at the Medical University of Innsbruck in Austria. “It is better than what we currently have, because we have only a couple of hours,” he says. So far, most uterus transplants have been planned operations involving organs from living donors.

Neuroscience has long been a field of divide and conquer. Researchers typically map specific cognitive functions to isolated brain regions—like motion to area V5 or faces to the fusiform gyrus—using models tailored to narrow experimental paradigms. While this has provided deep insights, the resulting landscape is fragmented, lacking a unified framework to explain how the human brain integrates multisensory information. Meta’s FAIR team has introduced TRIBE v2, a tri-modal foundation model designed to bridge this gap. By aligning the latent representations of state-of-the-art AI architectures with human brain activity, TRIBE v2 predicts high-resolution fMRI responses across diverse naturalistic and experimental conditions. https://ai.meta.com/research/publications/a-foundation-model-of-vision-audition-and-language-for-in-silico-neuroscience/ The Architecture: Multi-modal Integration TRIBE v2 does not learn to ‘see’ or ‘hear’ from scratch. Instead, it leverages the representational alignment between deep neural networks and the primate brain. The architecture consists of three frozen foundation models serving as feature extractors, a temporal transformer, and a subject-specific prediction block. 1. Feature Extraction The model processes stimuli through three specialized encoders: Text: Contextualized embeddings are extracted from LLaMA 3.2-3B. For every word, the model prepends the preceding 1,024 words to provide temporal context, which is then mapped to a 2 Hz grid. Video: The model uses V-JEPA2-Giant to process 64-frame segments spanning the preceding 4 seconds for each time-bin. Audio: Sound is processed through Wav2Vec-BERT 2.0, with representations resampled to 2 Hz to match the stimulus frequency (fstim) (f_{stim}). 2. Temporal Aggregation The resulting embeddings are compressed into a shared dimension (D=384)(D=384) and concatenated to form a multi-modal time series with a model dimension of Dmodel=3×384=1152D_{model} = 3 times 384 = 1152. This sequence is fed into a Transformer encoder (8 layers, 8 attention heads) that exchanges information across a 100-second window. 3. Subject-Specific Prediction To predict brain activity, the Transformer outputs are decimated to the 1 Hz fMRI frequency (ffMRI)(f_{fMRI}) and passed through a Subject Block. This block projects the latent representations to 20,484 cortical vertices (fsaverage5surface)(fsaverage5 surface) and 8,802 subcortical voxels. Data and Scaling Laws A significant hurdle in brain encoding is data scarcity. TRIBE v2 addresses this by utilizing ‘deep’ datasets for training—where a few subjects are recorded for many hours—and ‘wide’ datasets for evaluation. Training: The model was trained on 451.6 hours of fMRI data from 25 subjects across four naturalistic studies (movies, podcasts, and silent videos). Evaluation: It was evaluated across a broader collection totaling 1,117.7 hours from 720 subjects. The research team observed a log-linear increase in encoding accuracy as the training data volume increased, with no evidence of a plateau. This suggests that as neuroimaging repositories expand, the predictive power of models like TRIBE v2 will continue to scale. Results: Beating the Baselines TRIBE v2 significantly outperforms traditional Finite Impulse Response (FIR) models, the long-standing gold standard for voxel-wise encoding. Zero-Shot and Group Performance One of the model’s most striking capabilities is zero-shot generalization to new subjects. Using an ‘unseen subject’ layer, TRIBE v2 can predict the group-averaged response of a new cohort more accurately than the actual recording of many individual subjects within that cohort. In the high-resolution Human Connectome Project (HCP) 7T dataset, TRIBE v2 achieved a group correlation (Rgroup) (R_{group}) near 0.4, a two-fold improvement over the median subject’s group-predictivity. Fine-Tuning When given a small amount of data (at most one hour) for a new participant, fine-tuning TRIBE v2 for just one epoch leads to a two- to four-fold improvement over linear models trained from scratch. In-Silico Experimentation The research team argue that TRIBE v2 could be useful for piloting or pre-screening neuroimaging studies. By running virtual experiments on the Individual Brain Charting (IBC) dataset, the model recovered classic functional landmarks: Vision: It accurately localized the fusiform face area (FFA) and parahippocampal place area (PPA). Language: It successfully recovered the temporo-parietal junction (TPJ) for emotional processing and Broca’s area for syntax. Furthermore, applying Independent Component Analysis (ICA) to the model’s final layer revealed that TRIBE v2 naturally learns five well-known functional networks: primary auditory, language, motion, default mode, and visual. https://aidemos.atmeta.com/tribev2/ Key Takeaway A Powerhouse Tri-modal Architecture: TRIBE v2 is a foundation model that integrates video, audio, and language by leveraging state-of-the-art encoders like LLaMA 3.2 for text, V-JEPA2 for video, and Wav2Vec-BERT for audio. Log-Linear Scaling Laws: Much like the Large Language Models we use every day, TRIBE v2 follows a log-linear scaling law; its ability to accurately predict brain activity increases steadily as it is fed more fMRI data, with no performance plateau currently in sight. Superior Zero-Shot Generalization: The model can predict the brain responses of unseen subjects in new experimental conditions without any additional training. Remarkably, its zero-shot predictions are often more accurate at estimating group-averaged brain responses than the recordings of individual human subjects themselves. The Dawn of In-Silico Neuroscience: TRIBE v2 enables ‘in-silico’ experimentation, allowing researchers to run virtual neuroscientific tests on a computer. It successfully replicated decades of empirical research by identifying specialized areas like the fusiform face area (FFA) and Broca’s area purely through digital simulation. Emergent Biological Interpretability: Even though it’s a deep learning ‘black box,’ the model’s internal representations naturally organized themselves into five well-known functional networks: primary auditory, language, motion, default mode, and visual. Check out the Code, Weights and Demo. Also, feel free to follow us on Twitter and don’t forget to join our 120k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Meta Releases TRIBE v2: A Brain Encoding Model That Predicts fMRI Responses Across Video, Audio, and Text Stimuli appeared first on MarkTechPost.

Creating an AI agent for tasks like analyzing and processing documents autonomously used to require hours of near-endless configuration, code orchestration, and deployment battles.

This is today’s edition of The Download, our weekday newsletter that provides a daily dose of what’s going on in the world of technology. How a couple of ski bums built the internet’s best weather app  The best snow-forecasting app for skiers isn’t a federally-funded service or a big-name brand. It’s OpenSnow, a startup that uses government data, its own AI models, and decades of alpine-life experience to deliver the best predictions out there.  The app has proved especially vital this winter, one of the weirdest on record. It’s even made microcelebrities of its forecasters, who sift through reams of data to write “Daily Snow” reports for locations around the world.   We headed to the Tahoe mountains to hear how two broke ski bums became modern-day snow gods. Read the full story.  —Rachel Levin  Here’s why some people choose cryonics to store their bodies and brains after death  —Jessica Hamzelou  This week I reported on unusual research focused on the frozen brain of L. Stephen Coles.  Coles, a researcher who studied aging, was interested in cryonics—the long-term storage of human bodies and brains in the hope that they might one day be brought back to life. It’s a hope shared by many.  Over the past few years, I’ve spoken to people who run cryonics facilities, study cryopreservation, or just want to be cryogenically stored. All of them acknowledge that there’s a vanishingly small chance of being brought back to life. So why do they do it?  Read the full story to find out.  This article is from The Checkup, our weekly biotech newsletter. Sign up to receive it in your inbox every Thursday.  What’s next for space exploration?   Whether it’s the race to find life on Mars, the campaign to outsmart killer asteroids, or the quest to make the moon a permanent home to astronauts, scientists’ efforts in space can tell us more about where humanity is headed. To learn more about the progress and possibilities ahead, our features editor Amanda Silverman sat down with Robin George Andrews, an award-winning science journalist and author, on Wednesday. If you missed their conversation, fear not—you can catch up and watch the video here. You’ll need to be a subscriber to access it, but the good news is subscriptions are discounted right now. Bag yours if you haven’t already!  The must-reads  I’ve combed the internet to find you today’s most fun/important/scary/fascinating stories about technology.  1 The Pentagon’s ban on Anthropic has been halted A judge has paused its designation as a supply chain risk. (CBS News)  + She said the government was trying to “chill public debate.” (BBC) + Sam Altman claimed he tried to “save” Anthropic in the clash. (Axios)  2 Elon Musk has lost his lawsuit against an ad boycott on X A judge admonished the “fishing expedition.” (Ars Technica) + Ad revenue fell by more than half as advertisers fled X after Musk took over. (BBC)  3 OpenAI has put plans for an erotic chatbot on hold “indefinitely” Staff and investors had raised concerns. (The Information $) + The company is making a sharp strategic pivot. (FT $) + AI companions are the final stage of digital addiction. (MIT Technology Review)  4 A helium shortage has started impacting tech supply chains The problem stems from the Middle East conflict. (Reuters) + The era of cheap helium is over. (MIT Technology Review)  5 Trump’s new science advisers: 12 tech chiefs and just one academic They include at least nine billionaires. (Nature) + David Sacks is stepping down as Trump’s crypto and AI czar. (TechCrunch)  6 Anthropic is mulling an IPO as soon as October It’s racing OpenAI to hold an initial public offering. (Bloomberg $)  7 Wikipedia has banned all AI-generated content  LLM-related issues had overwhelmed editors. (404 Media) + Here’s what we’re getting wrong about AI’s truth crisis. (MIT Technology Review)  8 OpenAI’s ad pilot generated $100 million in under 2 months More than 600 advertisers are working on the trial. (CNBC) + Ads will arrive on ChatGPT free ‌and Go in the coming weeks. (Reuters)   9 An Irish village is giving kids a phone-free upbringing The ban works because almost everyone’s bought in. (NYT $)  10 Chatting with sycophantic AI makes you less kind New research found it encourages “uncouth behavior.” (Nature)  Quote of the day  “I don’t know if it’s ‘murder,’ but it looks like an attempt to cripple Anthropic.”  —Judge Rita Lin rules against the Pentagon’s ban on Anthropic, The Verge reports.  One More Thing  AURELIA INSTITUTE This futuristic space habitat is designed to self-assemble in orbit   More and more people are traveling beyond Earth, but the International Space Station can only hold 11 of them at a time.   Aurelia Institute, an architecture R&D lab based in Cambridge, MA, is building a solution: a habitat that launches in compact stacks of flat tiles—and self-assembles in orbit.   The concept may sound far-fetched, but it’s already won support from NASA. Read the full story.  —Sarah Ward  We can still have nice things  A place for comfort, fun and distraction to brighten up your day. (Got any ideas? Drop me a line.)  + These optical illusions are absolute brain-melters. + The web design museum lovingly visualizes the evolution of the internet. + Zara Picken’s modernist illustrations are a new window into the mid-20th century. + Explore our planet’s connections through the digital Knowledge Garden.