Uncategorized

arXiv:2604.00027v1 Announce Type: new Abstract: Large-scale EHR prediction across institutions is hindered by substantial heterogeneity in schemas and code systems. Although Common Data Models (CDMs) can standardize records for multi-institutional learning, the manual harmonization and vocabulary mapping are costly and difficult to scale. Text-based harmonization provides an alternative by converting raw EHR into a unified textual form, enabling pooled learning without explicit standardization. However, applying this paradigm to multi-national datasets introduces an additional layer of heterogeneity, which is “language” that must be addressed for truly scalable EHRs learning. In this work, we investigate multilingual multi-institutional learning for EHR prediction, aiming to enable pooled training across multinational ICU datasets without manual standardization. We compare two practical strategies for handling language barriers: (i) directly modeling multilingual records with multilingual encoders, and (ii) translating non-English records into English via LLM-based word-level translation. Across seven public ICU datasets, ten clinical tasks with multiple prediction windows, translation-based lingual alignment yields more reliable cross-dataset performance than multilingual encoders. The multi-institutional learning model consistently outperforms strong baselines that require manual feature selection and harmonization, and also surpasses single-dataset training. We further demonstrate that text-based framework with lingual alignment effectively performs transfer learning via few-shot fine-tuning, with additional gains. To our knowledge, this is the first study to aggregate multilingual multinational ICU EHR datasets into one predictive model, providing a scalable path toward language-agnostic clinical prediction and future global multi-institutional EHR research.

arXiv:2602.23440v3 Announce Type: replace Abstract: Reinforcement learning has emerged as an effective paradigm for training large language models to interleave reasoning with search engine calls. However, existing approaches face a fundamental credit assignment problem: methods like Search-R1 assign a single outcome reward to the entire multi-step trajectory, providing no signal about which reasoning or retrieval decisions were responsible for success or failure. Process-reward methods such as StepSearch introduce step-level supervision but still sample complete trajectories independently, so advantage estimates at any given step are contaminated by the randomness of all other steps. We propose SLATE (Step-Level Advantage estimation for Truncated Exploration), which addresses both problems through two complementary ideas. First, truncated step-level sampling generates k continuations from a shared prefix, isolating all variation to a single decision point. We prove this reduces the variance of advantage estimates by up to a factor of T compared to full-trajectory sampling for T-step trajectories, the first formal variance guarantee for step-level RL in retrieval-augmented reasoning. Second, dense, decomposed process rewards separately evaluate reasoning quality, query quality, and answer correctness on a ternary scale via an LLM judge, providing richer supervision than binary outcome signals or heuristic step-level scores. Experiments on seven QA benchmarks show that SLATE consistently outperforms both sparse-reward and process-reward baselines, achieving a 7.0% relative improvement over Search-R1 on the 7B model and 30.7% on the 3B model. Gains are largest on challenging multi-hop tasks, and ablations confirm that truncated sampling and dense rewards provide complementary benefits.

arXiv:2604.00130v1 Announce Type: new Abstract: Chain-of-Thought (CoT) prompting has significantly improved the reasoning capabilities of large language models (LLMs). However, conventional CoT often relies on unstructured, flat reasoning chains that suffer from redundancy and suboptimal performance. In this work, we introduce Hierarchical Chain-of-Thought (Hi-CoT) prompting, a structured reasoning paradigm specifically designed to address the challenges of complex, multi-step reasoning. Hi-CoT decomposes the reasoning process into hierarchical substeps by alternating between instructional planning and step-by-step execution. This decomposition enables LLMs to better manage long reasoning horizons and maintain logical coherence. Extensive evaluations across diverse LLMs and mathematical reasoning benchmarks show that Hi-CoT consistently improves average accuracy by 6.2% (up to 61.4% on certain models and tasks) while reducing reasoning trace length by 13.9% compared to CoT prompting. We further show that accuracy and efficiency are maximized when models strictly adhere to the hierarchical structure. Our code is available at https://github.com/XingshuaiHuang/Hi-CoT.

In this tutorial, we build a complete end-to-end pipeline using NVIDIA Model Optimizer to train, prune, and fine-tune a deep learning model directly in Google Colab. We start by setting up the environment and preparing the CIFAR-10 dataset, then define a ResNet architecture and train it to establish a strong baseline. From there, we apply FastNAS pruning to systematically reduce the model’s complexity under FLOPs constraints while preserving performance. We also handle real-world compatibility issues, restore the optimized subnet, and fine-tune it to recover accuracy. By the end, we have a fully working workflow that takes a model from training to deployment-ready optimization, all within a single streamlined setup. Check out the Full Implementation Coding Notebook. Copy CodeCopiedUse a different Browser !pip -q install -U nvidia-modelopt torchvision torchprofile tqdm import math import os import random import time import numpy as np import torch import torch.nn as nn import torch.nn.functional as F import torchvision import torchvision.transforms as transforms from torch.utils.data import DataLoader, Subset from torchvision.models.resnet import BasicBlock from tqdm.auto import tqdm import modelopt.torch.opt as mto import modelopt.torch.prune as mtp SEED = 123 random.seed(SEED) np.random.seed(SEED) torch.manual_seed(SEED) if torch.cuda.is_available(): torch.cuda.manual_seed_all(SEED) FAST_MODE = True batch_size = 256 if FAST_MODE else 512 baseline_epochs = 20 if FAST_MODE else 120 finetune_epochs = 12 if FAST_MODE else 120 train_subset_size = 12000 if FAST_MODE else None val_subset_size = 2000 if FAST_MODE else None test_subset_size = 4000 if FAST_MODE else None target_flops = 60e6 We begin by installing all required dependencies and importing the necessary libraries to set up our environment. We initialize seeds to ensure reproducibility and configure the device to leverage a GPU if available. We also define key runtime parameters, such as batch size, epochs, dataset subsets, and FLOP constraints, to control the overall experiment. Copy CodeCopiedUse a different Browser def seed_worker(worker_id): worker_seed = SEED + worker_id np.random.seed(worker_seed) random.seed(worker_seed) def build_cifar10_loaders(train_batch_size=256, train_subset_size=None, val_subset_size=None, test_subset_size=None): normalize = transforms.Normalize( mean=[0.4914, 0.4822, 0.4465], std=[0.2470, 0.2435, 0.2616], ) train_transform = transforms.Compose([ transforms.ToTensor(), transforms.RandomHorizontalFlip(), transforms.RandomCrop(32, padding=4), normalize, ]) eval_transform = transforms.Compose([ transforms.ToTensor(), normalize, ]) train_full = torchvision.datasets.CIFAR10( root=”./data”, train=True, transform=train_transform, download=True ) val_full = torchvision.datasets.CIFAR10( root=”./data”, train=True, transform=eval_transform, download=True ) test_full = torchvision.datasets.CIFAR10( root=”./data”, train=False, transform=eval_transform, download=True ) n_trainval = len(train_full) ids = np.arange(n_trainval) np.random.shuffle(ids) n_train = int(n_trainval * 0.9) train_ids = ids[:n_train] val_ids = ids[n_train:] if train_subset_size is not None: train_ids = train_ids[:min(train_subset_size, len(train_ids))] if val_subset_size is not None: val_ids = val_ids[:min(val_subset_size, len(val_ids))] test_ids = np.arange(len(test_full)) if test_subset_size is not None: test_ids = test_ids[:min(test_subset_size, len(test_ids))] train_ds = Subset(train_full, train_ids.tolist()) val_ds = Subset(val_full, val_ids.tolist()) test_ds = Subset(test_full, test_ids.tolist()) num_workers = min(2, os.cpu_count() or 1) g = torch.Generator() g.manual_seed(SEED) train_loader = DataLoader( train_ds, batch_size=train_batch_size, shuffle=True, num_workers=num_workers, pin_memory=torch.cuda.is_available(), worker_init_fn=seed_worker, generator=g, ) val_loader = DataLoader( val_ds, batch_size=512, shuffle=False, num_workers=num_workers, pin_memory=torch.cuda.is_available(), worker_init_fn=seed_worker, ) test_loader = DataLoader( test_ds, batch_size=512, shuffle=False, num_workers=num_workers, pin_memory=torch.cuda.is_available(), worker_init_fn=seed_worker, ) print(f”Train: {len(train_ds)} | Val: {len(val_ds)} | Test: {len(test_ds)}”) return train_loader, val_loader, test_loader train_loader, val_loader, test_loader = build_cifar10_loaders( train_batch_size=batch_size, train_subset_size=train_subset_size, val_subset_size=val_subset_size, test_subset_size=test_subset_size, ) We construct the full data pipeline by preparing CIFAR-10 datasets with appropriate augmentations and normalization. We split the dataset to reduce its size and speed up experimentation. We then create efficient data loaders that ensure proper batching, shuffling, and reproducible data handling. Copy CodeCopiedUse a different Browser def _weights_init(m): if isinstance(m, (nn.Linear, nn.Conv2d)): nn.init.kaiming_normal_(m.weight) class LambdaLayer(nn.Module): def __init__(self, lambd): super().__init__() self.lambd = lambd def forward(self, x): return self.lambd(x) class ResNet(nn.Module): def __init__(self, num_blocks, num_classes=10): super().__init__() self.in_planes = 16 self.layers = nn.Sequential( nn.Conv2d(3, 16, kernel_size=3, stride=1, padding=1, bias=False), nn.BatchNorm2d(16), nn.ReLU(), self._make_layer(16, num_blocks, stride=1), self._make_layer(32, num_blocks, stride=2), self._make_layer(64, num_blocks, stride=2), nn.AdaptiveAvgPool2d((1, 1)), nn.Flatten(), nn.Linear(64, num_classes), ) self.apply(_weights_init) def _make_layer(self, planes, num_blocks, stride): strides = [stride] + [1] * (num_blocks – 1) layers = [] for s in strides: downsample = None if s != 1 or self.in_planes != planes: downsample = LambdaLayer( lambda x: F.pad( x[:, :, ::2, ::2], (0, 0, 0, 0, planes // 4, planes // 4), “constant”, 0, ) ) layers.append(BasicBlock(self.in_planes, planes, s, downsample)) self.in_planes = planes return nn.Sequential(*layers) def forward(self, x): return self.layers(x) def resnet20(): return ResNet(num_blocks=3).to(device) We define the ResNet20 architecture from scratch, including custom initialization and shortcut handling through lambda layers. We structure the network using convolutional blocks and residual connections to capture hierarchical features. We finally encapsulate the model creation into a reusable function that moves it directly to the selected device. Copy CodeCopiedUse a different Browser class CosineLRwithWarmup(torch.optim.lr_scheduler._LRScheduler): def __init__(self, optimizer, warmup_steps, decay_steps, warmup_lr=0.0, last_epoch=-1): self.warmup_steps = warmup_steps self.warmup_lr = warmup_lr self.decay_steps = max(decay_steps, 1) super().__init__(optimizer, last_epoch) def get_lr(self): if self.last_epoch < self.warmup_steps: return [ (base_lr – self.warmup_lr) * self.last_epoch / max(self.warmup_steps, 1) + self.warmup_lr for base_lr in self.base_lrs ] current_steps = self.last_epoch – self.warmup_steps return [ 0.5 * base_lr * (1 + math.cos(math.pi * current_steps / self.decay_steps)) for base_lr in self.base_lrs ] def get_optimizer_scheduler(model, lr, weight_decay, warmup_steps, decay_steps): optimizer = torch.optim.SGD( filter(lambda p: p.requires_grad, model.parameters()), lr=lr, momentum=0.9, weight_decay=weight_decay, ) scheduler = CosineLRwithWarmup(optimizer, warmup_steps, decay_steps) return optimizer, scheduler def loss_fn_default(model, outputs, labels): return F.cross_entropy(outputs, labels) def train_one_epoch(model, loader, optimizer, scheduler, loss_fn=loss_fn_default): model.train() running_loss = 0.0 total = 0 for images, labels in loader: images = images.to(device, non_blocking=True) labels = labels.to(device, non_blocking=True) outputs = model(images) loss = loss_fn(model, outputs, labels) optimizer.zero_grad(set_to_none=True) loss.backward() optimizer.step() scheduler.step() running_loss += loss.item() * labels.size(0) total += labels.size(0) return running_loss / max(total, 1) @torch.no_grad() def evaluate(model, loader): model.eval() correct = 0 total = 0 for images, labels in loader: images = images.to(device, non_blocking=True) labels = labels.to(device, non_blocking=True) logits = model(images) preds = logits.argmax(dim=1) correct += (preds == labels).sum().item() total += labels.size(0) return 100.0 * correct / max(total, 1) def train_model(model, train_loader, val_loader, epochs, ckpt_path, lr=None, weight_decay=1e-4, print_every=1): if lr is None: lr = 0.1 * batch_size / 128 steps_per_epoch = len(train_loader) warmup_steps = max(1, 2 * steps_per_epoch if FAST_MODE else 5 * steps_per_epoch) decay_steps = max(1, epochs * steps_per_epoch) optimizer, scheduler = get_optimizer_scheduler( model=model, lr=lr, weight_decay=weight_decay, warmup_steps=warmup_steps, decay_steps=decay_steps, ) best_val = -1.0 best_epoch = -1 print(f”Training for {epochs} epochs…”) for epoch in tqdm(range(1, epochs + 1)): train_loss = train_one_epoch(model, train_loader, optimizer, scheduler) val_acc = evaluate(model, val_loader) if val_acc >= best_val: best_val = val_acc best_epoch = epoch torch.save(model.state_dict(), ckpt_path) if epoch == 1

arXiv:2604.01029v1 Announce Type: cross Abstract: Multi-LLM revision pipelines, in which a second model reviews and improves a draft produced by a first, are widely assumed to derive their gains from genuine error correction. We question this assumption with a controlled decomposition experiment that uses four matched conditions to separate second-pass gains into three additive components: re-solving, scaffold, and content. We evaluate this design across two model pairs on three benchmarks spanning knowledge-intensive MCQ and competitive programming. Our results show that the gains of multi-LLM revision are not monolithic, but depend on task structure, draft quality, and the type of draft information. On MCQ tasks, where the answer space is constrained and drafts provide little structural guidance, most gains are consistent with stronger-model re-solving, and directly routing queries to the stronger model can be more effective than revising a weak draft. On code generation tasks, however, two-stage prompting remains useful because even semantically null drafts can provide substantial structural scaffolding, while weak draft content can be harmful. Finally, role-reversed experiments show that strong drafts clearly benefit weak reviewers. Ultimately, our findings demonstrate that the utility of multi-LLM revision is dynamically bottlenecked by task structure and draft quality, necessitating more targeted pipeline designs rather than blanket revision strategies.

arXiv:2604.01113v1 Announce Type: new Abstract: Large language model (LLM) systems are increasingly used to support high-stakes decision-making, but they typically perform worse when the available evidence is internally inconsistent. Such a scenario exists in real-world healthcare settings, with patient-reported symptoms contradicting medical signs. To study this problem, we introduce MIMIC-DOS, a dataset for short-horizon organ dysfunction worsening prediction in the intensive care unit (ICU) setting. We derive this dataset from the widely recognized MIMIC-IV, a publicly available electronic health record dataset, and construct it exclusively from cases in which discordance between signs and symptoms exists. This setting poses a substantial challenge for existing LLM-based approaches, with single-pass LLMs and agentic pipelines often struggling to reconcile such conflicting signals. To address this problem, we propose CARE: a multi-stage privacy-compliant agentic reasoning framework in which a remote LLM provides guidance by generating structured categories and transitions without accessing sensitive patient data, while a local LLM uses these categories and transitions to support evidence acquisition and final decision-making. Empirically, CARE achieves stronger performance across all key metrics compared to multiple baseline settings, showing that CARE can more robustly handle conflicting clinical evidence while preserving privacy.

As the war in Iran continues to engulf the Middle East and the Strait of Hormuz stays closed, one of the most visible global economic ripple effects has been fossil-fuel prices. In particular, you can’t get away from news about the price of gasoline, which just topped an average of $4 a gallon in the US, its highest level since 2022. But looking ahead, further consequences for the global economy could be looming in plastics. Plastics are made using petrochemicals, and the supply chain impacts of the oil bottleneck near Iran are starting to build up.  Plastic production accounts for roughly 5% of global carbon dioxide emissions today. And our current moment shows just how embedded oil and gas products are in our lives. It goes far beyond their use for energy.  As I write this, I’m wearing clothes that contain plastic fibers, typing on a plastic keyboard, and looking through the plastic lenses of my glasses. It’s hard to imagine what our world looks like without plastic. And in some ways, moving away from fossil-derived plastic could prove even more complicated than decarbonizing our energy system.  Crude oil prices have been on a roller-coaster in recent weeks, and prices have recently topped $100 a barrel. Crude oil contains a huge range of hydrocarbons, and it’s typically refined by putting it through a distillation unit that separates the raw material into different fractions according to their boiling point. Those fractions then go on to be further processed into everything from jet fuel to asphalt binder. We’ve already seen the price spikes for some materials pulled out of crude oil, like gasoline and jet fuel. Let’s zoom in on another component, naphtha. It can be added to gasoline and jet fuel to improve performance. It can also be used as a solvent or as a raw material to make plastics. The Middle East currently accounts for about 20% of global naphtha production­ and supplies about 40% of the market in Asia, where prices are already up by 50% over the last month. We’re starting to see these effects trickle down already. The price of polypropylene (which is made from naphtha and used for food containers, bottle caps, and even automotive parts) is climbing, especially in Asia.   Typically, manufacturers have a bit of stock built up, but that’ll be exhausted soon, likely in the coming weeks. The largest supplier of water bottles in India recently announced that it would raise prices by 11% after its packaging costs went up by over 70%, according to reporting from Reuters. Toys could be more expensive this holiday season as manufacturers grapple with supply chain concerns. Americans will likely feel these ripples especially hard if disruptions continue. The average US resident used over 250 kilograms of new plastics in 2019, according to a 2022 report from the Organization for Economic Cooperation and Development. That’s an absolutely massive number—the global average is just 60 kilograms. The effects of higher prices for both fuels and feedstocks could compound and multiply, and alternatives aren’t widely available. Bio-based plastics made with materials like plant sugars exist, but they still make up a vanishingly tiny portion of the market. As of 2025, global plastics production totaled over 431 million metric tons per year. Bio-based and bio-degradable plastics made up about 0.5% of that, a share that could reach 1% by 2030. Bio-based plastics are much more expensive than their fossil-derived counterparts. And many are made using agricultural raw materials, so scaling them up too much could be harmful for the environment and might compete with other industries like food production. Recycling isn’t the easy answer either. Mechanical recycling is the current standard method used for materials like the plastics that make up water bottles and disposable coffee cups. But that degrades the materials over time, so they can’t be used infinitely. Chemical recycling has its own host of issues—the facilities that do it can be highly polluting, and today plastics that go into advanced recycling plants largely don’t actually go into new plastics. There’s been a lot of talk in recent weeks about how this energy crisis is going to push the world more toward renewable energy. Solar panels, electric vehicles, and batteries could suddenly become more attractive as we face the drastic consequences of a disruption in the global fossil-fuel supply. But when it comes to plastic, the future looks far more complicated. Even though the plastics industry is facing much the same disruptions as the energy sector, there aren’t the same obvious alternatives available for a transition. Our lives are tied up in plastic, with uses ranging from the essential (like medical equipment) to the mundane (my to-go coffee cup). Soon, our economy could feel the effects of just how much we rely on fossil-derived plastics, and how hard it’s going to be to replace them.  This article is from The Spark, MIT Technology Review’s weekly climate newsletter. To receive it in your inbox every Wednesday, sign up here.