Uncategorized

Designing algorithms for Multi-Agent Reinforcement Learning (MARL) in imperfect-information games — scenarios where players act sequentially and cannot see each other’s private information, like poker — has historically relied on manual iteration. Researchers identify weighting schemes, discounting rules, and equilibrium solvers through intuition and trial-and-error. Google DeepMind researchers proposes AlphaEvolve, an LLM-powered evolutionary coding agent that replaces that manual process with automated search. The research team applies this framework to two established paradigms: Counterfactual Regret Minimization (CFR) and Policy Space Response Oracles (PSRO). In both cases, the system discovers new algorithm variants that perform competitively against or better than existing hand-designed state-of-the-art baselines. All experiments were run using the OpenSpiel framework. Background: CFR AND PSRO CFR is an iterative algorithm that decomposes regret minimization across information sets. At each iteration it accumulates ‘counterfactual regret’ — how much a player would have gained by playing differently — and derives a new policy proportional to positive accumulated regret. Over many iterations, the time-averaged strategy converges to a Nash Equilibrium (NE). Variants like DCFR (Discounted CFR) and PCFR+ (Predictive CFR+) improve convergence by applying specific discounting or predictive update rules, all developed through manual design. PSRO operates at a higher level of abstraction. It maintains a population of policies for each player, builds a payoff tensor (the meta-game) by computing expected utilities for every combination of population policies, and then uses a meta-strategy solver to produce a probability distribution over the population. Best responses are trained against that distribution and added to the population iteratively. The meta-strategy solver — how the population distribution is computed — is the central design choice that the paper targets for automated discovery. All experiments use an exact best response oracle (computed via value iteration) and exact payoff values for all meta-game entries, removing Monte Carlo sampling noise from the results. THE AlphaEvolve FRAMEWORK AlphaEvolve is a distributed evolutionary system that uses LLMs to mutate source code rather than numeric parameters. The process: a population is initialized with a standard implementation (CFR+ as the seed for CFR experiments; Uniform as the seed for both PSRO solver classes). At each generation, a parent algorithm is selected based on fitness; its source code is passed to an LLM (Gemini 2.5 Pro) with a prompt to modify it; the resulting candidate is evaluated on proxy games; valid candidates are added to the population. AlphaEvolve supports multi-objective optimization — if multiple fitness metrics are defined, one is randomly selected per generation to guide parent sampling. The fitness signal is negative exploitability after K iterations, evaluated on a fixed set of training games: 3-player Kuhn Poker, 2-player Leduc Poker, 4-card Goofspiel, and 5-sided Liars Dice. Final evaluation is done on a separate test set of larger, unseen games. For CFR, the evolvable search space consists of three Python classes: RegretAccumulator, PolicyFromRegretAccumulator, and PolicyAccumulator. These govern regret accumulation, current policy derivation, and average policy accumulation respectively. The interface is expressive enough to represent all known CFR variants as special cases. For PSRO, the evolvable components are TrainMetaStrategySolverand EvalMetaStrategySolver— the meta-strategy solvers used during oracle training and during exploitability evaluation. Discovered Algorithm 1: VAD-CFR The evolved CFR variant is Volatility-Adaptive Discounted CFR (VAD-CFR). Rather than the linear averaging and static discounting used in the CFR family, the search produced three distinct mechanisms: Volatility-adaptive discounting. Instead of fixed discount factors α and β applied to cumulative regrets (as in DCFR), VAD-CFR tracks the volatility of the learning process using an Exponential Weighted Moving Average (EWMA) of the instantaneous regret magnitude. When volatility is high, discounting increases so the algorithm forgets unstable history faster; when volatility drops it retains more history. The EWMA decay factor is 0.1, with base α = 1.5 and base β = −0.1. Asymmetric instantaneous boosting. Positive instantaneous regrets are multiplied by a factor of 1.1 before being added to cumulative regrets. This asymmetry is applied to the instantaneous update, not the accumulated history, making the algorithm more reactive to currently good actions. Hard warm-start with regret-magnitude weighting. Policy averaging is postponed entirely until iteration 500. The regret accumulation process continues normally during this phase. Once accumulation begins, policies are weighted by a combination of temporal weight and instantaneous regret magnitude — prioritizing high-information iterations when constructing the average strategy. The 500-iteration threshold was generated by the LLM without knowledge of the 1000-iteration evaluation horizon. VAD-CFR is benchmarked against standard CFR, CFR+, Linear CFR (LCFR), DCFR, PCFR+, DPCFR+, and HS-PCFR+(30) across 1000 iterations with K = 1000. Exploitability is computed exactly. On the full 11-game evaluation, VAD-CFR matches or surpasses state-of-the-art performance in 10 of the 11 games, with 4-player Kuhn Poker as the sole exception. ALSO DISCOVERED: AOD-CFR An earlier trial on a different training set (2-player Kuhn Poker, 2-player Leduc Poker, 4-card Goofspiel, 4-sided Liars Dice) produced a second variant, Asymmetric Optimistic Discounted CFR (AOD-CFR). It uses a linear schedule for discounting cumulative regrets (α transitions from 1.0 → 2.5 over 500 iterations, β from 0.5 → 0.0), sign-dependent scaling of instantaneous regret, trend-based policy optimism via an Exponential Moving Average of cumulative regrets, and polynomial policy averaging with an exponent γ scaling from 1.0 → 5.0. The research team reports it achieves competitive performance using more conventional mechanisms than VAD-CFR. Discovered Algorithm 2: SHOR-PSRO The evolved PSRO variant is Smoothed Hybrid Optimistic Regret PSRO (SHOR-PSRO). The search produced a hybrid meta-solver that constructs a meta-strategy by linearly blending two components at every internal solver iteration: σ_ORM (Optimistic Regret Matching): Provides regret-minimization stability. Gains are computed, optionally normalized and diversity-adjusted, then used to update cumulative regrets via regret matching. A momentum term is applied to payoff gains. σ_Softmax (Smoothed Best Pure Strategy): A Boltzmann distribution over pure strategies biased toward high-payoff modes. A temperature parameter controls concentration — lower temperature means the distribution is more concentrated on the best pure strategy. σ_hybrid  =  (1 − λ) · σ_ORM  +  λ · σ_Softmax The training-time solver uses a dynamic annealing schedule over the outer PSRO iterations. The blending factor λ anneals from 0.3 → 0.05 (shifting from

MIT Technology Review Explains: Let our writers untangle the complex, messy world of technology to help you understand what’s coming next. You can read more from the series here. In January, Elon Musk’s SpaceX filed an application with the US Federal Communications Commission to launch up to one million data centers into Earth’s orbit. The goal? To fully unleash the potential of AI without triggering an environmental crisis on Earth. But could it work? SpaceX is the latest in a string of high-tech companies extolling the potential of orbital computing infrastructure. Last year, Amazon founder Jeff Bezos said that the tech industry will move toward large-scale computing in space. Google has plans to loft data-crunching satellites, aiming to launch a test constellation of 80 as early as next year. And last November Starcloud, a startup based in Washington State, launched a satellite fitted with a high-performance Nvidia H100 GPU, marking the first orbital test of an advanced AI chip. The company envisions orbiting data centers as large as those on Earth by 2030. Proponents believe that putting data centers in space makes sense. The current AI boom is straining energy grids and adding to the demand for water, which is needed to cool the computers. Communities in the vicinity of large-scale data centers worry about increasing prices for those resources as a result of the growing demand, among other issues. In space, advocates say, the water and energy problems would be solved. In constantly illuminated sun-synchronous orbits, space-borne data centers would have uninterrupted access to solar power. At the same time, the excess heat they produce would be easily expelled into the cold vacuum of space. And with the cost of space launches decreasing, and mega-rockets such as SpaceX’s Starship promising to push prices even lower, there could be a point at which moving the world’s data centers into space makes sound business sense. Detractors, on the other hand, tell a different story and point to a variety of technological hurdles, though some say it’s possible they may be surmountable in the not-so-distant future. Here are four of the must-haves we’d need to make space-based data centers a reality.  A way to carry away heat  AI data centers produce a lot of heat. Space might seem like a great place to dispel that heat without using up massive amounts of water. But it’s not so simple. To get the power needed to run 24-7, a space-based data center would have to be in a constantly illuminated orbit, circling the planet from pole to pole, and never hide in Earth’s shadow. And in that orbit, the temperature of the equipment would never drop below 80 °C, which is way too hot for electronics to operate safely in the long term.  Getting the heat out of such a system is surprisingly challenging. “Thermal management and cooling in space is generally a huge problem,” says Lilly Eichinger, CEO of the Austrian space tech startup Satellives. On Earth, heat dissipates mostly through the natural process of convection, which relies on the movement of gases and liquids like air and water. In the vacuum of space, heat has to be removed through the far less efficient process of radiation. Safely removing the heat produced by the computers, as well as what’s absorbed from the sun, requires large radiative surfaces. The bulkier the satellite, the harder it is to send all the heat inside it out into space. But Yves Durand, former director of technology at the European aerospace giant Thales Alenia Space, says that technology already exists to tackle the problem. The company previously developed a system for large telecommunications satellites that can pipe refrigerant fluid through a network of tubing using a mechanical pump, ultimately transferring heat from within a spacecraft to radiators on the exterior. Durand led a 2024 feasibility study on space-based data centers, which found that although challenges exist, it should be possible for Europe to put gigawatt-scale data centers (on par with the largest Earthbound facilities) into orbit before 2050. These would be considerably larger than those envisioned by SpaceX, featuring solar arrays hundreds of meters in size—larger than the International Space Station. Computer chips that can withstand a radiation onslaught The space around Earth is constantly battered by cosmic particles and lashed by solar radiation. On Earth’s surface, humans and their electronic devices are protected from this corrosive soup of charged particles by the planet’s atmosphere and magnetosphere. But the farther away from Earth you venture, the weaker that protection becomes. Studies show that aircraft crews have a higher risk of developing cancer because of their frequent exposure to high radiation at cruising altitude, where the atmosphere is thin and less protective. Electronics in space are at risk of three types of problems caused by high radiation levels, says Ken Mai, a principal systems scientist in electrical and computer engineering at Carnegie Mellon University. Phenomena known as single-event upsets can cause bit flips and corrupt stored data when charged particles hit chips and memory devices. Over time, electronics in space accumulate damage from ionizing radiation that degrades their performance. And sometimes a charged particle can strike the component in a way that physically displaces atoms on the chip, creating permanent damage, Mai explains. Traditionally, computers launched to space had to undergo years of testing and were specifically designed to withstand the intense radiation present in Earth’s orbit. These space-hardened electronics are much more expensive, though, and their performance is also years behind the state-of-the-art devices for Earth-based computing. Launching conventional chips is a gamble. But Durand says cutting-edge computer chips use technologies that are by default more resistant to radiation than past systems. And in mid-March, Nvidia touted hardware, including a new GPU, that is “bringing AI compute to orbital data centers.”  Nvidia’s head of edge AI marketing, Chen Su, told MIT Technology Review, that “Nvidia systems are inherently commercial off the shelf, with radiation resilience achieved at the system level rather than through radiation‑hardened silicon alone.” He

In this tutorial, we explore the full capabilities of Z.AI’s GLM-5 model and build a complete understanding of how to use it for real-world, agentic applications. We start from the fundamentals by setting up the environment using the Z.AI SDK and its OpenAI-compatible interface, and then progressively move on to advanced features such as streaming responses, thinking mode for deeper reasoning, and multi-turn conversations. As we continue, we integrate function calling, structured outputs, and eventually construct a fully functional multi-tool agent powered by GLM-5. Also, we understand each capability in isolation, and also how Z.AI’s ecosystem enables us to build scalable, production-ready AI systems. Copy CodeCopiedUse a different Browser !pip install -q zai-sdk openai rich import os import json import time from datetime import datetime from typing import Optional import getpass API_KEY = os.environ.get(“ZAI_API_KEY”) if not API_KEY: API_KEY = getpass.getpass(” Enter your Z.AI API key (hidden input): “).strip() if not API_KEY: raise ValueError( ” No API key provided! Get one free at: https://z.ai/manage-apikey/apikey-list” ) os.environ[“ZAI_API_KEY”] = API_KEY print(f” API key configured (ends with …{API_KEY[-4:]})”) from zai import ZaiClient client = ZaiClient(api_key=API_KEY) print(” ZaiClient initialized — ready to use GLM-5!”) print(“n” + “=” * 70) print(” SECTION 2: Basic Chat Completion”) print(“=” * 70) response = client.chat.completions.create( model=”glm-5″, messages=[ {“role”: “system”, “content”: “You are a concise, expert software architect.”}, {“role”: “user”, “content”: “Explain the Mixture-of-Experts architecture in 3 sentences.”}, ], max_tokens=256, temperature=0.7, ) print(“n GLM-5 Response:”) print(response.choices[0].message.content) print(f”n Usage: {response.usage.prompt_tokens} prompt + {response.usage.completion_tokens} completion tokens”) print(“n” + “=” * 70) print(” SECTION 3: Streaming Responses”) print(“=” * 70) print(“n GLM-5 (streaming): “, end=””, flush=True) stream = client.chat.completions.create( model=”glm-5″, messages=[ {“role”: “user”, “content”: “Write a Python one-liner that checks if a number is prime.”}, ], stream=True, max_tokens=512, temperature=0.6, ) full_response = “” for chunk in stream: delta = chunk.choices[0].delta if delta.content: print(delta.content, end=””, flush=True) full_response += delta.content print(f”nn Streamed {len(full_response)} characters”) We begin by installing the Z.AI and OpenAI SDKs, then securely capture our API key through hidden terminal input using getpass. We initialize the ZaiClient and fire off our first basic chat completion to GLM-5, asking it to explain the Mixture-of-Experts architecture. We then explore streaming responses, watching tokens arrive in real time as GLM-5 generates a Python one-liner for prime checking. Copy CodeCopiedUse a different Browser print(“n” + “=” * 70) print(” SECTION 4: Thinking Mode (Chain-of-Thought)”) print(“=” * 70) print(“GLM-5 can expose its internal reasoning before giving a final answer.”) print(“This is especially powerful for math, logic, and complex coding tasks.n”) print(“─── Thinking Mode + Streaming ───n”) stream = client.chat.completions.create( model=”glm-5″, messages=[ { “role”: “user”, “content”: ( “A farmer has 17 sheep. All but 9 run away. ” “How many sheep does the farmer have left? ” “Think carefully before answering.” ), }, ], thinking={“type”: “enabled”}, stream=True, max_tokens=2048, temperature=0.6, ) reasoning_text = “” answer_text = “” for chunk in stream: delta = chunk.choices[0].delta if hasattr(delta, “reasoning_content”) and delta.reasoning_content: if not reasoning_text: print(” Reasoning:”) print(delta.reasoning_content, end=””, flush=True) reasoning_text += delta.reasoning_content if delta.content: if not answer_text and reasoning_text: print(“nn Final Answer:”) print(delta.content, end=””, flush=True) answer_text += delta.content print(f”nn Reasoning: {len(reasoning_text)} chars | Answer: {len(answer_text)} chars”) print(“n” + “=” * 70) print(” SECTION 5: Multi-Turn Conversation”) print(“=” * 70) messages = [ {“role”: “system”, “content”: “You are a senior Python developer. Be concise.”}, {“role”: “user”, “content”: “What’s the difference between a list and a tuple in Python?”}, ] r1 = client.chat.completions.create(model=”glm-5″, messages=messages, max_tokens=512, temperature=0.7) assistant_reply_1 = r1.choices[0].message.content messages.append({“role”: “assistant”, “content”: assistant_reply_1}) print(f”n User: {messages[1][‘content’]}”) print(f” GLM-5: {assistant_reply_1[:200]}…”) messages.append({“role”: “user”, “content”: “When should I use a NamedTuple instead?”}) r2 = client.chat.completions.create(model=”glm-5″, messages=messages, max_tokens=512, temperature=0.7) assistant_reply_2 = r2.choices[0].message.content print(f”n User: {messages[-1][‘content’]}”) print(f” GLM-5: {assistant_reply_2[:200]}…”) messages.append({“role”: “assistant”, “content”: assistant_reply_2}) messages.append({“role”: “user”, “content”: “Show me a practical example with type hints.”}) r3 = client.chat.completions.create(model=”glm-5″, messages=messages, max_tokens=1024, temperature=0.7) assistant_reply_3 = r3.choices[0].message.content print(f”n User: {messages[-1][‘content’]}”) print(f” GLM-5: {assistant_reply_3[:300]}…”) print(f”n Conversation: {len(messages)+1} messages, {r3.usage.total_tokens} total tokens in last call”) We activate GLM-5’s thinking mode to observe its internal chain-of-thought reasoning streamed live through the reasoning_content field before the final answer appears. We then build a multi-turn conversation where we ask about Python lists vs tuples, follow up on NamedTuples, and request a practical example with type hints, all while GLM-5 maintains full context across turns. We track how the conversation grows in message count and token usage with each successive exchange. Copy CodeCopiedUse a different Browser print(“n” + “=” * 70) print(” SECTION 6: Function Calling (Tool Use)”) print(“=” * 70) print(“GLM-5 can decide WHEN and HOW to call external functions you define.n”) tools = [ { “type”: “function”, “function”: { “parameters”: { “type”: “object”, “properties”: { “city”: { “type”: “string”, “description”: “City name, e.g. ‘San Francisco’, ‘Tokyo'”, }, “unit”: { “type”: “string”, “enum”: [“celsius”, “fahrenheit”], “description”: “Temperature unit (default: celsius)”, }, }, “required”: [“city”], }, }, }, { “type”: “function”, “function”: { “name”: “calculate”, “description”: “Evaluate a mathematical expression safely”, “parameters”: { “type”: “object”, “properties”: { “expression”: { “type”: “string”, “description”: “Math expression, e.g. ‘2**10 + 3*7′”, } }, “required”: [“expression”], }, }, }, ] def get_weather(city: str, unit: str = “celsius”) -> dict: weather_db = { “san francisco”: {“temp”: 18, “condition”: “Foggy”, “humidity”: 78}, “tokyo”: {“temp”: 28, “condition”: “Sunny”, “humidity”: 55}, “london”: {“temp”: 14, “condition”: “Rainy”, “humidity”: 85}, “new york”: {“temp”: 22, “condition”: “Partly Cloudy”, “humidity”: 60}, } data = weather_db.get(city.lower(), {“temp”: 20, “condition”: “Clear”, “humidity”: 50}) if unit == “fahrenheit”: data[“temp”] = round(data[“temp”] * 9 / 5 + 32) return {“city”: city, “unit”: unit or “celsius”, **data} def calculate(expression: str) -> dict: allowed = set(“0123456789+-*/.()% “) if not all(c in allowed for c in expression): return {“error”: “Invalid characters in expression”} try: result = eval(expression) return {“expression”: expression, “result”: result} except Exception as e: return {“error”: str(e)} TOOL_REGISTRY = {“get_weather”: get_weather, “calculate”: calculate} def run_tool_call(user_message: str): print(f”n User: {user_message}”) messages = [{“role”: “user”, “content”: user_message}] response = client.chat.completions.create( model=”glm-5″, messages=messages, tools=tools, tool_choice=”auto”, max_tokens=1024, ) assistant_msg = response.choices[0].message messages.append(assistant_msg.model_dump()) if assistant_msg.tool_calls: for tc in assistant_msg.tool_calls: fn_name = tc.function.name fn_args = json.loads(tc.function.arguments) print(f” Tool call: {fn_name}({fn_args})”) result = TOOL_REGISTRY[fn_name](**fn_args) print(f” Result: {result}”) messages.append({ “role”: “tool”, “content”: json.dumps(result,

arXiv:2604.01195v2 Announce Type: replace Abstract: Search agents, which integrate language models (LMs) with web search, are becoming crucial for answering complex user queries. Constructing training datasets for deep research tasks, involving multi-step retrieval and reasoning, remains challenging due to expensive human annotation, or cumbersome prerequisites. In this work, we introduce ORBIT, a training dataset with 20K reasoning-intensive queries with short verifiable answers, generated using a frugal framework without relying on paid API services. The modular framework relies on four stages: seed creation, question-answer pair generation, and two stages of verification: self and external. ORBIT spans 15 domains and each training pair requires 4-5 reasoning steps, with external search verification required from the complete web. We train Qwen3-4B as the base model on ORBIT using GRPO and evaluate it on Wikipedia question answering tasks. Extensive experiment results demonstrate that ORBIT-4B achieves strong performance among sub-4B LLMs as search agents, proving the utility of synthetic datasets. Our framework, code and datasets are open-sourced and available publicly.

arXiv:2604.01639v1 Announce Type: new Abstract: Large language models demonstrate strong performance on mathematical reasoning benchmarks, yet remain surprisingly fragile to meaning-preserving surface perturbations. We systematically evaluate three open-weight LLMs, Mistral-7B, Llama-3-8B, and Qwen2.5-7B, on 677 GSM8K problems paired with semantically equivalent variants generated through name substitution and number format paraphrasing. All three models exhibit substantial answer-flip rates (28.8%-45.1%), with number paraphrasing consistently more disruptive than name swaps. To trace the mechanistic basis of these failures, we introduce the Mechanistic Perturbation Diagnostics (MPD) framework, combining logit lens analysis, activation patching, component ablation, and the Cascading Amplification Index (CAI) into a unified diagnostic pipeline. CAI, a novel metric quantifying layer-wise divergence amplification, outperforms first divergence layer as a failure predictor for two of three architectures (AUC up to 0.679). Logit lens reveals that flipped samples diverge from correct predictions at significantly earlier layers than stable samples. Activation patching reveals a stark architectural divide in failure localizability: Llama-3 failures are recoverable by patching at specific layers (43/60 samples), while Mistral and Qwen failures are broadly distributed (3/60 and 0/60). Based on these diagnostic signals, we propose a mechanistic failure taxonomy (localized, distributed, and entangled) and validate it through targeted repair experiments: steering vectors and layer fine-tuning recover 12.2% of localized failures (Llama-3) but only 7.2% of entangled (Qwen) and 5.2% of distributed (Mistral) failures.

arXiv:2604.01762v1 Announce Type: cross Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a crucial paradigm for adapting large language models (LLMs) under constrained computational budgets. However, standard PEFT methods often struggle in multi-task fine-tuning settings, where diverse optimization objectives induce task interference and limited parameter budgets lead to representational deficiency. While recent approaches incorporate mixture-of-experts (MoE) to alleviate these issues, they predominantly operate in the spatial domain, which may introduce structural redundancy and parameter overhead. To overcome these limitations, we reformulate adaptation in the spectral domain. Our spectral analysis reveals that different tasks exhibit distinct frequency energy distributions, and that LLM layers display heterogeneous frequency sensitivities. Motivated by these insights, we propose FourierMoE, which integrates the MoE architecture with the inverse discrete Fourier transform (IDFT) for frequency-aware adaptation. Specifically, FourierMoE employs a frequency-adaptive router to dispatch tokens to experts specialized in distinct frequency bands. Each expert learns a set of conjugate-symmetric complex coefficients, preserving complete phase and amplitude information while theoretically guaranteeing lossless IDFT reconstruction into real-valued spatial weights. Extensive evaluations across 28 benchmarks, multiple model architectures, and scales demonstrate that FourierMoE consistently outperforms competitive baselines in both single-task and multi-task settings while using significantly fewer trainable parameters. These results highlight the promise of spectral-domain expert adaptation as an effective and parameter-efficient paradigm for LLM fine-tuning.

arXiv:2604.01630v1 Announce Type: new Abstract: Despite emerging use in Indonesian classrooms, there is limited large-scale, teacher-centred evidence on how AI is used in practice and what support teachers need, hindering the development of context-appropriate AI systems and policies. To address this gap, we conduct a nationwide survey of 349 K-12 teachers across elementary, junior high, and senior high schools. We find increasing use of AI for pedagogy, content development, and teaching media, although adoption remains uneven. Elementary teachers report more consistent use, while senior high teachers engage less; mid-career teachers assign higher importance to AI, and teachers in Eastern Indonesia perceive greater value. Across levels, teachers primarily use AI to reduce instructional preparation workload (e.g., assessment, lesson planning, and material development). However, generic outputs, infrastructure constraints, and limited contextual alignment continue to hinder effective classroom integration.