Committee

Something stinks in California’s climate policies. Years ago, the state set up a system that pays cattle farmers across the country to turn the methane emitted from cattle manure into natural gas, encouraging the dairy sector to produce a gas we burn instead of one that just pollutes the air. It’s become wildly popular because the subsidies are extremely lucrative. But a growing body of research suggests the program is a case study in the shortcomings of our preferred approaches to climate action. Instead of simply forcing industries to directly cut their pollution or pay for it as a cost of doing business, legislators have repeatedly opted to set up convoluted incentive systems that swap climate responsibilities between parties and regions. As studies have shown again and again, these carbon offsetting and trading schemes often dramatically overstate the emissions reductions actually achieved in the one place that matters: the atmosphere. The dairy program illustrates a particular version of this problem, muddling the impacts of different types of greenhouse gases in a way that researchers argue will lock in more warming in the future. Despite this and other concerns, California regulators decided in 2024 to extend parts of the program beyond 2050. And a recent proposal by the state’s air resources board could send millions of additional dollars to dairy farmers as part of a plan that would ease restrictions on major greenhouse-gas producers. Here’s how the system works: The state’s climate regulations require the transportation fuels industry to lower the carbon dioxide levels in its products over time—or purchase credits from other parties that cut fuel emissions, including cattle farmers. Dairies generally spray cattle manure into giant open lagoons, where microbes gobble up organic matter and produce methane as a by-product. But if farmers set up what are known as anaerobic digesters, the sludge is redirected into covered vessels that capture the biogas, which can be converted into natural gas and injected into a pipeline. It can then be used to fuel certain vehicles or generate electricity in a power plant. Either way, petroleum companies can pay those farmers for Low Carbon Fuel Standard (LCFS) credits, to meet regulatory requirements in lieu of reducing the emissions from their own fuels. Burning biogas in a bus or turbine still releases carbon dioxide, but the idea is that this process reduces market demand to extract natural gas from the ground and avoids the release of methane, which is a far more powerful greenhouse gas (at least initially). In fact, methane is so much more powerful that under California’s program, “adding one average biogas-powered vehicle to the fleet would produce enough LCFS credits to cover the deficits incurred by 26 similar gasoline-powered vehicles,” according to Aaron Smith, a UC Berkeley economist. But there’s a problem with this carbon math. California assumes that methane exerts about 25 times the warming effect of carbon dioxide over a 100-year period. That’s not how it really works in the atmosphere, though. Methane is very powerful, but it also breaks down quickly, generally within a couple of decades. Meanwhile, carbon dioxide builds up cumulatively in the atmosphere—and much of whatever we emit will continue heating up the planet for hundreds to thousands of years. So, in effect, the state has created a system that reduces short-term warming at the cost of increasing all-but-permanent warming. Any methane that digesters capture today would have caused extra-powerful warning if released, but by 2050 that effect would have mostly faded away. Meanwhile, that additional carbon dioxide we permitted in its place could continue warming the world for millennia. It is a good idea to cut methane emissions, and dairy digesters achieve this (though not always as effectively as hoped). But we can’t swap a decrease in short-lived greenhouse gases for an increase in long-lived ones if we hope to keep global temperatures within relatively safe levels in the coming century, as researchers have long warned. We have to slash both. The problem I keep returning to, after years of covering carbon markets and offsets, is this: We need to clean up every sector, completely, over the next few decades. It’s increasingly untenable for so many of our climate ambitions to turn on getting one industry to make progress on paper by paying another one to reduce emissions, at a point when every business in every industry needs to be racing toward net zero. It’s time to move past the idea that we need to reward sectors for doing us the favor of not polluting the atmosphere, and simply require them to stop unloading the huge environmental burden of their business onto society. This article is from The Spark, MIT Technology Review’s weekly climate newsletter. To receive it in your inbox every Wednesday, sign up here.

Artificial intelligence may have captured the public imagination through chatbots and image generators, but some of its most consequential use cases are unfolding far from consumer-facing tools. In industries where physical infrastructure, operational continuity, and safety are paramount, AI is becoming a core operating layer. With its sprawling industrial systems and constant stream of operational data, the energy sector offers a glimpse into what that future could look like. At Woodside Energy, AI adoption did not begin with generative models or enterprise copilots. The company has spent years building predictive analytics, optimization systems, and machine learning tools across exploration, drilling, maintenance, and plant operations. “We’ve always had very large volumes of operational data coming from the equipment and the plants and the assets that we operate,” says the company’s vice president for digital Andrew Melouney. “Those have created really clear, quite high-value use cases for us.” That long-term investment in infrastructure and governance is now enabling a broader shift toward agentic AI systems that can support complex industrial workflows. Rather than replace human operators, Woodside designs AI systems to augment expertise in high-stakes environments. A prime example is its “Startup Advisor,” an AI copilot that helps operators manage the complex process of starting liquefied natural gas (LNG) plants. “We’re really thinking about, how does it support the people in the organization in terms of empowering them to make better decisions, to make faster decisions,” Melouney explains. The company’s approach reflects a wider evolution taking place across industrial AI: graduating from isolated experiments to enterprise-wide systems built on standardized platforms, governed data, and repeatable deployment patterns. That transition, Melouney argues, requires organizations to rethink both their technology stacks and how work itself gets done. “We’re not just bolting AI onto an existing process,” he says. “We’re deeply thinking about how that work needs to be reimagined.” Melouney’s motto has become: “Think big, prototype small, and scale fast.” As AI systems become more autonomous and interconnected, the companies poised to succeed may be those that spent years building the operational foundations beneath the hype. “Our ambition is really for an autonomous enterprise, where we have agents with agency that are able to really deeply interact with our core workflows,” says Melouney. This episode of Business Lab is produced in partnership with Infosys. Full Transcript: Megan Tatum: From MIT Technology Review, I’m Megan Tatum, and this is Business Lab, the show that helps business leaders make sense of new technologies coming out of the lab and into the marketplace. This episode is produced in partnership with Infosys. Now, when people think about artificial intelligence, they often picture chatbots or productivity tools, but some of the most sophisticated and high impact uses of AI are actually happening far from consumer apps, inside complex industrial environments where safety, reliability, and physical systems matter. The global energy sector is a prime example. Companies like Woodside Energy, a global energy producer headquartered in Western Australia, have been applying AI for more than a decade now, from advanced analytics and operations, to remote decision support, to smarter maintenance, and energy efficiency across large scale assets. Today, Woodside is scaling that experience, embedding AI more deeply across its operations and the enterprise with a strong focus on governance, data quality, and human accountability. Two words for you: technological fuel. My guest today is Andrew Melouney, vice president for digital at Woodside Energy. Welcome, Andrew. Andrew Melouney: Thanks, Megan. It’s great to be here. Megan: Lovely to have you. Now, Andrew, as I said there, the energy sector has approached AI quite differently from technology or consumer businesses. Early value has emerged in operational and industrial environments, rather than consumer-facing generative AI tools. Why is that? And what differentiates the energy sector’s AI journey? Andrew: Megan, I think it really comes down to the nature of the work we do. Energy operations and what Woodside does is very asset intensive, it’s very safety critical, and it’s highly physical. And when you think about how Woodside operates, we operate across the full value chain. We do exploration through to drilling and subsurface work, to project development, all the way through to operating assets, which are often operated in harsh and remote locations, and then global energy portfolio marketing and trading as well. We’ve always had very large volumes of operational data coming from the equipment and the plants and the assets that we operate, and those have created really clear, quite high-value use cases for us. When you think about reliability, when you think about safety and efficiency, those are really critical things for a company like Woodside. We’ve been doing traditional AI for many years now. If you think about analytics, if you think about optimization, if you think about things like predictive models, those techniques we’ve been applying to our data sets and to our business since around 2015. And more recently with the advent of generative AI, we’ve really found that we’ve got a pretty strong and awesome foundation to build on top of and to really solve problems in the service of improving the business. And again, whether that is keeping people safe, keeping the environments we operate in safe, or improving returns for the organization. Megan: Fantastic. I mean you touched on it there, but how has this reality shaped your own AI strategy at Woodside? Where did you start, and where did the technology prove most impactful in those early days? Andrew: Well, like I said, we’ve had a very long journey, in terms of understanding our operational data, recognizing the value of it, and collecting it at scale so that we can use it. And we’ve been very deliberate in that approach, Megan. We’ve really thought about where the value is and where the risks were manageable. And we’ve started looking at, in today’s world from an agentic AI perspective, we’ve started looking at the problems that were solved with traditional AI and machine learning and data science in the past. And we’ve

In this tutorial, we explore CUP, Baidu’s Common Useful Python library, as a practical utility toolkit for building stronger Python workflows. We begin by setting up the library in a Colab-friendly environment and then move through its major subsystems step by step, including logging, decorators, nested configuration, caching, ID generation, thread pools, interruptible threads, delayed execution, time utilities, Linux resource monitoring, file locking, networking helpers, object storage interfaces, type maps, and built-in testing assertions. As we progress, we do not just call functions at random; we observe how each module fits into real-world development tasks such as monitoring, automation, concurrency, configuration management, and reliability checks. CUP Setup and Logging Copy CodeCopiedUse a different Browser import os import sys import time import threading import tempfile import datetime import subprocess def banner(title): line = “=” * 70 print(“n” + line + “n” + title + “n” + line) def skip(exc): “””Report a gracefully-skipped section without aborting the notebook.””” print(” [skipped — {}: {}]”.format(type(exc).__name__, exc)) banner(“0. SETUP (install + cup.platforms + cup.version)”) subprocess.run( [sys.executable, “-m”, “pip”, “install”, “-q”, “cup”, “pytz”], check=False, ) import cup ver = getattr(cup, “__version__”, None) if ver is None: try: from cup import version as _v ver = getattr(_v, “VERSION”, None) or getattr(_v, “__version__”, “unknown”) except Exception: ver = “unknown” print(“CUP version :”, ver) print(“Python :”, sys.version.split()[0]) try: from cup import platforms print(“is_linux :”, platforms.is_linux()) print(“is_mac :”, platforms.is_mac()) print(“is_windows :”, platforms.is_windows()) print(“is_py3 :”, platforms.is_py3()) except Exception as e: skip(e) banner(“1. LOGGING (cup.log)”) LOGFILE = os.path.join(tempfile.gettempdir(), “cup_tutorial.log”) try: from cup import log log.init_comlog( “cup_tutorial”, log.INFO, LOGFILE, log.ROTATION, 10 * 1024 * 1024, True, False, ) log.info(“hello from cup.log — written to file AND stdout”) log.warning(“a warning line”) log.info_if(2 > 1, “info_if(True) -> emitted”) log.info_if(1 > 2, “info_if(False) -> you will NOT see this”) log.setloglevel(log.DEBUG) log.debug(“debug visible after setloglevel(DEBUG)”) try: with open(LOGFILE) as fh: last = [ln for ln in fh.read().splitlines() if ln.strip()][-1] parsed = log.parse(last) print(“parsed last log line ->”) for k in (“loglevel”, “date”, “time”, “pid”, “srcline”, “msg”): if isinstance(parsed, dict) and k in parsed: print(” {:8}: {}”.format(k, parsed[k])) except Exception as e: skip(e) except Exception as e: skip(e) We begin by setting up the CUP tutorial environment and installing the required packages directly from Python. We define helper functions that keep the notebook readable and allow failed sections to be skipped safely. We then explore CUP version details, platform checks, and structured logging to understand the library’s foundation. Decorators and Nested Config Copy CodeCopiedUse a different Browser banner(“2. DECORATORS (cup.decorators)”) try: from cup import decorators @decorators.Singleton class AppConfig(object): def __init__(self): self.created_at = time.time() a, b = AppConfig(), AppConfig() print(“Singleton: a is b ->”, a is b, “(same created_at:”, a.created_at == b.created_at, “)”) @decorators.TraceUsedTime( b_print_stdout=True, enter_msg=”event_id=0xABCDE enter”, leave_msg=”event_id=0xABCDE leave”, ) def heavy_compute(): time.sleep(0.2) return sum(range(200000)) print(“heavy_compute() =”, heavy_compute()) @decorators.needlinux def linux_only(): return “this body is allowed to run on Linux” print(“needlinux ->”, linux_only()) except Exception as e: skip(e) banner(“3. RICH NESTED CONFIG (cup.util.conf)”) CONF_PATH = os.path.join(tempfile.gettempdir(), “cup_demo.conf”) CONF_TEXT = “”” # —- global scalars (layer 0) —- host: abc.com port: 12345 debug: false [monitor] enabled: true interval: 60 regex: sshd [.thresholds] cpu_max: 90 mem_max: 80 [..actions] on_breach: alert [storage] @path: /data/disk1 @path: /data/disk2 @path: /data/disk3 “”” try: from cup.util import conf with open(CONF_PATH, “w”) as fh: fh.write(CONF_TEXT) cfg = conf.Configure2Dict(CONF_PATH, separator=”:”).get_dict() print(“host :”, cfg[“host”]) print(“port :”, cfg[“port”]) print(“monitor.enabled :”, cfg[“monitor”][“enabled”]) print(“monitor.regex :”, cfg[“monitor”][“regex”]) print(“monitor.thresholds.cpu_max :”, cfg[“monitor”][“thresholds”][“cpu_max”]) print(“monitor.thresholds.actions :”, cfg[“monitor”][“thresholds”][“actions”][“on_breach”]) print(“storage.path (repeated @ -> list):”, list(cfg[“storage”][“path”])) cfg[“port”] = “10085” cfg[“monitor”][“thresholds”][“actions”][“on_breach”] = “restart” NEW_PATH = CONF_PATH + “.new” conf.Dict2Configure(cfg, separator=”:”).write_conf(NEW_PATH) re_read = conf.Configure2Dict(NEW_PATH, separator=”:”).get_dict() print(“round-trip port :”, re_read[“port”], “(was 12345)”) print(“round-trip on_breach :”, re_read[“monitor”][“thresholds”][“actions”][“on_breach”], “(was alert)”) except Exception as e: skip(e) We move on to CUP decorators and see how they help us create single-instance classes, track execution time, and protect Linux-only functions. We then work with CUP’s rich configuration system and load a nested configuration file with sections, child sections, and repeated values. We also update the configuration and write it back to disk to confirm that the read-modify-write flow works correctly. Caching, IDs, Thread Pools Copy CodeCopiedUse a different Browser banner(“4. IN-MEMORY KV CACHE (cup.cache)”) try: from cup import cache kv = cache.KVCache(name=”demo”) kv.set({“user:1”: “alice”, “user:2”: “bob”}, expire_sec=2) kv.set({“config:flag”: “on”}, expire_sec=None) print(“size after sets :”, kv.size()) print(“get user:1 :”, kv.get(“user:1”)) print(“get missing key :”, kv.get(“nope”)) print(“sleeping 2.2s to let the 2s-TTL keys expire …”) time.sleep(2.2) print(“get user:1 (expired) :”, kv.get(“user:1”)) print(“get config:flag (eternal):”, kv.get(“config:flag”)) reclaimed = kv.pop_n_expired(0) print(“pop_n_expired reclaimed :”, list(reclaimed.keys()) if reclaimed else []) except Exception as e: skip(e) banner(“5. UNIQUE ID GENERATION (cup.services.generator)”) try: from cup.services import generator gman = generator.CGeneratorMan() print(“uniqname :”, gman.get_uniqname()) print(“next_uniq_num :”, gman.get_next_uniq_num()) print(“next_uniq_num (again) :”, gman.get_next_uniq_num(), “(monotonic)”) if hasattr(gman, “get_uuid”): try: print(“get_uuid :”, gman.get_uuid()) except Exception as e: skip(e) if hasattr(gman, “get_random_str”): try: print(“get_random_str(16) :”, gman.get_random_str(16)) except Exception as e: skip(e) print(“singleton check :”, generator.CGeneratorMan() is gman) try: cyc = generator.CycleIDGenerator(“127.0.0.1”, 8080) i1, i2 = cyc.next_id(), cyc.next_id() print(“CycleIDGenerator id #1 :”, i1) print(“CycleIDGenerator id #2 :”, i2, “(incremented)”) print(“id #1 as hex :”, generator.CycleIDGenerator.id2_hexstring(i1)) except Exception as e: skip(e) except Exception as e: skip(e) banner(“6. THREAD POOL (cup.services.threadpool)”) try: from cup.services import threadpool pool = threadpool.ThreadPool(minthreads=2, maxthreads=4, name=”demo-pool”) pool.start() results, rlock = [], threading.Lock() def square(n): time.sleep(0.03) with rlock: results.append(n * n) return n * n for i in range(8): pool.add_1job(square, i) callback_log = [] def on_done(ok, result): callback_log.append((ok, result)) pool.add_1job_with_callback(on_done, square, 100) def will_fail(): raise RuntimeError(“boom inside worker”) pool.add_1job_with_callback(on_done, will_fail) time.sleep(0.5) print(“live stats :”, pool.get_stats()) pool.stop() print(“squares collected :”, sorted(results)) print(“callback results :”, callback_log) except Exception as e: skip(e) We use CUP’s in-memory cache to store key-value pairs with temporary and permanent lifetimes. We then generate unique names, counters, UUID-style values, random strings, and cycling IDs for distributed-style identifiers. We also create a thread pool, submit jobs, collect results, and observe callback behavior for both successful and failed tasks. Threads, Scheduling, Time Utilities Copy CodeCopiedUse a different Browser banner(“7. INTERRUPTIBLE THREADS + RW LOCK (cup.thread)”) try: from cup import thread as cupthread rw = cupthread.RWLock() rw.acquire_readlock() rw.acquire_readlock() print(“acquired 2 read locks concurrently”) rw.release_readlock() rw.release_readlock() rw.acquire_writelock() print(“acquired exclusive write lock”) rw.release_writelock()

NVIDIA has released Nemotron-Labs-TwoTower, a diffusion language model built on a pretrained autoregressive backbone. It ships as open weights under the NVIDIA Nemotron Open Model License. The release targets a throughput bottleneck in text generation. Autoregressive (AR) models decode one token at a time. That serial process caps generation throughput. Discrete diffusion language models take another route. They generate tokens in parallel and refine them iteratively. Most diffusion language models use one network for two jobs. It represents clean tokens and denoises corrupted ones at every step. TwoTower separates these jobs into two towers. It keeps 98.7% of the AR baseline’s aggregate benchmark quality. It also reports 2.42× higher wall-clock generation throughput. TL;DR TwoTower splits diffusion into a frozen AR context tower and a trained denoiser tower. It retains 98.7% of AR quality at 2.42× throughput (γ=0.8, S=16, 2×H100). The denoiser trained on ~2.1T tokens; the backbone used 25T. One checkpoint runs diffusion, mock-AR, and AR decoding modes. Nemotron-Labs-TwoTower TwoTower is a block-wise autoregressive diffusion model. It is instantiated on Nemotron-3-Nano-30B-A3B, an open-weight hybrid backbone. That backbone interleaves Mamba-2, self-attention, and mixture-of-experts (MoE) layers. Each tower has 52 layers: 23 Mamba-2, 6 self-attention, and 23 MoE. The released checkpoint ships both towers, roughly 60B total parameters. Active parameters per token are about 3B per tower. The MoE uses 128 routable experts, of which 6 activate, plus 2 shared experts. Both towers start as copies of the same backbone checkpoint. Only the denoiser tower is trained. The AR context tower stays frozen. The denoiser was trained on ~2.1T tokens, a fraction of the backbone’s 25T-token pretraining. How the Two Towers Work The AR context tower runs causally over the prompt and committed tokens. It produces per-layer KV cache and final Mamba-2 states. It preserves the backbone’s autoregressive capability. The diffusion denoiser tower refines noisy blocks. Within a block, it uses bidirectional in-block attention. It stays causal with respect to past clean blocks. The towers connect layer-by-layer. Denoiser layer i cross-attends to context tower layer i. This layer-aligned cross-attention gives multi-scale access to the backbone’s representations. Prior approaches broadcast only the last hidden state. Two more denoiser modifications matter. Mamba-2 layers seed their initial state from the context tower’s Mamba state. The diffusion timestep modulates each layer through adaLN-single time conditioning. That adaLN module adds only ~1.5M parameters. Generation runs block by block. Each block starts as S [MASK] tokens. The denoiser refines it over T steps, then commits it. The context tower then processes committed tokens to update its caches. This explains why multiple denoising steps can still beat one-token decoding. Autoregressive decoding commits exactly one token per step. TwoTower commits multiple tokens per step early in refinement. Benchmarks Evaluations use BF16 on 2×H100 GPUs. The default operating point is confidence unmasking, threshold γ=0.8, block size S=16. The table compares the AR baseline against TwoTower diffusion decoding. Task Nemotron-3-Nano-30B-A3B (AR) Nemotron-Labs-TwoTower (diffusion) MMLU (5-shot, acc) 78.56 78.24 MMLU-Pro (5-shot, CoT EM) 62.59 60.93 ARC-Challenge (25-shot, acc_norm) 91.72 92.66 WinoGrande (5-shot, acc) 76.09 76.09 RACE (0-shot, acc) 88.90 88.90 HumanEval (0-shot) 79.27 75.58 MBPP-Sanitized (3-shot) 74.71 74.28 GSM8K (8-shot, acc) 92.49 90.14 MATH-500 (4-shot) 84.40 80.60 MMLU Global Lite (5-shot) 73.97 73.94 MGSM (8-shot, avg acc) 80.80 80.40 Quality retained 100% 98.7% Generation throughput (× AR) 1.0× 2.42× General knowledge stays within about one point of the AR baseline. Code and math show modest degradation. Commonsense and multilingual scores are recovered or slightly improved. Lowering γ commits more tokens per step and raises throughput, with reduced quality. Running It: Three Generation Modes The checkpoint exposes three inference paths. Full two-tower diffusion uses 2 GPUs, about 59GB per GPU in BF16. AR-only mode runs on a single 80GB GPU. Copy CodeCopiedUse a different Browser import torch from transformers import AutoTokenizer, AutoModelForCausalLM model_name = “nvidia/Nemotron-Labs-TwoTower-30B-A3B-Base-BF16” tokenizer = AutoTokenizer.from_pretrained(model_name) model = AutoModelForCausalLM.from_pretrained( model_name, torch_dtype=torch.bfloat16, trust_remote_code=True, ) # context tower -> GPU 0, denoiser tower -> GPU 1 model.place_towers_on_devices(“cuda:0”, “cuda:1”) model.eval() prompt = “France is a country ” inputs = tokenizer(prompt, return_tensors=”pt”).to(“cuda:0”) outputs = model.generate_mask_diffusion( inputs[“input_ids”], max_new_tokens=128, block_size=16, steps_per_block=16, mask_token_id=3, temperature=0.1, confidence_threshold=0.8, eos_token_id=tokenizer.eos_token_id, ) print(tokenizer.decode(outputs[0][inputs[“input_ids”].shape[1]:], skip_special_tokens=True)) The three modes are generate_mask_diffusion(), generate_mock_ar(), and generate_ar(). Mask diffusion commits up to block_size tokens per step. Mock-AR and AR commit one token per step. Where It Fits: Use Cases The most direct use case is faster batch generation. A data team producing synthetic text can trade a small quality drop for throughput. At γ=0.8, that trade is 1.3% quality for 2.42× speed. A second use case is tuning the quality–throughput trade-off. Raising γ preserves more quality, as per the NVIDIA’s paper. Lowering γ commits more tokens per step for speed. A third use case is drop-in adaptation. The context tower keeps its LM head for speculative decoding, verification, or AR scoring. Teams can run AR and diffusion from one checkpoint. Strengths and Weaknesses Strengths: Open weights under the NVIDIA Nemotron Open Model License; ready for commercial use 98.7% of AR quality retained at 2.42× throughput at the default operating point One checkpoint supports diffusion, mock-AR, and AR decoding Denoiser trained on ~2.1T tokens, not a full re-pretrain Sequence-length cache memory scales like the AR baseline Weaknesses: Full two-tower diffusion needs 2 GPUs and ~59GB per GPU in BF16 Code and math degrade more than general knowledge (HumanEval 79.27 → 75.58) Keeping both towers resident raises the fixed model-weight memory footprint Released checkpoint is a base model, before instruction tuning or alignment Throughput past 3× comes with larger quality loss Interactive Explainer Check out the Paper and Weights. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post NVIDIA Releases Nemotron-Labs-TwoTower: an Open-Weight Diffusion Language Model Built on a Frozen Autoregressive Nemotron-3-Nano-30B-A3B Backbone appeared first on MarkTechPost.

Google Research introduced TabFM, a foundation model built for tabular data. TabFM performs classification and regression without dataset-specific training. Every prediction comes from a single forward pass. The model reframes tabular prediction as an in-context learning problem. It is available now on Hugging Face and GitHub. TL;DR TabFM predicts on unseen tables with no training, tuning, or feature engineering. It reads the full dataset as one prompt, then predicts via in-context learning. The architecture combines TabPFN-style row/column attention with TabICL-style in-context learning. Training used hundreds of millions of synthetic datasets from structural causal models. Google BigQuery will expose TabFM through an AI.PREDICT SQL command soon. What is TabFM? Tabular data forms the backbone of enterprise data infrastructure. Tasks like customer churn and financial fraud detection live in tables. For years, tree-based methods dominated this space. XGBoost, AdaBoost, and random forests offered robust results on structured data. Google frames TabFM as the tabular counterpart to TimesFM, its zero-shot time-series model. That reliability carried a cost. Fitting XGBoost to a new dataset is rarely one .fit() call. Data scientists spend hours on hyperparameter optimization and feature engineering. They do this just to extract a reliable signal from raw data. TabFM targets exactly that bottleneck. TabFM applies the zero-shot logic that large language models made familiar. LLMs learn new tasks from in-context examples, without updating any weights. This technique is called in-context learning (ICL). TabFM brings the same idea to tables. It generates predictions on previously unseen tables in one pass. How It Works Traditional models update parameters for each dataset’s distribution. TabFM skips that step entirely. It takes the whole dataset as a single unified prompt. That prompt holds both training examples and target testing rows. The model reads column and row relationships at inference time. Tables are not text. They are two-dimensional and inherently orderless. Swapping two rows or two columns does not change their meaning. Standard language models process one-dimensional, ordered sequences instead. To bridge that gap, TabFM synthesizes TabPFN and TabICL into a hybrid design. It relies on three mechanisms: Alternating row and column attention: The raw table passes through a multilayer attention module. Following TabPFN, attention alternates across columns (features) and rows (examples). This deep contextualization captures feature interactions and dependencies. It performs work that would otherwise need manual feature crafting. Row compression: Each row’s cross-attended information compresses into a single dense vector. In-context learning: A dedicated Transformer runs over these compressed embeddings. Following TabICL, attending to compressed rows cuts computation cost sharply. Prediction stays efficient even on much larger datasets. Training On Synthetic Data at Scale Foundation models need vast, diverse data. High-quality tabular datasets are scarce in the open-source space. Industrial tables carry proprietary schemas and sensitive information. That makes them inaccessible for broad pre-training. Synthetic tables can be generated to be arbitrarily large. Google’s research team calls them effectively the only viable option at this scale. So TabFM trains entirely on hundreds of millions of synthetic datasets. These are generated dynamically using structural causal models (SCMs). Each incorporates a wide variety of random functions. The approach captures distributions and complex feature relationships found in real tables. The research team reports the model generalizes well to unseen real-world data. Performance and Benchmarking The research team evaluated TabFM on TabArena. TabArena is a living benchmark that computes Elo scores from head-to-head win rates. The evaluation spans 38 classification datasets and 13 regression datasets. Sample sizes range from 700 to 150,000. Two configurations were tested. Plain TabFM runs out-of-the-box in a single forward pass. It needs no tuning or cross-validation. TabFM-Ensemble adds cross features and SVD (Singular Value Decomposition) features. It computes optimal weights for a 32-way ensemble using a non-negative least squares solver. For classification, it also adds Platt scaling as a calibration step. The research team reports TabFM consistently outperforms heavily tuned, industry-standard supervised algorithms. Full per-fold metrics and head-to-head win rates sit on the GitHub page. Aspect Traditional GBDT (XGBoost) TabFM TabFM-Ensemble Per-dataset training Required None (in-context learning) None Hyperparameter tuning Extensive, manual None Ensemble weights via NNLS Feature engineering Manual, domain-specific Learned by attention Adds cross + SVD features Prediction After full training Single forward pass 32-way ensemble Calibration Manual (optional) — Platt scaling (classification) Getting Started: Installation and Code Installation clones the repository and installs it locally. The base install uses CPU-only JAX. A cuda extra pulls the CUDA 12 plugin and NVIDIA libraries for GPU runs. Core requirements are specific. You need Python 3.11 or later. It pins jax==0.10.1 and flax==0.12.7, using the modern flax.nnx API. Hugging Face Hub downloads the pre-trained weights automatically. Copy CodeCopiedUse a different Browser import numpy as np import pandas as pd from tabfm import tabfm_v1_0_0 from tabfm import TabFMClassifier # Load pre-trained TabFM v1.0.0 (downloads from Hugging Face) model = tabfm_v1_0_0.load() # scikit-learn compatible classifier clf = TabFMClassifier(model=model) X_train = pd.DataFrame({ “age”: [25.0, 45.0, 35.0, 50.0], “job”: [“engineer”, “manager”, “engineer”, “manager”], “income”: [80000, 120000, 90000, 130000] }) y_train = np.array([“low_risk”, “high_risk”, “low_risk”, “high_risk”]) X_test = pd.DataFrame({ “age”: [30.0, 48.0], “job”: [“engineer”, “manager”], “income”: [85000, 125000] }) clf.fit(X_train, y_train) predictions = clf.predict(X_test) probabilities = clf.predict_proba(X_test) print(“Predictions:”, predictions) print(“Class Probabilities:n”, probabilities) Here fit() prepares ordinal encoders and numerical scalers. It does not train model weights on your data. The regressor mirrors this pattern with TabFMRegressor and reg.predict(). Use Cases With Examples The API fits common predictive tasks directly. For customer churn, the context holds past customers labeled churned or retained. TabFM scores churn risk for new customers in one pass. For credit risk, rows carry age, job, and income features. Labels mark low_risk or high_risk, as in the sample code. New applicants get scored without a training cycle. For regression, house price prediction is a natural fit. Context rows carry square footage and neighborhood. TabFM returns a predicted price for unseen listings. Interactive Explainer Check out the Repo and Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.