Committee

This article is divided into four parts; they are: • How Logits Become Probabilities • Temperature • Top- k Sampling • Top- p Sampling When you ask an LLM a question, it outputs a vector of logits.

Gen AI in software engineering has moved well beyond autocomplete. The emerging frontier is agentic coding: AI systems capable of planning changes, executing them across multiple steps and iterating based on feedback. Yet despite the excitement around “AI agents that code,” most enterprise deployments underperform. The limiting factor is no longer the model. It’s context: The structure, history and intent surrounding the code being changed. In other words, enterprises are now facing a systems design problem: They have not yet engineered the environment these agents operate in. The shift from assistance to agency The past year has seen a rapid evolution from assistive coding tools to agentic workflows. Research has begun to formalize what agentic behavior means in practice: The ability to reason across design, testing, execution and validation rather than generate isolated snippets. Work such as dynamic action re-sampling shows that allowing agents to branch, reconsider and revise their own decisions significantly improves outcomes in large, interdependent codebases. At the platform level, providers like GitHub are now building dedicated agent orchestration environments, such as Copilot Agent and Agent HQ, to support multi-agent collaboration inside real enterprise pipelines. But early field results tell a cautionary story. When organizations introduce agentic tools without addressing workflow and environment, productivity can decline. A randomized control study this year showed that developers who used AI assistance in unchanged workflows completed tasks more slowly, largely due to verification, rework and confusion around intent. The lesson is straightforward: Autonomy without orchestration rarely yields efficiency. Why context engineering is the real unlock In every unsuccessful deployment I’ve observed, the failure stemmed from context. When agents lack a structured understanding of a codebase, specifically its relevant modules, dependency graph, test harness, architectural conventions and change history. They often generate output that appears correct but is disconnected from reality. Too much information overwhelms the agent; too little forces it to guess. The goal is not to feed the model more tokens. The goal is to determine what should be visible to the agent, when and in what form. The teams seeing meaningful gains treat context as an engineering surface. They create tooling to snapshot, compact and version the agent’s working memory: What is persisted across turns, what is discarded, what is summarized and what is linked instead of inlined. They design deliberation steps rather than prompting sessions. They make the specification a first-class artifact, something reviewable, testable and owned, not a transient chat history. This shift aligns with a broader trend some researchers describe as “specs becoming the new source of truth.” Workflow must change alongside tooling But context alone isn’t enough. Enterprises must re-architect the workflows around these agents. As McKinsey’s 2025 report “One Year of Agentic AI” noted, productivity gains arise not from layering AI onto existing processes but from rethinking the process itself. When teams simply drop an agent into an unaltered workflow, they invite friction: Engineers spend more time verifying AI-written code than they would have spent writing it themselves. The agents can only amplify what’s already structured: Well-tested, modular codebases with clear ownership and documentation. Without those foundations, autonomy becomes chaos. Security and governance, too, demand a shift in mindset. AI-generated code introduces new forms of risk: Unvetted dependencies, subtle license violations and undocumented modules that escape peer review. Mature teams are beginning to integrate agentic activity directly into their CI/CD pipelines, treating agents as autonomous contributors whose work must pass the same static analysis, audit logging and approval gates as any human developer. GitHub’s own documentation highlights this trajectory, positioning Copilot Agents not as replacements for engineers but as orchestrated participants in secure, reviewable workflows. The goal isn’t to let an AI “write everything,” but to ensure that when it acts, it does so inside defined guardrails. What enterprise decision-makers should focus on now For technical leaders, the path forward starts with readiness rather than hype. Monoliths with sparse tests rarely yield net gains; agents thrive where tests are authoritative and can drive iterative refinement. This is exactly the loop Anthropic calls out for coding agents. Pilots in tightly scoped domains (test generation, legacy modernization, isolated refactors); treat each deployment as an experiment with explicit metrics (defect escape rate, PR cycle time, change failure rate, security findings burned down). As your usage grows, treat agents as data infrastructure: Every plan, context snapshot, action log and test run is data that composes into a searchable memory of engineering intent, and a durable competitive advantage. Under the hood, agentic coding is less a tooling problem than a data problem. Every context snapshot, test iteration and code revision becomes a form of structured data that must be stored, indexed and reused. As these agents proliferate, enterprises will find themselves managing an entirely new data layer: One that captures not just what was built, but how it was reasoned about. This shift turns engineering logs into a knowledge graph of intent, decision-making and validation. In time, the organizations that can search and replay this contextual memory will outpace those who still treat code as static text. The coming year will likely determine whether agentic coding becomes a cornerstone of enterprise development or another inflated promise. The difference will hinge on context engineering: How intelligently teams design the informational substrate their agents rely on. The winners will be those who see autonomy not as magic, but as an extension of disciplined systems design:Clear workflows, measurable feedback, and rigorous governance. Bottom line Platforms are converging on orchestration and guardrails, and research keeps improving context control at inference time. The winners over the next 12 to 24 months won’t be the teams with the flashiest model; they’ll be the ones that engineer context as an asset and treat workflow as the product. Do that, and autonomy compounds. Skip it, and the review queue does. Context + agent = leverage. Skip the first half, and the rest collapses. Dhyey Mavani is accelerating generative AI at LinkedIn. Read more from our guest writers. Or, consider submitting a post of your own!

OpenAI team has released their openai/circuit-sparsity model on Hugging Face and the openai/circuit_sparsity toolkit on GitHub. The release packages the models and circuits from the paper ‘Weight-sparse transformers have interpretable circuits‘. https://arxiv.org/pdf/2511.13653 What is a weight sparse transformer? The models are GPT-2 style decoder only transformers trained on Python code. Sparsity is not added after training, it is enforced during optimization. After each AdamW step, the training loop keeps only the largest magnitude entries in every weight matrix and bias, including token embeddings, and zeros the rest. All matrices maintain the same fraction of nonzero elements. The sparsest models have approximately 1 in 1000 nonzero weights. In addition, the OpenAI team enforced mild activation sparsity so that about 1 in 4 node activations are nonzero, covering residual reads, residual writes, attention channels and MLP neurons. Sparsity is annealed during training. Models start dense, then the allowed nonzero budget gradually moves toward the target value. This design lets the research team scale width while holding the number of nonzero parameters fixed, and then study the capability interpretability tradeoff as they vary sparsity and model size. The research team show that, for a given pretraining loss, circuits recovered from sparse models are roughly 16 times smaller than those from dense models. https://arxiv.org/pdf/2511.13653 So, what is a sparse circuit? The central object in this research work is a sparse circuit. The research team defines nodes at a very fine granularity, each node is a single neuron, attention channel, residual read channel or residual write channel. An edge is a single nonzero entry in a weight matrix that connects two nodes. Circuit size is measured by the geometric mean number of edges across tasks. To probe the models, the research team built 20 simple Python next token binary tasks. Each task forces the model to choose between 2 completions that differ in one token. Examples include: single_double_quote, predict whether to close a string with a single or double quote bracket_counting, decide between ] and ]] based on list nesting depth set_or_string, track whether a variable was initialized as a set or a string For each task, they prune the model to find the smallest circuit that still achieves a target loss of 0.15 on that task distribution. Pruning operates at the node level. Deleted nodes are mean ablated, their activations are frozen to the mean over the pretraining distribution. A learned binary mask per node is optimized with a straight through style surrogate so that the objective trades off task loss and circuit size. https://arxiv.org/pdf/2511.13653 Example circuits, quote closing and counting brackets The most compact example is the circuit for single_double_quote. Here the model must emit the correct closing quote type given an opening quote. The pruned circuit has 12 nodes and 9 edges. The mechanism is two step. In layer 0.mlp, 2 neurons specialize: a quote detector neuron that activates on both ” and ‘ a quote type classifier neuron that is positive on ” and negative on ‘ A later attention head in layer 10.attn uses the quote detector channel as a key and the quote type classifier channel as a value. The final token has a constant positive query, so the attention output copies the correct quote type into the last position and the model closes the string correctly. https://arxiv.org/pdf/2511.13653 bracket_counting yields a slightly larger circuit but with a clear algorithm. The embedding of [ writes into several residual channels that act as bracket detectors. A value channel in a layer 2 attention head averages this detector activation over the context, effectively computing nesting depth and storing it in a residual channel. A later attention head thresholds this depth and activates a nested list close channel only when the list is nested, which leads the model to output ]]. A third circuit, for set_or_string_fixedvarname, shows how the model tracks the type of a variable called current. One head copies the embedding of current into the set() or “” token. A later head uses that embedding as query and key to copy the relevant information back when the model must choose between .add and +=. https://arxiv.org/pdf/2511.13653 https://arxiv.org/pdf/2511.13653 Bridges, connecting sparse models to dense models The research team also introduces bridges that connect a sparse model to an already trained dense model. Each bridge is an encoder decoder pair that maps dense activations into sparse activations and back once per sublayer. The encoder uses a linear map with an AbsTopK activation, the decoder is linear. Training adds losses that encourage hybrid sparse dense forward passes to match the original dense model. This lets the research team perturb interpretable sparse features such as the quote type classifier channel and then map that perturbation into the dense model, changing its behavior in a controlled way. https://arxiv.org/pdf/2511.13653 What Exactly has OpenAI Team released? The OpenAI team as released openai/circuit-sparsity model on Hugging Face. This is a 0.4B parameter model tagged with custom_code, corresponding to csp_yolo2 in the research paper. The model is used for the qualitative results on bracket counting and variable binding. It is licensed under Apache 2.0. Copy CodeCopiedUse a different Browser import torch from transformers import AutoModelForCausalLM, AutoTokenizer if __name__ == “__main__”: PROMPT = “def square_sum(xs):n return sum(x * x for x in xs)nnsquare_sum([1, 2, 3])n” tok = AutoTokenizer.from_pretrained(“openai/circuit-sparsity”, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( “openai/circuit-sparsity”, trust_remote_code=True, torch_dtype=”auto”, ) model.to(“cuda” if torch.cuda.is_available() else “cpu”) inputs = tok(PROMPT, return_tensors=”pt”, add_special_tokens=False)[“input_ids”].to( model.device ) with torch.no_grad(): out = model.generate( inputs, max_new_tokens=64, do_sample=True, temperature=0.8, top_p=0.95, return_dict_in_generate=False, ) print(tok.decode(out[0], skip_special_tokens=True)) “` :contentReference[oaicite:14]{index=14} Key Takeaways Weight sparse training, not post hoc pruning: Circuit sparsity trains GPT-2 style decoder models with extreme weight sparsity enforced during optimization, most weights are zero so each neuron has only a few connections. Small, task specific circuits with explicit nodes and edges: The research team defines circuits at the level of individual neurons, attention channels and residual channels, and recovers circuits that often have tens of nodes and few edges for 20 binary Python next token tasks. Quote closing and type tracking are fully instantiated

__________________________Thai Space Expo October 16-18, 2025 ___Bangkok, Thailand It’s a scorching October day in Bangkok and I’m wandering through the exhibits at the Thai Space Expo, held in one of the city’s busiest shopping malls, when I do a double take. Amid the flashy space suits and model rockets on display, there’s a plain-looking package of Thai basil chicken. I’m told the same kind of vacuum-­sealed package has just been launched to the International Space Station. “This is real chicken that we sent to space,” says a spokesperson for the business behind the stunt, Charoen Pokphand Foods, the biggest food company in Thailand. It’s an unexpected sight, one that reflects the growing excitement within the Southeast Asian space sector. At the expo, held among designer shops and street-food stalls, enthusiastic attendees have converged from emerging space nations such as Vietnam, Malaysia, Singapore, and of course Thailand to showcase Southeast Asia’s fledgling space industry. While there is some uncertainty about how exactly the region’s space sector may evolve, there is plenty of optimism, too. “Southeast Asia is perfectly positioned to take leadership as a space hub,” says Candace Johnson, a partner in Seraphim Space, a UK investment firm that operates in Singapore. “There are a lot of opportunities.” A sample package of pad krapow was also on display.COURTESY OF THE AUTHOR For example, Thailand may build a spaceport to launch rockets in the next few years, the country’s Geo-Informatics and Space Technology Development Agency announced the day before the expo started. “We don’t have a spaceport in Southeast Asia,” says Atipat Wattanuntachai, acting head of the space economy advancement division at the agency. “We saw a gap.” Because Thailand is so close to the equator, those rockets would get an additional boost from Earth’s rotation. All kinds of companies here are exploring how they might tap into the global space economy. VegaCosmos, a startup based in Hanoi, Vietnam, is looking at ways to use satellite data for urban planning. The Electricity Generating Authority of Thailand is monitoring rainstorms from space to predict landslides. And the startup Spacemap, from Seoul, South Korea, is developing a new tool to better track satellites in orbit, which the US Space Force has invested in. It’s the space chicken that caught my eye, though, perhaps because it reflects the juxtaposition of tradition and modernity seen across Bangkok, a city of ancient temples nestled next to glittering skyscrapers. In June, astronauts on the space station were treated to this popular dish, known as pad krapow. It’s more commonly served up by street vendors, but this time it was delivered on a private mission operated by the US-based company Axiom Space. Charoen Pokphand is now using the stunt to say its chicken is good enough for NASA (sadly, I wasn’t able to taste it to weigh in). Other Southeast Asian industries could also lend expertise to future space missions. Johnson says the region could leverage its manufacturing prowess to develop better semiconductors for satellites, for example, or break into the in-space manufacturing market. I left the expo on a Thai longboat down the Chao Phraya River that weaves through Bangkok, with visions of astronauts tucking into some pad krapow in my head and imagining what might come next. Jonathan O’Callaghan is a freelance space journalist based in Bangkok who covers commercial spaceflight, astrophysics, and space exploration.