Uncategorized

Biomolecular structure prediction has shifted from single-target runs to proteome-scale worklists. The bottleneck is no longer whether a model can fold a protein. It is how fast an entire queue of independent targets moves through parsing, featurization, GPU inference, and output writing. NVIDIA’s new technical deep dive walks through BioNeMo Inference Runtime (BioIR), a Python library that accelerates supported structure-prediction models on NVIDIA GPUs while keeping the standard PyTorch workflow. BioIR has already run at production scale. It powered the recent expansion of the AlphaFold Database, generating protein-complex structures across 4,777 proteomes, about 31 million candidate complexes, with 1.81 million released as high-confidence predictions. Is it deployable? Yes. BioIR is available now as an open GitHub repository with a wheel containing precompiled CUBINs. Runtime use needs Python 3.12+, a compatible NVIDIA GPU and driver, a staged model checkpoint, and per-chain A3M MSAs. It does not require nvcc, CUDA source, CMake, or the CUDA toolkit. What is BioIR BioIR targets the operations that general-purpose inference stacks do not fully optimize. These include Pairformer and Evoformer stacks, triangle operations, pairwise attention, diffusion transformers, and atom-level modules. Models stay ordinary torch.nn.Module objects. There is no engine build, export step, or separate artifact between a checkpoint and a forward pass. There are 2 ways to use it. The end-to-end processor moves an InputRequest through parsing, tokenization, feature generation, GPU inference, and PDB or mmCIF writing. Direct PyTorch integration lets developers construct a supported model or reuse selected optimized modules inside custom code. The tutorial demonstrates the processor path with Boltz-2 (model_source=”boltz-2″). Each protein chain requires an A3M MSA. Paired or unpaired MSAs are accepted for inputs with multiple non-identical protein chains. Templates can be supplied manually because BioIR does not run HHsearch or HMMsearch. The processor supports ligand structure prediction but not ligand-affinity prediction. Three Layers of Acceleration BioIR optimizes at 3 distinct layers, each targeting a different bottleneck: Kernel selection: Supported operations pick compatible BioIR custom, cuEquivariance, or PyTorch fallback implementations based on model configuration, GPU, data type, and tensor shape. Module optimization: A separate optimize() mechanism enables CUDA Graph capture for compatible modules, cutting launch overhead. Pipeline scaling: A Ray executor places 1 complete model replica on each visible GPU in a node and distributes independent inputs among them. CPU stages (parsing, featurization, writing) overlap with GPU folding. Note: Ray does not split a single forward pass across GPUs. Replica mode scales worklists, not individual targets. Per the support matrix, context-parallel folding is planned but not yet available. The capacity rule is simple: engine_stage.compute x num_gpus must not exceed visible GPUs. At the model-forward level, NVIDIA’s early benchmarking reports geometric-mean speedups over an OSS torch.compile baseline of 1.55x (OpenFold3), 1.78x (Boltz2), and 2.56x (OpenFold2 monomer) on H100. H200 numbers are similar at 1.54x, 1.75x, and 2.61x. These were measured across 17 inputs spanning 29 to 1,734 residues. The Benchmark: 1,000 Human Dimers on 8xH100 To quantify end-to-end delivery, NVIDIA team ran a matched benchmark on 1,000 human dimer targets with combined sequence lengths below 2,800 residues. The comparison pitted BioIR-accelerated Boltz-2 against a torch-compiled open-source Boltz-2 implementation on 8xH100 80GB GPUs. Both used identical targets, staged MSAs, inference recipe (3 recycles, 200 sampling steps, 5 diffusion samples), and GPU configuration. The results: BioIR completed all 1,000 targets and delivered 58.5K successfully folded residues per allocated GPU-hour. The public implementation delivered 20.2K residues per GPU-hour and ran out of memory on 29 targets. Net result: a 2.90x improvement in residue-normalized throughput. These numbers are folding-stage measurements specific to this dataset and hardware. They exclude MSA generation, preprocessing CPU allocations, storage, data transfer, and retries. The blog explicitly warns against generalizing them to all BioIR-supported models or datasets. Energy at One Million Targets Extrapolating the benchmark linearly to 1 million comparable targets, BioIR is estimated to need 11 MWh versus 35 MWh for the public implementation using 8-GPU TDP equivalents. Using full-node maximum-power equivalents, the estimate is 21 MWh versus 64 MWh. These are rated-power, folding-only estimates for IT equipment, not metered measurements, and exclude data center overhead such as PUE. Still, a 23 to 43 MWh saving per million targets is a material number for proteome-scale campaigns. Key Takeaways BioIR accelerates Boltz-2, OpenFold2, and OpenFold3 inference on NVIDIA GPUs while staying in plain PyTorch. Matched 8xH100 benchmark: 58.5K vs 20.2K folded residues per GPU-hour, a 2.90x throughput gain. Ray replica mode scales independent worklists; it never splits 1 forward pass across GPUs. Estimated energy for 1M targets drops from 35 MWh to 11 MWh at 8-GPU TDP equivalents. Already proven at scale: 31M candidate complexes generated for the AlphaFold Database expansion. Check out the technical blog, GitHub repo, docs, and the BioNeMo Agent Toolkit for agentic orchestration. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post NVIDIA Details BioNeMo Inference Runtime (BioIR): 2.90x Higher Boltz-2 Folding Throughput and 58.5K Residues per GPU-Hour on 8xH100 appeared first on MarkTechPost.

OpenAI has released the Agents API in public beta. It gives developers the same harness and infrastructure that run Codex. OpenAI hosts and maintains the harness. Developers run the agent’s compute in an OpenAI-managed sandbox, their own infrastructure, or a partner sandbox. Is it deployable? Yes. It is live for all developers in public beta. Data stays US-only, and Zero Data Retention is unsupported. What OpenAI Shipped The Agents API is a managed service built on the open-source Codex harness. OpenAI team states scaling Codex and ChatGPT for Work showed what long-running agents need. They need a harness that manages context, uses tools efficiently, and coordinates subagents. They also need infrastructure that keeps them running reliably for days. The official docs organize the API around 4 concepts: Agent: the model, instructions, tools, and MCP servers available to it. Environment: an optional sandbox where the agent accesses files, loads skills, and runs commands. Session: a durable agent instance that works on tasks and responds to input. Events and items: the inputs sent to the agent and the output it produces. A session runs in 4 steps. You create it and give it a task. Then you follow progress through streaming or webhooks. Finally, you continue with a new task or steer the current turn. One API Call OpenAI’s announcement shows an incident-investigation agent created in a single call: Copy CodeCopiedUse a different Browser import OpenAI from “openai”; const client = new OpenAI(); const session = await client.beta.agents.sessions.create({ agent: { model: “gpt-6-astra”, tools: [ { type: “mcp”, server_label: “observability”, transport: { type: “http”, server_url: “https://observability.example.com/mcp”, }, }, ], multi_agent: { enabled: true, max_concurrent_subagents: 3 }, }, vault_ids: [“vault_YOUR_VAULT_ID”], environment: { type: “openai_hosted”, capability_directories: [“/workspace/capabilities/skills”], }, input: “Investigate service-api’s elevated 5xx rate over the last 30 minutes. ” + “Delegate deployment, error, and dependency analysis to subagents. ” + “Save findings, evidence, and recommended mitigation in /workspace/outputs.”, }); The quickstart covers API key permissions and SDK setup. Where the Agent Runs Environment choice is the main architectural decision. The Agents API supports 3 sandbox options, and it can also run without a sandbox. OpenAI-hosted sandbox: uses the sandboxing infrastructure behind Codex and ChatGPT. You can configure it with files, packages, skills, and plugins. Self-hosted: you run codex exec-server inside your environment. It registers with a restricted key and connects over WebSocket. All connections are outbound. Partner sandboxes: Blaxel, Cloudflare, Daytona, DigitalOcean, E2B, Modal, Oracle, Runloop, and Vercel have first-class integrations. What the Harness Handles OpenAI maintains the harness alongside its models, with versioned access at each model launch. Long sessions: The API automatically compacts earlier context as a session nears its limit. Developers do not write their own compaction logic. Efficient tool use: Tool search loads tool definitions only when needed. This reduces token usage and cost while preserving the model’s cache. Programmatic tool calling lets agents run calls in parallel and chain operations. Agents filter or combine results in code, so only relevant data returns into context. Supported tools include MCP, custom functions, and built-in tools like web search. Subagents: With multi-agent support, the main agent splits complex tasks into independent pieces. Each subagent keeps its own context. The main agent coordinates them and combines the results. Agents API vs Agents SDK vs Responses API OpenAI’s runtime comparison positions the 3 options this way: Agents API Agents SDK Responses API Where the agent runs OpenAI runs a managed Codex harness Inside your application Your application, with optional hosted orchestration Integration effort Low Medium High State between tasks Saved session configuration, turns, and items Your storage and SDK sessions Manual history, response chaining, or Conversations Execution environment OpenAI-hosted, self-hosted, or no sandbox Your runtime and sandbox providers Your own environment Early Customer Results OpenAI published these customer-reported numbers. They are vendor-supplied, not independent benchmarks. Ciridae: evaluation score rose from 0.71 to 0.85, with a 4x latency reduction on subagent flows. SafetyKit: 60% lower cost per case after migrating its case review workflow. Hypha: 86% fewer failed agent responses after separating the harness from the sandbox. Nash.ai: runs thousands of long-running agents across global logistics networks. Key Takeaways OpenAI’s Agents API exposes the managed Codex harness as a public beta API. Agents run in OpenAI-hosted, self-hosted, or 9 partner sandboxes. Compaction, tool search, programmatic tool calling, and subagents come built in. There is no extra fee; you pay for tokens, tools, and container time. US-only data residency and no ZDR limit regulated workloads for now. Check out the Technical details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post OpenAI Launches the Agents API in Public Beta, Putting the Codex Harness Behind One API Call appeared first on MarkTechPost.

Production LLM applications rarely receive a question nobody has asked before. Support assistants and RAG pipelines field the same intents thousands of times a day, each phrased differently, and most stacks treat every phrasing as a fresh, fully billed request. Redis LangCache is a fully managed semantic caching service that sits between the application and the model, matches incoming prompts against previously answered ones by meaning rather than exact text, and returns the stored response when a close enough match exists. Redis reports API cost savings of up to 90% and cache-hit responses up to 15x faster than re-querying the model. Is it deployable? Yes. LangCache is available today as a public preview on Redis Cloud, accessed through a REST API with Python and JavaScript SDKs, and Redis notes that features and behavior may change during the preview. The Problem: Paraphrases Are Still Full LLM Calls Consider three requests to a customer-support assistant: “Can I get a refund after buying the monthly plan?” “Is the monthly subscription refundable?” “Can I cancel the plan and get my money back?” The wording differs, but the question and answer are identical. Without a semantic cache, each version triggers a complete generation: input tokens processed, output tokens decoded, user waiting. Prefix caching only removes part of that cost. When requests share a system prompt or context, the engine reuses the KV states computed for that prefix, but the request still reaches the LLM, new tokens still get processed, and the full answer still gets decoded. A prefix-cache hit is a cheaper generation call, not an avoided one. How LangCache Works LangCache moves the cache outside the model and stores the generated response itself. The architecture is a two-call loop: Before invoking the model, the app sends the prompt to POST /v1/caches/{cacheId}/entries/search. LangCache generates an embedding for the prompt and runs a vector search over stored entries. If a semantically similar entry clears the configured similarity threshold, the cached response is returned and no LLM call occurs. On a miss, the app calls its chosen LLM as usual, then stores the prompt and new response through POST /v1/caches/{cacheId}/entries for future matches. Embedding generation is handled by the service, with default models or bring-your-own. Cache behavior is controlled through similarity thresholds, TTLs, and eviction policies, plus adaptive controls that tune precision and recall. Built on Redis’s vector database and exposed as a REST API, it works with any LLM provider and language. Hit rates and savings are monitored from the Redis Cloud console. What a Cache Hit Actually Saves A cache hit removes the input tokens, the output tokens, and the decoding latency of an additional model call. In a demo run comparing both paths on a paraphrased question, direct inference took 2.232 seconds and consumed 514 input tokens plus 250 output tokens. LangCache returned the earlier response in 0.37 seconds with zero LLM input or output tokens, roughly 6x faster in that run. The Redis documentation is careful about how savings accrue. On a cached response you do not pay for output tokens, while input token costs are typically offset by embedding and storage costs. The suggested estimate is: Est. monthly savings = (Monthly output token costs) x (Cache hit rate) With $200 of monthly LLM spend, 60% of it on output tokens, and a 50% hit rate, that works out to $60 saved per month. Redis also publishes a savings calculator for annual estimates. Redis’s public preview announcement cited up to 15x faster responses on cache hits and up to 70% lower token usage, while the current product page states savings of up to 90%. Customer Mangoes.ai reports a 70% hit rate on its patient-care voice app, cutting LLM spend by 70% with 4x faster responses. The actual result depends on how much safe repetition exists in the traffic. Where Semantic Caching Needs Care Deciding which questions can safely share an answer is a production concern, not a configuration detail. A threshold set too low returns a refund policy to a customer asking about upgrades. Set too high, nearly every paraphrase goes back to the model and the cache stops paying for itself. Production setups need well-tuned thresholds, expiration policies so stale answers age out, data isolation between tenants, and monitoring for incorrect matches. LangCache covers these with access scopes, custom filtering, TTL and eviction controls, and monitoring through Redis Cloud. Data stays on the customer’s Redis servers, and Redis states it does not access that data or use it to train models. Key Takeaways Prefix caching cuts prompt-processing cost; semantic caching eliminates the LLM call entirely on a hit. LangCache is a two-call REST integration: search before the model, store after it. Savings come mainly from avoided output tokens; the docs give the formula output cost x hit rate. Redis claims up to 90% cost savings and up to 15x faster cache hits; a demo run showed 6x. Thresholds, TTLs, isolation, and false-match monitoring decide whether a semantic cache is safe. Check out redis.io/langcache and follow the API and SDK examples. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Meet Redis LangCache: A Managed Semantic Cache That Cuts LLM API Costs by Up to 90% and Returns Cache Hits Up to 15x Faster appeared first on MarkTechPost.

Battery installations hit a new record in the US in the second quarter of 2026. In total, 20.2 gigawatt-hours of new capacity came online, according to a new report. That’s enough to supply the daily electricity needs of about 700,000 homes. The surge is putting the country on a trajectory to see 71 gigawatt-hours of batteries installed in 2026, a 20% increase over last year. This growth is being driven by a combination of cheaper batteries and an urgent need for more energy storage capacity as renewables such as solar and onshore wind power are added to the grid.  Massive, utility-scale systems are leading the way; they’re responsible for most of the record-setting quarter. Seven new gigascale battery installations (those with a capacity of over one gigawatt-hour) came online during the three-month stretch, according to the report, published by Benchmark Mineral Intelligence and the Solar Energy Industries Association. “It really came down to a handful of big projects,” says Shan Tomouk, energy storage and energy lead for Benchmark Mineral Intelligence. But there was also growth in the category of so-called behind-the-meter batteries, which include both residential and industrial battery storage systems. These projects, generally smaller than utility-scale installations, are typically owned and operated by homeowners or businesses rather than utilities or power providers.  In the behind-the-meter category, data centers led the way, making up about three-quarters of new batteries in the commercial sector. But residential batteries saw a sharp slowdown. These systems are often installed in homes to store power from solar panels or serve as a backup source in case of a blackout. Home installations are projected to drop by 16% in 2026 compared with last year, according to the report. That drop happened largely because a tax credit that helped subsidize home battery systems ended in 2025, Tomouk says. Home installations should recover by the end of the decade, he adds. And tax credits for nonresidential batteries have largely survived. Overall, batteries are a bright spot in energy right now. “This is one of the strong sectors in the US,” says Isshu Kikuma, an energy storage analyst at BloombergNEF, an energy consultancy. As the battery market continues to grow, one major trend to keep an eye on is a move toward US-made technology. Today, nearly all the systems coming online use cells made in China, though some are put together into complete energy storage systems in the US. Tariffs were already pushing the US energy storage industry toward domestic production. And beginning this year, energy storage tax credits required projects to limit their reliance on batteries imported from China. There’s a lot of manufacturing capacity set to come online in the US, though these factories probably won’t be able to meet demand until at least 2030 or so, Tomouk says, so prices could tick up.

Google has open-sourced Mantis, a stack-agnostic toolkit of security review skills that lets an AI coding agent run the whole vulnerability lifecycle. It finds a suspected flaw, strips the false positives, reproduces the bug inside a sandbox, writes a minimal patch, re-attacks that patch, and scores the residual risk. Mantis is not a scanner you aim at a repository and walk away from. It is a set of slash commands your existing coding agent loads, plus a strict set of rules about where that agent is allowed to execute code. Is it deployable? Yes for local and internal evaluation, not yet for production. You can clone it today and run it with Gemini CLI, Antigravity CLI, the Google ADK, or any comparable agent framework. The pipeline Mantis publishes each stage as a separate skill directory, invoked as a slash command and chained sequentially. A supervisor skill, /mantis-meta-agent, can drive the whole loop in a long-lived session. The early stages learn the target: /mantis-history mines version control history for past security fixes, /mantis-summarize writes the directory maps, /mantis-architecture builds a Markdown knowledge base, /mantis-threat-model derives trust boundaries, and /mantis-plan produces a targeted roadmap. The middle stages find and filter: /mantis-researcher sweeps files against the plan, then /mantis-dedupe, /mantis-review and /mantis-critic collapse duplicates, apply negative rules, and drop issues that cannot occur in a release build. The late stages prove and fix: /mantis-reproduce executes payloads in gVisor or a VM with networking disabled, /mantis-chain assembles multi-step exploit chains from individually confirmed findings, /mantis-patch applies and verifies the fix, /mantis-calibrate assigns a risk score from 1 to 10, /mantis-reflect writes learnings back for the next pass, and /mantis-report produces the human-readable review packet. A newer skill, /mantis-advise, inverts the flow. It queries the accumulated threat model, past bug lineages and verified patch patterns before you write code, so the same class of bug does not land twice. But why? Most agentic security tooling stops at generating findings. Mantis is interesting because it treats the reproducer and the re-attack as the trust boundary, and because it publishes the inter-stage contracts so teams can wrap the skills in a deterministic harness instead of trusting an LLM to orchestrate shell commands. Key Takeaways Mantis is a modular skills toolkit for coding agents, not a standalone scanner or a supported Google product. Its differentiator is grounding: sandboxed reproduction and patch re-attack, not model confidence. A hierarchical summary tree cuts token overhead by over 85 percent, per Google. Google cites sub-7 percent true-positive rates for naive AI code scanning as the problem Mantis targets. Deployable locally under Apache 2.0 but not recommended yet for production. Check out the google/mantis on GitHub, Agent Reference Guide, Cloud CISO Perspectives, and Getting started with Mantis. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Google Open-Sources Mantis: A Modular Skills Toolkit That Lets Coding Agents Find, Reproduce and Patch Vulnerabilities appeared first on MarkTechPost.