Committee

Time to first token (TTFT) is the metric teams use to pick an inference API for voice. It is also the metric that misleads them. TTFT marks when generation starts; a text-to-speech model cannot speak until a full clause arrives. Between those two points sits the difference between an agent that feels conversational and one that gets interrupted. This piece benchmarks every layer of the voice stack including LLM, speech-to-text, text-to-speech, and speech-to-speech. Why TTFT Is the Right Entry Point and the Wrong Finish Line A voice agent is a latency budget with a language model inside it. Every stage spends milliseconds the user can hear. Time to first token (TTFT) is the interval between sending an inference request and receiving the first token back. IBM’s definition frames it as the moment a system transitions from idle to visibly active. For chat, TTFT is close to the whole story. For voice, it is one term in a sum. The reason is mechanical. A text-to-speech model cannot synthesize half a word. It needs a complete clause or sentence before it produces audio. LiveKit calls the resulting metric time-to-first-sentence (TTFS), and argues in its Gemma 4 deployment post that TTFS is what users actually feel. That gives you two knobs rather than one. TTFT controls when generation starts. Tokens per second controls how fast the first sentence completes. A provider that wins one and loses the other will not feel fast. The Latency Budget: What One Voice Turn Actually Costs LiveKit’s voice agents overview breaks a turn into STT at roughly 100–200ms, LLM at 300–500ms with streaming, TTS at 100–200ms, and network at 50–150ms over WebRTC. It puts the practical end-to-end target at 700ms to 1.2s. Kwindla Hultman Kramer, co-creator of Pipecat, has advised targeting 800ms median voice-to-voice latency, with a looser 1,500ms acceptable for a proof of concept. His rough arithmetic splits that four ways at roughly 200ms each: transport and media processing, STT plus phrase endpointing, LLM inference, and TTS. Daily’s earlier work on the fastest voice bot supplies the human baseline. Typical human response time in conversation is around 500ms. Pauses beyond 800ms start to feel unnatural. Daily’s February 2026 voice-agent LLM benchmark translates that into an LLM requirement directly. Natural conversation needs voice-to-voice under 1,500ms, which works out to roughly 700ms of TTFT budget for a text-mode LLM inside a transcription-to-LLM-to-voice harness. That 700ms number is the bar to hold every provider against. How to Read a TTFT Benchmark Without Being Misled Before the tables, five methodology facts that change what the numbers mean: 1. Workload shape dominates: Artificial Analysis changed its default workload in March 2026. The site now reports 10k input token prompts rather than 1k. Longer prompts raise both TTFT and output speed. LiveKit argues this is closer to reality for voice, because production agents front-load policy, persona, escalation rules, retrieved data, and tool schemas. 2. Server location is baked in: Artificial Analysis tests from a virtual machine in Google Cloud’s us-central1-a zone. It states plainly that TTFT includes network latency and may advantage or disadvantage providers based on where they serve. 3. Reasoning tokens count: In the Artificial Analysis definition, TTFT for a reasoning model is the first reasoning token, not the first answer token. Those are separate columns. 4. Measure from the receiving side: Daily notes that model providers sometimes quote TTFT internal to their inference stacks. Daily measures from request send to first usable token off the API. 5. Runs are not repeatable: Daily is blunt about this: TTFT varies substantially between benchmark runs, and providers change inference stacks and sometimes weights without changing model names. Layer 1: LLM Time to First Token Figures below are from the Artificial Analysis API providers leaderboard, retrieved August 30, 2026. The “first chunk” column is TTFT. Workload is 10k input tokens, single prompt, median over 72 hours. Lowest measured first-chunk latency Provider Model TTFT Output speed Baseten gpt-oss-120b (high) 0.23s 266 tok/s Baseten gpt-oss-120b (low) 0.24s 271 tok/s DeepInfra Nemotron 3 Ultra 0.28s 371 tok/s Cohere North Mini Code 0.32s 104 tok/s Cohere Command A+ 0.40s 239 tok/s Baseten Inkling Small 0.42s 337 tok/s Modular Gemma 4 31B (NVFP4) 0.44s 243 tok/s Nebius GLM-5.3-Flash 0.46s 206 tok/s Fireworks Nemotron 3.5 Lightning 0.46s 501 tok/s Together AI Kimi K2.7 Code 0.47s 245 tok/s Cerebras gpt-oss-120b (high) 0.49s 1,697 tok/s The throughput trap Silicon vendors optimize for a different metric than voice agents need. Provider Model TTFT Output speed Cerebras gpt-oss-120b (high) 0.49s 1,697 tok/s Celeris Celeris-1 0.62s 1,612 tok/s Cerebras Gemma 4 31B 0.53s 1,351 tok/s Groq gpt-oss-20b (high) 0.82s 957 tok/s SambaNova gpt-oss-120b (high) 0.92s 706 tok/s Groq gpt-oss-120b (low) 0.69s 473 tok/s Inception Mercury 2 3.07s 770 tok/s Mercury 2 is the clearest illustration. It is a diffusion-based language model, and it generates 770 tokens per second. Its first chunk arrives at 3.07s. That is four times the entire LLM budget for a natural conversation. Cerebras and Groq are a different case. Their TTFT is respectable and their throughput is exceptional. For TTFS specifically, that combination is strong, because the sentence completes almost immediately after the first token lands. Frontier and proprietary endpoints Provider Model TTFT Output speed Amazon Bedrock GPT-5.6 Luna (non-reasoning) 0.59s 181 tok/s Amazon Bedrock GPT-5.6 Terra (non-reasoning) 0.72s 103 tok/s OpenAI GPT-5.6 Luna (non-reasoning) 0.74s 113 tok/s Google Gemini 3.7 Flash (low), AI Studio 0.84s 315 tok/s Anthropic Claude 4.5 Haiku (non-reasoning) 0.84s 82 tok/s Amazon Bedrock Nova Micro 0.86s 264 tok/s Google Gemini 3.5 Flash (minimal), AI Studio 0.90s 202 tok/s OpenAI GPT-5.6 Sol (non-reasoning) 1.06s 71 tok/s Note the same model on different hosts. GPT-5.6 Luna non-reasoning measures 0.59s on Amazon Bedrock and 0.74s on OpenAI’s own API. Hosting and routing matter as much as the weights. The vendor-measured outlier LiveKit publishes TTFT figures for its own inference product. Gemma 4 31B on LiveKit Inference measured 192ms, against Gemini 2.5 Flash at 911ms, GPT-5.5 at 966ms, GPT-4.1 at 1,006ms, and the same Gemma 4 31B via OpenRouter

Most robotics launches ask you to trust a demo video. Pollen Robotics, the Bordeaux robotics team at Hugging Face, is instead shipping the training loop. This week it opened pre-orders for Microduck, a 25 cm bipedal robot in which every movement — walking, sitting, kicking, roller-skating, standing back up after a fall — is a neural policy trained in a physics simulator and exported to the hardware. It costs $399. The training environments, the reward functions, the domain-randomization settings, and the sim-to-real recipe are all public on GitHub. Microduck follows Reachy Mini, which has shipped more than 10,000 units, but reverses its premise: where Reachy Mini was built to sit on a desk and interact, Microduck is built to leave the desk, fall over, and get back up. The Hardware Microduck is 25 cm tall, 14 cm wide, and under 800 g. It carries 15 motors across legs, neck, and head, plus an articulated beak that picks objects off the floor. Compute is a Rockchip RK3566 with an AI accelerator, 1 GB of RAM, and 32 GB of storage. The sensor stack is unusually complete for the price. A front camera sits behind a dedicated camera-use indicator. Two IMUs are fitted, one in the body and one in the head. Range sensing is a compact LiDAR, an 8×8 time-of-flight matrix. There are microphones and a speaker, two NFC antennas, plus Wi-Fi and Bluetooth. Power is a removable NP-F550 battery, 2600 mAh, good for about an hour. Seven trained moves ship in the box, driven by a bundled game controller before you write code: walk, sit and stand, kick, grab, roller-skate, and self-recovery. The robot does not speak. Each unit generates its own audio identity on first wake and keeps that voice permanently. How the behaviors are actually trained Policies are trained in microduck_rl, built on mjlab (MuJoCo Warp) with PPO. Pollen reports roughly one to two hours on a CUDA GPU for a usable gait at 4096 parallel environments. Without a local GPU, appending –hf-jobs runs the same command on Hugging Face Jobs. The sim-to-real work sits in the actuator model. Each servo uses the BAM M6 model of the Dynamixel XL330 — voltage control law, back-EMF, and Coulomb, Stribeck, and load-dependent friction — rather than an ideal PD controller. Per-environment randomization covers battery voltage, voltage sag under load, command delay, and friction magnitude. Backlash variants train against ±1° of gear play, 2° total, in series with each of the 14 servo joints in the RL layout. Because the real encoder sits on the output side of that play, the observations read through it. Trained policies export to ONNX with the observation normalizer baked into the graph. Pollen warns against deploying hand-converted checkpoints for exactly this reason. On the robot, a Rust runtime drives the 50 Hz control loop and the motor bus. Every policy shares a 61-dimensional actor observation: 48 proprioception dimensions plus commands for twist (3), head pose (4), and body pose (6). That shared contract is what lets walk, recover, and trick policies hot-swap mid-run. Environments that ignore a command slot zero-pad it rather than dropping it. The published registry covers 13 tasks: velocity tracking, stand-up, sit-stand, ground pick, ball kick (70 mm, 15 g ball, actor ball-blind), roulade, and five roller-skating environments. Key Takeaways $399 open-source-software biped, pre-orders open August 27, 2026, deliveries targeted before Christmas. 15 motors, camera, LiDAR, two IMUs, NFC, Wi-Fi/Bluetooth, RK3566, ~1 hour runtime. Policies train in mjlab/MuJoCo Warp with PPO, ~1–2 hours for a gait at 4096 envs. Sim-to-real hinges on a BAM actuator model plus voltage, delay, friction, and ±1° backlash randomization. Software is Apache-2.0; the mechanical and electronic design files are not open. Check out the Microduck product page, launch blog post, press kit and spec sheet, microduck runtime repo, microduck_rl training repo and announcement from Thomas Wolf. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. The post Hugging Face Unveils Microduck: A $399 Open-Source 25 cm Biped You Train with Reinforcement Learning appeared first on MarkTechPost.

Google has released Gemini Omni 1.1 Flash (gemini-omni-1.1-flash), a production update to its native multimodal video generation and editing model. The release moves Omni from a capable generator to a directable one: scene extension now reads up to 10 seconds of prior context instead of a single final frame, first and last frames can be pinned to control camera movement, drafts render in 360p at a third of 720p cost, finals upscale to 4K, and video clips can be passed as references for character consistency. Gemini Omni Flash is built on three properties Google distinguishes from prior video models: native multimodality (text, image, audio, and video processed together), conversational editing through the Interactions API, and world knowledge inherited from Gemini. Editing is stateful — you pass previous_interaction_id and the model applies your change while preserving what you did not mention, without re-uploading the prior video. Is it deployable? It is available through the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform, with Adobe, Figma Weave, GMI Cloud, and Runway already named as production users. Scene Extension is the Main Change Omni 1.1 analyzes up to 10 seconds of prior context when continuing a clip. Google states that previous models referenced only the final second. Extensions run in 10-second increments to a cumulative 40 seconds, and the model generates a 3–10 second continuation per call. Some final frames of the input are edited to make the seam continuous. The constraints are specific. Extension appends to the end of a clip only — no prepending, no mid-clip insertion. Uploaded input videos must be 10 seconds or shorter, unless you are extending a model-generated video in multi-turn. You cannot add new dialogue when extending an uploaded video where someone is speaking; spoken dialogue is supported in multi-turn extension via previous_interaction_id. Keyframes and Video References You can now supply a first and last frame and have the model generate the continuous video between them, which is the mechanism behind orbits, dolly-zooms, and seamless loops. Prompts bind media to roles with tags: <FIRST_FRAME>, <LAST_FRAME>, <IMAGE_REF_N>, and <VIDEO_REF_N>. Video references accept a maximum of three clips, up to three seconds each, and work best for likenesses. Audio inside a video reference is ignored. Reasoning across multiple videos is not supported and may degrade output. Cost Control: Draft in 360p, Ship in 4K The resolution parameter in response_format takes 360p, 720p (default), 1080p, and 4k, with the top two upscaled. Google reports 360p previews generate up to 60% faster and at a third of the cost of 720p, based on system throughput of 360p versus 720p. That makes the draft-then-upscale loop the intended production pattern: iterate cheaply, render once. Pricing, provenance, and limits Input is $1.50 per 1M tokens (text, image, video, audio). Output is $9.00 per 1M text tokens and $17.50 per 1M video tokens. Video billing runs at 5,792 tokens per second of 720p, an effective ~$0.10 per second under standard pricing. Every generated video carries SynthID watermarking — invisible to viewers, programmatically detectable for provenance. Notable gaps: no system instructions, temperature, top_p, stop sequences, or negative prompts (negatives go in the prompt text); voice editing is unsupported; audio references are unsupported; YouTube URLs cannot be used as a source. English is fully supported; other languages are unevaluated. For outputs above 4MB, use delivery=”uri” and poll the Files API until the file is ACTIVE. Google names Adobe (Firefly), Figma Weave, GMI Cloud, and Runway as customers already running Omni Flash in production. The model is also live in Google Flow for AI Plus, Pro, and Ultra subscribers, with scene extension in the Gemini app. Comparison Key Takeaways Scene extension now reads 10s of prior context, up from one final second, and stacks to 40s total. First/last frame interpolation plus <VIDEO_REF_N> tags give shot-level camera and character control. 360p drafts run up to 60% faster at a third of 720p cost; 1080p and 4K are upscaled outputs. Paid tier only — ~$0.10 per second of 720p video, no free tier, no provisioned throughput. Check out the Google blog announcement, Gemini API Omni documentation, Gemini API pricing, and Omni quickstart cookbook. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well. Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us The post Google AI Releases Gemini Omni 1.1 Flash: 40-Second Scene Extension, First/Last Frame Control, and 4K Upscaling appeared first on MarkTechPost.

In this tutorial, we build an ensemble weather forecasting workflow with NVIDIA Earth2Studio. We install the required Earth2Studio components while preserving Colab’s existing CUDA-enabled PyTorch environment, load the FCN prognostic model, and retrieve atmospheric initial conditions from GFS. We then implement a custom wind-power diagnostic that converts 10-meter wind components into turbine capacity factors, along with a variable-scaled perturbation system that applies physically appropriate noise amplitudes to different atmospheric variables while retaining an unperturbed control member. Using Earth2Studio’s low-level iterator, coordinate-mapping, batching, and Zarr APIs, we construct our own ensemble execution pipeline, write forecast and diagnostic fields to a coordinate-aware data store, and verify the forecasts against GFS analyses using latitude-weighted RMSE, fair CRPS, ensemble spread, and spread-skill ratios. Finally, we visualize ensemble uncertainty through spatial maps, geopotential-height spaghetti contours, point-based fan charts, wind-capacity-factor forecasts, and lead-time skill curves. Copy CodeCopiedUse a different Browser import importlib.util, os, subprocess, sys if importlib.util.find_spec(“earth2studio”) is None: import numpy as _np, torch as _torch cfile = os.path.join(os.getcwd(), “e2s_constraints.txt”) with open(cfile, “w”) as f: f.write(f”torch=={_torch.__version__.split(‘+’)[0]}n”) f.write(f”numpy=={_np.__version__}n”) env = {**os.environ, “PIP_CONSTRAINT”: cfile} subprocess.check_call( [sys.executable, “-m”, “pip”, “install”, “-q”, “earth2studio[fcn,data,perturbation,statistics]”], env=env) print(“n>>> Install done. If the imports below fail: Runtime > Restart session, re-run.n”) os.environ.setdefault(“EARTH2STUDIO_CACHE”, “/content/e2s_cache”) os.makedirs(“outputs”, exist_ok=True) from collections import OrderedDict from datetime import datetime, timedelta, timezone from tqdm.auto import tqdm from earth2studio.data import GFS, fetch_data from earth2studio.io import ZarrBackend from earth2studio.models.batch import batch_coords, batch_func from earth2studio.models.px import FCN from earth2studio.statistics import rmse from earth2studio.utils import handshake_coords, handshake_dim from earth2studio.utils.coords import map_coords from earth2studio.utils.time import to_time_array from earth2studio.utils.type import CoordSystem if DEVICE.type == “cpu”: print(“!! No GPU detected — this will be very slow. Runtime > Change runtime type > T4 GPU”) NENSEMBLE = 8 BATCH_SIZE = 2 NSTEPS = 8 SAVE_VARS = [“t2m”, “z500”, “u10m”, “v10m”, “tcwv”] VERIFY_VARS = [“t2m”, “z500”, “u10m”] INIT = (datetime.now(timezone.utc) – timedelta(days=7)).replace() INIT_STR = INIT.strftime(“%Y-%m-%dT%H:%M:%S”) POI = (“New Delhi”, 28.61, 77.21) print(f”Initialization: {INIT_STR} | device: {DEVICE}”) We install Earth2Studio while preserving Colab’s existing CUDA-enabled PyTorch and NumPy environment through package constraints. We configure the model cache, import the forecasting, data, statistics, plotting, and coordinate-management utilities, and detect the available compute device. We also define the ensemble size, batch size, forecast duration, saved variables, verification variables, initialization time, and New Delhi point of interest. Copy CodeCopiedUse a different Browser class WindPowerCF(torch.nn.Module): “””Turbine capacity factor [0,1] from 10 m winds via power-law shear + power curve.””” def __init__(self, lat, lon, hub=100.0, alpha=0.143, cut_in=3.0, rated=12.0, cut_out=25.0): super().__init__() self.lat, self.lon = lat, lon self.hub, self.alpha = hub, alpha self.cut_in, self.rated, self.cut_out = cut_in, rated, cut_out def input_coords(self) -> CoordSystem: return OrderedDict({ “batch”: np.empty(0), “variable”: np.array([“u10m”, “v10m”]), “lat”: self.lat, “lon”: self.lon, }) @batch_coords() def output_coords(self, input_coords: CoordSystem) -> CoordSystem: target = self.input_coords() for i, (key, _) in enumerate(target.items()): if key != “batch”: handshake_dim(input_coords, key, i) handshake_coords(input_coords, target, key) oc = OrderedDict({ “batch”: np.empty(0), “variable”: np.array([“wind_cf”]), “lat”: self.lat, “lon”: self.lon, }) oc[“batch”] = input_coords[“batch”] return oc @batch_func() def __call__(self, x: torch.Tensor, coords: CoordSystem): oc = self.output_coords(coords) u, v = x[…, 0:1, :, :], x[…, 1:2, :, :] ws10 = torch.sqrt(u * u + v * v) ws = ws10 * (self.hub / 10.0) ** self.alpha ramp = (ws ** 3 – self.cut_in ** 3) / (self.rated ** 3 – self.cut_in ** 3) cf = torch.zeros_like(ws) cf = torch.where((ws >= self.cut_in) & (ws < self.rated), ramp.clamp(0, 1), cf) cf = torch.where((ws >= self.rated) & (ws <= self.cut_out), torch.ones_like(cf), cf) return cf, oc class VariableScaledNoise: “””Spatially correlated noise with per-variable amplitudes + control member.””” def __init__(self, amplitudes: dict, default: float = 0.0, control_member: bool = True): self.amplitudes, self.default, self.control = amplitudes, default, control_member try: from earth2studio.perturbation import SphericalGaussian self.sampler, self.kind = SphericalGaussian(noise_amplitude=1.0), “SphericalGaussian” except Exception: from earth2studio.perturbation import Brown self.sampler, self.kind = Brown(noise_amplitude=1.0), “Brown” def __call__(self, x: torch.Tensor, coords: CoordSystem): noise, _ = self.sampler(torch.zeros_like(x), coords) vax = list(coords).index(“variable”) amps = torch.tensor([self.amplitudes.get(str(v), self.default) for v in coords[“variable”]], device=x.device, dtype=x.dtype) shape = [1] * x.ndim; shape[vax] = amps.numel() pert = noise * amps.reshape(shape) if self.control and “ensemble” in coords: eax = list(coords).index(“ensemble”) mask = torch.tensor((np.asarray(coords[“ensemble”]) != 0).astype(np.float32), device=x.device, dtype=x.dtype) mshape = [1] * x.ndim; mshape[eax] = mask.numel() pert = pert * mask.reshape(mshape) return x + pert, coords We create a custom diagnostic model that converts 10-meter wind components into hub-height wind speed and turbine capacity factor. We validate coordinate compatibility through Earth2Studio’s handshake utilities and support batched inputs with the provided decorators. We also implement variable-specific spatial perturbations that retain member zero as an unperturbed control forecast. Copy CodeCopiedUse a different Browser def write_vars(io, x, coords, names): “””Write selected channels of a (…, variable, lat, lon) tensor to the IO backend.””” vax = list(coords).index(“variable”) sub = OrderedDict((k, v) for k, v in coords.items() if k != “variable”) for name in names: hit = np.where(np.asarray(coords[“variable”]) == name)[0] if hit.size: io.write(x.select(vax, int(hit[0])).cpu(), sub, name) def run_ensemble(time, nsteps, nensemble, batch_size, prognostic, diagnostic, perturbation, data, io, save_vars, device): time = to_time_array(time) ic = prognostic.input_coords() x0, c0 = fetch_data(source=data, time=time, lead_time=ic[“lead_time”], variable=ic[“variable”], device=device) print(f”Initial condition tensor: {tuple(x0.shape)} dims={list(c0)}”) oc = prognostic.output_coords(ic) dt = oc[“lead_time”] prog_vars = [v for v in save_vars if v in set(map(str, oc[“variable”]))] total = OrderedDict({ “ensemble”: np.arange(nensemble), “time”: time, “lead_time”: np.asarray([dt * i for i in range(nsteps + 1)]).flatten(), “lat”: oc[“lat”], “lon”: oc[“lon”], }) io.add_array(total, prog_vars + [“wind_cf”]) dx_target = OrderedDict((k, v) for k, v in diagnostic.input_coords().items() if k != “batch”) nbatch = int(np.ceil(nensemble / batch_size)) with torch.inference_mode(): for b in tqdm(range(nbatch), desc=”ensemble batches”): lo = b * batch_size n = min(batch_size, nensemble – lo) x = x0.unsqueeze(0).repeat(n, *([1] * x0.ndim)) coords = OrderedDict({“ensemble”: np.arange(lo, lo + n), **c0}) x, coords = perturbation(x, coords) x, coords = map_coords(x, coords, ic) for step, (xs, cs) in enumerate(prognostic.create_iterator(x, coords)): write_vars(io, xs, cs, prog_vars) xw, cw = map_coords(xs, cs, dx_target) xw, cw = diagnostic(xw, cw) write_vars(io, xw, cw, [“wind_cf”]) if step >= nsteps: break torch.cuda.empty_cache() if device.type == “cuda” else None return io model = FCN.load_model(FCN.load_default_package()).to(DEVICE) grid = model.output_coords(model.input_coords()) LAT, LON = grid[“lat”], grid[“lon”] diagnostic = WindPowerCF(LAT, LON).to(DEVICE) pert = VariableScaledNoise( amplitudes={“t2m”: 0.20, “t850”: 0.20, “z500”: 40.0, “z850”: 25.0, “u10m”: 0.25, “v10m”: 0.25,