{"id":95559,"date":"2026-06-06T17:38:16","date_gmt":"2026-06-06T17:38:16","guid":{"rendered":"https:\/\/youzum.net\/nvidia-releases-nemotron-3-5-asr-a-600m-parameter-cache-aware-streaming-model-transcribing-40-language-locales-in-real-time\/"},"modified":"2026-06-06T17:38:16","modified_gmt":"2026-06-06T17:38:16","slug":"nvidia-releases-nemotron-3-5-asr-a-600m-parameter-cache-aware-streaming-model-transcribing-40-language-locales-in-real-time","status":"publish","type":"post","link":"https:\/\/youzum.net\/zh\/nvidia-releases-nemotron-3-5-asr-a-600m-parameter-cache-aware-streaming-model-transcribing-40-language-locales-in-real-time\/","title":{"rendered":"NVIDIA Releases Nemotron 3.5 ASR: A 600M-Parameter Cache-Aware Streaming Model Transcribing 40 Language-Locales in Real Time"},"content":{"rendered":"

NVIDIA\u2019s Nemotron Speech team has released Nemotron 3.5 ASR<\/a>. It is a 600M-parameter streaming Automatic Speech Recognition (ASR) model. A single checkpoint transcribes 40 language-locales in real time. Punctuation and capitalization are built in natively. The model ships as open weights on Hugging Face. The license is OpenMDW-1.1. The architecture is a Cache-Aware FastConformer-RNNT.<\/p>\n

What is Nemotron 3.5 ASR<\/strong><\/h2>\n

Nemotron 3.5 ASR extends nvidia\/nemotron-speech-streaming-en-0.6b<\/code> to many languages. It adds prompt-based language-ID conditioning to the base model. That lets one 600M-parameter checkpoint cover 40 language-locales. No per-language model or model-swapping is required.<\/p>\n

The model targets two workloads. The first is low-latency streaming for live audio. The second is high-throughput batch transcription. Output is production-ready text with proper casing and punctuation. No separate punctuation-restoration step is needed.<\/p>\n

\n
\"\"
Image source: https:\/\/huggingface.co\/nvidia\/nemotron-3.5-asr-streaming-0.6b<\/figcaption><\/figure>\n<\/div>\n

How Cache-Aware FastConformer-RNNT Works<\/strong><\/h2>\n

The model has two main pieces. The first is a Cache-Aware FastConformer encoder with 24 layers. FastConformer is an efficient evolution of the Conformer architecture. It uses linearly scalable attention. The second piece is an RNNT (Recurrent Neural Network Transducer) decoder. RNNT emits text frame by frame as audio streams in.<\/p>\n

The \u201ccache-aware\u201d design is the efficiency lever. Buffered streaming re-processes overlapping audio windows at every step. That repeats the same work and adds delay. This model caches encoder self-attention and convolution activations instead. It reuses those cached states as new audio arrives. So each audio frame is processed exactly once, with no overlap. Compute and end-to-end latency both drop, without an accuracy penalty.<\/p>\n

The Latency Knob: att_context_size<\/code><\/strong><\/h2>\n

One inference setting controls the latency-accuracy tradeoff. It is the attention context size, att_context_size<\/code>. Smaller context emits text sooner but sees less future audio. Larger context raises accuracy at higher latency.<\/p>\n

The same checkpoint covers the full range. Settings map to chunk sizes of 80ms, 160ms, 320ms, 560ms, and 1.12s. For example, [56,0]<\/code> gives an 80ms ultra-low-latency mode. The [56,13]<\/code> setting gives 1.12s for highest accuracy. Teams pick the operating point at inference time, with no retraining.<\/p>\n

Language Detection and Coverage<\/strong><\/h2>\n

The 40 language-locales include English, Spanish, German, and French variants. They also cover Arabic, Japanese, Korean, Mandarin, Hindi, and Thai. Several other European and Nordic languages are included too.<\/p>\n

Language conditioning works two ways. Setting target_lang<\/code> to a known locale usually gives the best accuracy. Setting target_lang=auto<\/code> lets the model detect the language itself. In auto mode, it emits a language tag after terminal punctuation. One deployment can then transcribe mixed-language traffic. No separate language-ID component is required.<\/p>\n

Comparison<\/strong><\/h2>\n
\n\n\n\n\n\n\n\n\n\n\n
Product<\/th>\nCompany<\/th>\nAccess<\/th>\nNative streaming<\/th>\nLanguage coverage<\/th>\nReported latency<\/th>\nPricing model<\/th>\n<\/tr>\n<\/thead>\n
Nemotron 3.5 ASR<\/strong><\/td>\nNVIDIA<\/td>\nOpen weights (OpenMDW-1.1), self-host; hosted on DeepInfra<\/td>\nYes \u2014 cache-aware FastConformer-RNNT<\/td>\n40 language-locales<\/td>\n80ms\u20131.12s, configurable at inference<\/td>\nFree to self-host; usage-based via host<\/td>\n<\/tr>\n
Whisper large-v3<\/td>\nOpenAI<\/td>\nOpen weights (MIT), self-host; API<\/td>\nNo \u2014 offline\/batch<\/td>\n~99 languages <\/td>\nNot streaming-native<\/td>\nSelf-host free; API ~$0.006\/min (batch) <\/td>\n<\/tr>\n
Nova-3<\/td>\nDeepgram<\/td>\nClosed API; on-premise\/self-host (enterprise)<\/td>\nYes \u2014 streaming + batch<\/td>\nMultilingual; +10 monolingual languages added Jan 2026 <\/td>\nLow-latency streaming (reported sub-300ms)<\/td>\n~$0.0077\/min (Nova-3 Monolingual, PAYG) <\/td>\n<\/tr>\n
Universal-3 Pro Streaming<\/td>\nAssemblyAI<\/td>\nClosed API (EU endpoint available)<\/td>\nYes<\/td>\n6 languages: English, Spanish, French, German, Italian, Portuguese <\/td>\nSub-300ms (official); first partial ~750ms <\/td>\nUsage-based (PAYG)<\/td>\n<\/tr>\n
Scribe v2 Realtime<\/td>\nElevenLabs<\/td>\nClosed API<\/td>\nYes<\/td>\n90+ languages (99 per ElevenLabs) <\/td>\n~150ms (p50) <\/td>\n~$0.28\/hour <\/td>\n<\/tr>\n
Ursa \/ streaming<\/td>\nSpeechmatics<\/td>\nAPI + on-premise + edge<\/td>\nYes \u2014 streaming + batch<\/td>\n50+ languages with automatic identification <\/td>\nUltra-low latency (positioned)<\/td>\nEnterprise\/usage<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
\n

Fine-Tuning Results<\/strong><\/h2>\n

Because the weights are open, teams can fine-tune for a language, domain, or accent. NVIDIA published a worked example on Greek and Bulgarian. It fine-tuned the base checkpoint with the same Cache-Aware FastConformer-RNNT recipe. Each clip carried a target_lang<\/code> tag for language conditioning. Training data came from public corpora, including Granary, Common Voice, and FLEURS.<\/p>\n

Results were measured as WER on held-out FLEURS, at the 80ms setting. Greek WER fell from 35 to 24, a 32% relative improvement. Bulgarian fell from 22 to 15, a 31% relative improvement. These are raw WER percentages at the lowest-latency streaming mode. NVIDIA notes that evaluating at deployment latency, on held-out data, gives honest numbers.<\/p>\n

Strengths and Considerations<\/strong><\/h2>\n

Strengths:<\/strong><\/h4>\n